Searching for audio by sketching mental images of sound : a brave new idea for audio retrieval in creative music production

Peter Knees, H.K.G. Andersen

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

10 Citaten (Scopus)


We propose a new paradigmfor searching for sound by allow- ing users to graphically sketch their mental representation of sound as query. By conducting interviews with professional music producers and creators, we find that existing, text- based indexing and retrieval methods based on file names and tags to search for sound material in large collections (e.g., sample databases) do not reflect their mental concepts, which are often rooted in the visual domain and hence are far from their actual needs, work practices, and intuition. As a consequence, when creating new music on the basis of existing sounds, the process of finding these sounds is cum- bersome and breaks their work flow due to being forced to resort to browsing the collection. Prior work on organizing sound repositories aiming at bridging this conceptual gap between sound and vision builds upon psychological findings (often alluding to synaesthetic phenomena) or makes use of ad-hoc, technology-driven mappings. These methods fore- most aim at visualizing the contents of collections or individ- ual sounds and, again, facilitating browsing therein. For the purpose of indexing and querying, such methods have not been applied yet. We argue that the development of a search system that allows for visual queries to audio collections is desired by users and should inform and drive future research in audio retrieval. To explore this notion, we test the idea of a sketch interface with music producers in a semi-structured interview process by making use of a non-functional proto- type. Based on the outcomes of this study, we propose a conceptual software prototype for visually querying sound repositories using image manipulation metaphors.
Originele taal-2Engels
TitelICMR'16 :proceedings of the 2016 ACM International Conference on Multimedia Retrieval : June 6-9, 2016, New York, NY, USA
Plaats van productieNew York
UitgeverijAssociation for Computing Machinery, Inc
Aantal pagina's8
ISBN van geprinte versie978-1-4503-4359-6
StatusGepubliceerd - 2016
Extern gepubliceerdJa
Evenement2016 ACM on International Conference on Multimedia Retrieval (ICMR 2016) - New York, Verenigde Staten van Amerika
Duur: 6 jun 20169 jun 2016


Congres2016 ACM on International Conference on Multimedia Retrieval (ICMR 2016)
Verkorte titelICMR'16
LandVerenigde Staten van Amerika
StadNew York
Internet adres

Vingerafdruk Duik in de onderzoeksthema's van 'Searching for audio by sketching mental images of sound : a brave new idea for audio retrieval in creative music production'. Samen vormen ze een unieke vingerafdruk.

Citeer dit