Presented by: Shailesh Deshpande (shailesh@vt)

1. Music Information Retrieval-or-how to search for (and maybefind) music and do away with incipitsMichael FingerhutMultimedia Library andEngineering BureauIRCAM – Centre PompidouIAML - IASA 2004 Congress, OsloIRCAM - Institut de Recherche et Coordination Acoustique/MusiqueIAML- International Association of Music LibrariesIASA – International association of sound archives Presented by: Shailesh Deshpande (shailesh@vt.edu) 06/28/2009

2. Agenda • Introduction • Why MIR? • Take 1: multi-disciplinary domain • Take 2: schematic • Take 3: typology • Challenges • IRCAM cataloging tool

3. Introduction • Music information retrieval (MIR) is the interdisciplinary science of retrieving information from music • Paper presents three views of this domain • Challenges • What is an incipit? • First few words or opening line of a book. In music – first few notes of a composition.

4. Why MIR? • Storage => increased availability of musical content in digital form (locally) • CD’s, DVD’s, iPods • Computing power => faster processing of large volumes of digitized content • Networks => increased availability of musical content in digital form (remotely) • Pandora, Yahoo Music, iTunes • Technological advances + demand from consumers = attention of research and industry

5. Take 1: multi-disciplinary domain • General • Computer Science, Data Processing, AI, Pattern Recognition, Library & Information Sciences • Philosophy and Psychology • Sensory Perception, Emotions & feelings, Mental processes & intelligence • Social Sciences • Sociology & Anthropology, Culture & Institutions, Law, Commerce • Natural Science & Mathematics • General Technology • Electric, Electronic, Magnetic, Communications & Computer Engineering • The Arts • Music, Aesthetics, Composition

6. Take 2: schematic representation of MIR

7. Take 3: a typology of MIR

8. Music terms used in MIR • Pitch – perceived fundamental frequency of a sound. Maybe different from actual frequency because of harmonics. • Timbre – the quality of a musical note that distinguishes different types of sound production, such as voices or musical instruments (saxophone vs. trumpet – with same pitch and loudness) • Rhythm (aka beat) - the variation of the length and accentuation of a series of sounds • Tempo – the speed or pace of a musical piece. Usually affects the Mood of a song. • Melody – a linear succession of musical tones which is perceived as a single entity (‘horizontal’ aspect of music) • Harmony – simultaneous use of different pitches (‘vertical’ aspect of music) • Monophony – musical texture consisting of melody without accompanying harmony • Polyphony -  is a texture consisting of two or more independent melodic voices

9. Common Methods • Modeling: start from a theory, look for patterns • Look for melodies, harmonic progressions • Attempt to find elements in data that correspond to such entities • Statistical methods: look for patterns, build a theory • Perform statistical analysis on data, find common patterns and group them in clusters • Attempt to interpret their occurrence in musical pieces

10. MIR Challenges • The integration of audiovisual, symbolic and textual data • Fingerprinting - unique small set of features excerpted from a sound file, allowing to discriminate it from any other sound file • Music Summarization- how to select a representative excerpt that gives a good idea of the work (similar to thumbnails for image files) • Computing Similarity – no unique way in which two pieces may be similar • Melodic, Rhythmic, Timbre, Genre, Style similarities • Indexing a musical piece by melody – to allow QBH interface

11. MIR Challenges contd.. • Encoding of music – at acoustic, structural and semantic levels • Query-by-example – search for music by singing, humming, whistling or playing an audio excerpt • Watermarking – adding identification information to digital audio for DRM • Benchmarking - limited number of standardized test collections available for evaluation of MIR systems

12. A tool to catalog and extract audioCD contents for online distribution • Automatic identification of CDs • Compute CDDB of the CD • CDDB - a binary number reflecting the offsets (start time) and lengths of the tracks of the CD • Metadata retrieval and correction • Query Internet CDDB for metadata • Allow correction • Extraction and compression • Transfer to a Web server

13. IRCAM tool interface • When a CD is inserted in the computer: • The tool computes its CDDB • Retrieves the metadata if available (freedb.org, cddb.com, allmusic.com) • - Allows the librarian to correct errors, structure the tracks into works and select names from authority lists. • - When done, it adds the • metadata to the catalog, and extracts the tracks, compresses them and sends • them to the audio server.

14. Information sources • The International Society for Music Information Retrieval (http://www.ismir.net/) • University of Illinois’ Graduate School of Library and Information Science (http://www.music-ir.org/) • IRCAM (http://www.ircam.fr/) • http://articles.ircam.fr/textes/Fingerhut04b/ • The Listen Game — UCSD Computer Audition Lab MIR music ranking game (Herd It on Facebook) • Multi-player game where you listen to music with lots of other people (aka the Herd). You are asked to describe the music (genre, mood, singer etc.) and get points when the Herd agrees with you. • Innovative way to harness the power of social networking and collect metadata for MIR