Instrument Classification in a Polyphonic Music Environment

1. Instrument Classification in a Polyphonic Music Environment Yingkit Chow Spring 2005

2. Objective • To develop and expand on processing techniques for classification of instruments in a polyphonic music environment Incentive • Classification of instruments can prove useful for automatic music transcription.

3. General Background • Most previous research with instrument classification works with monophonic audio • Work in Polyphonic music usually deals with “Blackboard” system that accesses knowledge sources and works with a top-down approach • Difficulties with polyphonic music: • Overlap of tones can be detrimental to extracting the correct identifying features • Confusion of octave

4. Features Used for Classification in Monophonic Music Analysis [1] Features Used: • RMS Envelope • 60 % Accuracy, 60% Reliability • CQT Frequency Spectrum (PCA) • 66 % Accuracy, 68% Reliability • MSA Trajectories (PCA) • 75 % Accuracy, 76 % Reliability • Combined: • 82% Accuracy, 83% Reliability

5. Results from [1], Monophonic Audio • CONFUSION MATRIX NNC Combined: k=5 WEIGHTED MAJORITY, Confusion Matrix weighted (reliability)

6. Missing Feature Approach[2],For Polyphonic musicJana Eggink and Guy J. Brown • Find fundamental frequency and compare against harmonic sieves • Spectral peaks of fundamental and harmonics are the features used for classification (50-6kHz, 60 Hz window) • Instruments tested in this paper: • (flute, clarinet, violin, cello, oboe)

7. Missing-Feature Approach, Special Conditions[2] • Frequency regions with energy from interfering tones are excluded from classification process. • Cepstral coefficients are not used as features since they correspond to all frequencies. Local spectral features are used. • Cannot handle drum or “untuned percussion instrument”

8. Results with Missing Feature Approach to monophonic music • Confusion Matrix for the 5 instruments in a Monophonic music environment • The identification of the family of instrument (String, Woodwind) is about 85%

9. Missing Feature Approach,Instrument Classification in 2 tone samples • Confusion Matrix for the 5 instruments in a polyphonic music environment (2 simultaneous tones)

10. Project Goal • I will test the Missing Feature Model against a different set of instruments: • Instruments to be selected based on available sample set and to provide a variety of instruments from different families. • Alternative source would be taking input from a synthesized version of the instrument (MIDI to Wav). • Add system over the missing feature model to include information from neighboring frames (in time) and use information of partials in the classification scheme

11. Testing • I will test my system first, within monophonic music, to get an upper bound on the capabilities of the system for each instrument. • Secondly, I will test the system within a 2 note environment

12. References 1. “Multi-feature Musical Instrument Sound Classifier”, I. Kaminskyj, http://www.mikropol.net/volume6/kaminskyj_i/kaminskyj_i.html 2. “A Missing Feature Approach to Instrument Identification in Polyphonic Music”, by Jana Eggink and Guy J. Brown http://www.dcs.shef.ac.uk/%7Ejana/egginkICASSP03.pdf 3. “Instrument Recognition in Acompanied Sonatas and Concertos”, by Jana Eggink and Guy J. Brown http://www.dcs.shef.ac.uk/%7Ejana/egginkICASSP04.pdf 4. Music Samples from the University of Iowa, http//:theremin.music.uiowa.edu/