1 / 54

LAM: Musical Audio Similarity

LAM: Musical Audio Similarity. Michael Casey Centre for Cognition, Computation and Culture Department of Computing Goldsmiths College, University of London. Overview. Machine Music Understanding Features / Classes / Clusters Real-Time Audio Matching Feature Extraction

greg
Download Presentation

LAM: Musical Audio Similarity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LAM: Musical Audio Similarity Michael Casey Centre for Cognition, Computation and Culture Department of Computing Goldsmiths College, University of London

  2. Overview • Machine Music Understanding • Features / Classes / Clusters • Real-Time Audio Matching • Feature Extraction • Feature Similarity (Indexing / Retrieval) • PD/MSP Tools • Music Similarity Applications • Sound object matching • Texture matching

  3. Sound Understanding Signal Processing Sound Understanding

  4. Feature Extraction

  5. Feature Extraction

  6. Feature Extraction

  7. Feature Extraction

  8. Feature Extraction

  9. Feature Extraction

  10. Statistical Learningfor Decision Making Partitioning of feature space p( | ) * P( ) P( | )= p( ) Decision boundary Music Speech

  11. MPEG-7 Audio Tools Audio

  12. MPEG-7 Audio Tools Log Frequency Spectrogram Audio AudioSpectrumEnvelopeD

  13. MPEG-7 Audio Tools Decorrelating Transform / Dimension Reduction Log Frequency Spectrogram Log Amplitude Audio AudioSpectrumEnvelopeD AudioSpectrumProjectionD

  14. SoundModelStatePathD Use estimated state sequence as a feature State Path

  15. MPEG-7 Audio Tools Decorrelating Transform / Dimension Reduction Log Frequency Spectrogram Hidden Markov Model Log Amplitude Audio AudioSpectrumEnvelopeD SoundModelDS AudioSpectrumProjectionD

  16. MPEG-7 Audio StringsAcoustic Lexicons Decorrelating Transform / Dimension Reduction Log Frequency Spectrogram Hidden Markov Model Log Amplitude Audio AudioSpectrumEnvelopeD SoundModelDS State Path AudioSpectrumProjectionD SoundModelStatePathD ? 7 1 V 7 1 0 1 ... SYMBOL STRING

  17. State Symbol Sequence (40 State Model) ?71V7101 ...

  18. State Symbol Sequence (40 State Model) ?71V7101 ...

  19. State Symbol Sequence (40 State Model) ?71V7101 ...

  20. State Symbol Sequence (40 State Model) ?71V7101 ...

  21. SoundModelStateHistogramD state index 0.01s Frames state index seconds

  22. Self-Similarity Matrix

  23. Self-Similarity Matrix

  24. Self-Similarity Matrix

  25. Self-Similarity Matrix a

  26. Self-Similarity Matrix a b

  27. Self-Similarity Matrix a b

  28. Self-Similarity Matrix

  29. S-Matrix

  30. Efficient Storage / Retrieval • Real-Time Access • Large Databases • Distributed Databases

  31. PostgreSQL Database Representation of State Path “Strings” and Histograms

  32. Similarity • Compute distance between feature pairs • Features == SoundModelStateHistogramD • Similarity Metric • dist(a,b) >= 0 • dist(a,b)== 0 iff a==b • dist(a,b) + dist(b,c) >= dist(a,c) • Vector Dot Product

  33. Similarity of Feature Trajectories

  34. Dynamic Time Warping

  35. Acousticon Strings • Distance Metric • String Edit Distance (Levenschtein) • Scalable to Large Databases • PostgreSQL Implementation • Can use built-in Index Structures • Scalable to Real-Time Implementation • matching and audio streaming (< 20ms )

  36. Information Retrievalfor Creativity • Utilize sound extant database for new material • Take the structure of a music clip but replace the content. • New interfaces for music creativity.

  37. Audio Information Retrieval MPEG-7 Database A pre-indexed Collection of Sounds

  38. Audio Information Retrieval MPEG-7 Database Extract Segment Match Audio Query A Sound or Scene or List of Sounds Result List

  39. Audio Information Retrieval MPEG-7 Database Extract Segment Match Audio Query Feature extraction from audio. Result List

  40. Audio Information Retrieval MPEG-7 Database Extract Segment Match Audio Query Partitioning of audio into chunks. Result List

  41. Audio Information Retrieval MPEG-7 Database Extract Segment Match Audio Query Result List Find similar chunks of Audio

  42. Real-Time Matching

  43. Real-Time Matching Musaics

  44. Real-Time Matching Real-Time Matching Musaics

  45. Real-Time Matching Musaics

  46. Real-Time Matching Musaics

  47. Real-Time Matching Musaics

  48. Real-Time Matching Musaics

  49. Real-Time Matching Musaics

More Related