Song Intersection by Approximate Nearest Neighbours

1. Song Intersection by Approximate Nearest Neighbours Michael Casey, Goldsmiths Malcolm Slaney, Yahoo! Inc.

2. Overview • Large Databases: Everywhere! • 8B web pages • 50M audio files on web • 2M songs • Find duplicates with shingles • Text-based • LSH - Randomized projections • Results • Best features • 2018 song subset

3. The Need for Normalization • Recommendations • Apply one song’s rating to another • – > Better matches • Playlists • Find matches to user requests • Remove adult/child music • Search results • Don’t show duplicates

4. Specificity Spectrum Fingerprinting Remixes Cover songs Genre Look for specific exact matches Our work (nearestneighbor) Bag of Features model

5. Remixes of One Title

6. Remix Examples Abba Gimme Gimme Madonna Hung Up Tracy Young Remix of Hung Up Tracy Young Remix 2 of Hung Up

7. How Remix Recognition Works • Algorithm • Matched filter best (ICASSP2005 result) • Nearest neighbor in 360–1200D space • Ill posed? • Efficient implementation • Audio shingles • Like web-duplicate search • Locality-sensitive hashing • Probabilistic guarantee

8. Audio Processing

9. Remix Distance Matched filter (implemented as nearest neighbor) N-best matches

10. Choosing r0

11. Hashing • Types of hashes • String : put casey vs cased in different bins • Locality sensitive : find nearest neighbors • High-dimensional and probabilistic • Two Nearest Neighbor implementations • Pair-wise distance computation • 1,000,000,000,000 comparisons in 2M song database • Hash bucket collisions • 1,000,000,000 hash projections

12. Random Projections • Random projections estimate distance • Multiple projections improve estimate

13. Locality Sensitive Hashing Distant Vector • Hash function is a random projection • No pair-wise computation • Collisions are nearest neighbors Distant Vector

14. Remix Nearest Neighbour Algorithm 1 • Extract database audio shingles • Eliminate shingles < song’s mean power • Compute remix distance for all pairs • Choose pairs with remix distance < r0

15. Remix Nearest Neighbour Algorithm Revisited • Extract database audio shingles • Eliminate shingles < song’s mean power • Hash remaining shingles, bin width=r0 • Collisions are near neighbour shingles

16. Method • Choose 20 Query Songs • Each has 3-10 Remixes • 306 Madonna Songs • 2018 Madonna+Miles

17. Results

18. Conclusions • Remixes are hard, but well-posed • Brute force distances too expensive • LSH is 1-2 orders of magnitude faster • LSH Remix Recognition is Accurate

19. Conclusions • Remixes are hard, but well-posed • Brute force distances too expensive • LSH is 1-2 orders of magnitude faster • LSH Remix Recognition is Accurate

20. Conclusions • Remixes are hard, but well-posed • Brute force distances too expensive • LSH is 1-2 orders of magnitude faster • LSH Remix Recognition is Accurate

21. Conclusions • Remixes are hard, but well-posed • Brute force distances too expensive • LSH is 1-2 orders of magnitude faster • LSH Remix Recognition is Accurate