Comparative analysis of multiple musical performances

1. Comparative analysis of multiple musical performances Paper by Craig Stuart Sapp 2007 & 2008 Presented by SaleheErfanianEbadi QMUL ELE021/ELED021/ELEM021 5 March 2012

2. Outline • A technique for comparing numerous performances of an identical selection of music • The basic methodology is to split a one-dimensional sequence into all possible sequential sub-sequences, perform some operation on these sequences, and then display a summary of the results as a two-dimensional plot • The current focus is on beat-level information for tempo and dynamics as well as commixtures of the two

3. Methodology • The primary operation used on each sub-sequence is correlation between a reference performance and analogous • The result is a useful navigational aid for coping with large numbers of performances of the same piece of music and for searching for possible influence between performances. segments of other performances • Collected over 2,500 recorded performances for 49 of Chopin’s mazurkas—on average over 50 performances for each mazurka

4. Some points about performances • Keeping track of differences and similarities between performances is difficult • A written score contains only the most basic of expressive instructions (The unwritten rules of a composition are transmitted aurally between performers) • So to help exploring influences between performances tempo and dynamics are extracted from each, and then correlated against each other • Each performance in a plot is assigned a color

5. Raw Data • Beat duration & Loudness (easier to extract) using Sonic Visualizer • Many other facets (note timings, voicing, articulation, …) are ignored

6. Raw Data • For comparisons of musical dynamics between performances, a smoothed version of the raw power calculated for the audio signal every 10 ms is sampled at each beat location • Loudness detection uses the smoothed data, with a delay of 70 ms because smoothing introduces a delay in data

7. Analysis Tools • Normalized (Pearson) Correlation • Values range from −1.0 to +1.0, with 1.0 being an exact match, and 0.0 indicating no predictable relation between the sequences being compared

8. Analysis Tools • Scape Plots • Correlation values are hard to interpret in isolation • Originally designed for timbral and harmony structural analysis

9. Comparative performance scapes • Choose one performance to be the reference for a particular plot • For each cell in the scape plot, measure the correlation between the reference performance and all other performances, then make note of the performance which yields the highest correlation value • Color the cell with a unique hue assigned to that highest-correlating performance

10. Timescapes

11. Timescapes

12. Dynascapes

13. Scape plots and parallel feature sequences

14. HYBRID NUMERIC/RANK SIMILARITY METRICS FOR MUSICAL PERFORMANCE ANALYSIS

15. Outline • Comparing numerical method for examining similarities among tempo and loudness to Pearson Correlation • Other concepts such as “noise-floor” are used to generate more refined measurements than the correlation alone • The measurements are evaluated and compared to plain correlation in their ability to identify performances of the same Chopin mazurka played by the same pianist out of a collection of recordings by various pianists

16. Data • Almost 3,000 recordings of Chopin mazurkas were collected to analyze the stylistic evolution of piano playing over the past 100 years of recording history, which equates to about 60 performances of each mazurka

17. Methodology • Like before, beat timings extracted using Sonic Visualizer • Markup and manual corrections done • Dynamics then extracted as smoothed loudness values • Other musical features ignored, yet important in characterizing a performance: pianists don't play right left hands together; legato and staccato hard to extract but important; tempo and dynamic, useful features (kept), allow listeners to focus their attention on specific areas

18. Derivations and Definitions • Type-0 score • Plain correlation • Type-1 score • Nearest neighbor performances in terms of correlation at all timescales • Type-2 score • Removing Hatto effect- removing best matches step by step • Type-3 score • Removing noise floor • Type-4 score • One additional refinement (taking the geometric mean)

19. Type-0 score • This type of correlation is related to dot-product correlation used in Fourier analysis • Correlation values between extracted musical features typically have a range between 0.20 and 0.97 for different performances of mazurkas • Though it's hard to only interpret similarity directly from correlation values

20. Type-0 score • The correlation values are consistent only in relation to a particular composition, and these absolute values cannot be compared directly between different mazurkas • Different compositions will have different expected correlation distributions between performances

21. Type-1 score • In order to compensate partially for this variability in correlation distributions, scapeplots were developed which only display nearest-neighbor performances in terms of correlation at all timescales for a particular reference performance

22. Type-2 score • Scape displays are sensitive to Hatto effect: • If an identical performance to the reference, or query, performance is present in the target set of performances, then correlation values at all time resolutions will be close to the maximum value for the identical performance, and the comparative scape plot will show a solid color. All other performances would have an S1 score of approximately 0 regardless of how similar they might otherwise seem to the reference performance • To compensate for this problem, remove the best match from the scape plot in order to calculate the next best match

23. Type-2 score Schematic of nearest-neighbor matching method used in comparative timescapes

24. Type-3 score • Using the concept of noise-floor: • Definition of a performance noise-floor is somewhat arbitrary but splitting the performance database into two equal halves seems the most flexible rule to use • In any case, it is preferable that the noise floor does not appear to have any favored matches, and should consist of uniform small blotches at all timescales in the scape plot representing many different performers as is the example shown

25. Type-3 score

26. Type-4 score • Type-3 scores require one additional refinement in order to be useful since performances are not necessarily evenly distributed in the feature space • Therefore, the geometric mean is used to mix the S3 scorewith the reverse-query score (S3r)

27. Type-4 score

28. Evaluation • Presumably pianists will tend to play more like their previous performances over time rather than like other pianists. If this is true, then better similarity metrics should match two performances by the same pianist more closely to each other than to other performances by different pianists

29. Sources • Sapp, C. S. (2007). Comparative Analysis of Multiple Musical Performances. In Proceedings of the 8th International Conference on Music Information Retrieval. • Sapp, C. S. (2008). Hybrid Numeric/Rank Similarity Metrics for Musical Performance Analysis. In Proceedings of the 9th International Conference on Music Information Retreival.

30. Thank You