Studying Music Performance: A Probabilistic Approach David Temperley

1. Studying Music Performance: A Probabilistic Approach David Temperley Eastman School of Music University of Rochester

2. Four goals of music performance: Render the notes (and other score information) correctly Convey the emotional content of the piece Clarify the structure of the piece Modulate the flow of information

3. Four goals of music performance: Render the notes (and other score information) correctly Convey the emotional content of the piece Clarify the structure of the piece Modulate the flow of information The topic of today’s talk

4. Part I: Clarification of structure • Three examples • The circularity problem (performance  perception or • perception  performance?) • Ambiguity avoidance

5. Clarification of structure in music performance Many authors have observed that one goal of a good performance is to convey the structure of the piece (Sloboda 1985, Clarke 1988, Gabrielson 1999). (“Structure” might include many things—meter, harmony, phrase structure, form, motivic connections, etc.) Sometimes this point is made indirectly—e.g. when it is said simply that a good performer must be aware of the structure of the piece. (Why is it beneficial to be aware of this? Presumably so that it can be conveyed to the listener.)

6. Clarification of structure – Example 1: Accentuation of metrically strong events The perception of meter is a complex process, involving a variety of different cues or “phenomenal accents” (Lerdahl & Jackendoff, 1983): length, loudness, articulation, harmony, parallelism... Performers can facilitate this process by aligning phenomenal accents with strong beats—that is, by emphasizing metrically strong events.

7. Clarification of meter, cont’d. The same notes (pitches and rhythms) in different metrical contexts will be played quite differently: metrically strong notes are louder, longer, more legato (Sloboda 1983, 1985). …And listeners can identify which melody is being performed.

8. Clarification of meter: Dynamics Performers tend to play metrically strong notes somewhat louder than others, though the difference is relatively small. Data from Drake & Palmer (1993) for piano performance, showing average “hammer velocity” values for notes on beats 1-4 of a 4/4 measure. (Note: range for entire piece was 30 to 600 units!)

9. Clarification of structure – Example 2: Phrase-final lengthening Performers tend to decrease tempo at phrase boundaries, and more markedly at larger sectional boundaries (Todd 1985; Palmer 1996; Windsor & Clarke 1997). (smooth curve) (jagged line) phrase boundaries Data from Windsor & Clarke 1997

10. Clarification of structure – Example 3: Individuation of melodic lines The perception of polyphonic music requires the identification of individual lines—voice separation or “contrapuntal analysis” (Temperley 2001). Performance teachers (of polyphonic instruments) often urge students to individuate different lines of the texture in some way—for example, by emphasizing the melody. A specific case in point is melodic lead—the tendency of the melody notes to anticipate other notes (Palmer 1996). This could be an strategy to individuate melodic notes by making them asynchronous with other notes.

11. According to the “clarification of structure” view, these phenomena (accentuation of metrically strong events, phrase-final lengthening, melodic lead) arose, at least in part, as ways of clarifying musical structure for the listener.

12. A possible circularity in this argument: therefore Listeners expect metrically Performers play metrically strong notes to be louder / longer strong notes louder / longer

13. A possible circularity in this argument: therefore Listeners expect metrically Performers play metrically strong notes to be louder / longer strong notes louder / longer therefore

14. How to address the circularity problem? Three responses... • 1. Perhaps the process really is circular. Performance tendencies • gave rise to perceptual tendencies, but were then reinforced • by them. • 2. Perceptual tendencies may have come first and may have • originated outside music, e.g. in language: • - Stressed syllables are normally longer and louder • (Fry 1955). • - Phrase-final lengthening is a cue to syntactic boundaries • in speech (Lehiste 1973).

15. 3. The case for perception influencing performance is strengthened if it can be shown that the performance tendencies are strongest in cases that would otherwise be ambiguous perceptually.

16. 3. The case for perception influencing performance is strengthened if it can be shown that the performance tendencies are strongest in cases that would otherwise be ambiguous perceptually. Ambiguity avoidance is well-documented in language: For example, speakers exaggerate prosodic marking of phrase structure (e.g. lengthening at syntactic boundaries) when a sentence is ambiguous in context (Snedeker & Trueswell 2003).

17. Ambiguity Avoidance in Music – Example 1: Organ performance On instruments where dynamic accentuation is not possible, do performers increase other kinds of accentuation on metrically strong notes? In organ pedagogy, performers are taught to emphasize metrically strong notes through duration (i.e. with longer inter-onset intervals to the following note—[A]) or articulation (playing them slightly more legato— [B]) (e.g. Hurford 1988). : . : . A B

18. Ambiguity Avoidance – Example 2: Phrase-final lengthening at ambiguous points Do performers exaggerate phrase-final lengthening in cases where the phrase boundary would otherwise be unclear or ambiguous?

19. Brahms, Intermezzo Op. 116 No. 6, mm. 1-8 Rubinstein’s performance (1959) Question: Why does Brahms mark sostenuto and a fermata at the phrase ending in mm. 7-8 – when the performer would normally slow down at a phrase boundary anyway? Answer: Given the unusual harmonic context (the sudden move to C# major), the boundary might not easily be noticed here unless especially reinforced by the performer.

20. Schumann, “Von fremden Ländern und Menschen” from Kinderszenen Question: Why does Schumann mark ritardano before the return of the opening theme? Answer: The return happens at an unexpected place in the phrase structure—after only a six-measure phrase—and with no clear change of harmony; thus the structural boundary might be missed if it were not especially marked by phrase-final lengthening.

21. Ambiguity Avoidance – Example 3: The rise of “swing tempo” in jazz Swing tempo: The lengthening of the first half of the beat in relation to the second.

22. Maple Leaf Rag, as performed by Scott Joplin (1918) and Jelly Roll Morton (1938) (my transcriptions) Excerpt 1 (Joplin) (Morton)

23. “Maple Leaf Rag”, Excerpt 2 (Joplin) (Morton)

24. Morton’s performance features a great deal more syncopation than Joplin’s—especially left-hand events on weak eighth-note beats; this potentially obscures the quarter- • note level of the meter. But it also features a swing • tempo, which clarifies the quarter-note level of the meter (by making strong-beat notes longer than weak-beat ones). • And in general, the transition from ragtime to jazz features... • a shift from the “even-note” rhythms of ragtime to the “swing” feel of jazz (Schuller, 1967) • an increase in the degree and complexity of syncopation (Waterman, 1974; Sales, 1984)

25. In each of these cases of ambiguity avoidance... • - Accentuation of metrically strong notes through length and articulation in organ performance • - “Disambiguating” ritards marked at potentially ambiguous phrase boundaries • - The rise of swing tempo in jazz • ...we see increased use of “clarifying” performance cues in situations that would otherwise be ambiguous—supporting the idea that these cues arose, in part, as a way of clarifying structure.

26. Part II: Modulation of Information Flow

27. Information is the negative log of probability. So in a sequence of events, the lower the probability of an event, the more information it conveys. (Also known as surprisal...) Information 6 5 4 3 2 1 0 Probability 1.0

28. The Uniform Information Density Hypothesis (Levy & Jaeger, 2007): A certain rate of information flow is optimal in language perception, and language production tends to be adjusted to maintain this rate.

29. Uniform Information Density in Language (Levy & Jaeger, 2007) In prosody, words that are more probable are pronounced more quickly: - More frequent words are shortened (Jurafsky et al., 2001) - Words that are more probable in local context (e.g. P(wn | wn–1)) are shortened (Jurafsky et al. 2001; Aylett & Turk, 2004) - Words that are more probable given larger context (e.g. that have already occurred in the discourse) are shortened (Fowler & Jousum, 1987)

30. Modulation of information flow: What does this predict for music performance? One prediction: Less probable events should be performed more slowly.

31. Modulation of information flow: What does this predict for music performance? One prediction: Less probable events should be performed more slowly. What about unexpected harmonies, e.g. chromatic harmonies? Sundberg’s (1988) rules for good expressive performance include the “Harmonic Charge Rule”: Slow down on chords whose pitches are far from the tonic on the circle of fifths. (Never really tested with regard to performance.)

32. Expressive performance as modulation of information flow: Anecdotal evidence Brahms, Intermezzo Op. 116 No. 6, mm. 1-8 (again) Question: Why does Brahms mark sostenuto and a fermata at the phrase ending in mm. 7-8? Possible Answer: Given the modulation to a distant key and the rapid harmonic rhythm, this passage is high in information content; slowing down smooths out the information flow.

33. How does harmony affect expressive timing? Do performers slow down on unexpected harmonies? Chris Bartlette’s dissertation (Eastman, 2007) examined this experimentally. Skilled pianists were given pairs of passages to practice and perform. Within each pair, the passages were identical except for one chord, which was diatonic (within the key) in one case and chromatic (outside the key) in the other case. Effect of harmony on performance expression was examined (timing, dynamics, asynchrony).

34. Examples (Barlette, 2007) The second half of a diatonic passage (above) and a chromatic passage (below). (MIDI files generated from experimental trial, reflecting actual timing and dynamics of performance.) E: ii6 E: bII6

35. Result: Both the “chord of interest” and the previous chord are lengthened when the chord of interest is chromatic. Cn-1CnCn+1 (“chord of interest”) Why? To allow more time for the chord to be processed... “smoothing the information flow.”

36. Various sources of evidence... - Sundberg’s rules - Bartlette’s dissertation - anecdotal evidence from score markings ...suggest a preference in performance to slow down at unexpected harmonies. This can be seen as an example of modulation of information flow.

37. Part III. The Syncopation-Rubato Trade-Off Syncopation - non-alignment between beats and phenomenal accents Rubato - irregularity or fluctuation in tempo for expressive purposes The Syncopation-Rubato Trade-Off: Music with more syncopation seems to permit less rubato.

38. Rhythm in traditional sub-Saharan African music (compared to Western classical music) - Greater use of syncopation (Jones 1959, Chernoff 1977, other authors) - Greater strictness of tempo (less rubato): “When we Europeans imagine we are beating strict time, the African will merely smile at the ‘roughness’ of our beating.” (Jones 1959, p. 38) Also: rock and jazz - a high degree of syncopation (compared to classical Western music); very little tempo fluctuation.

39. An Experimental Study of the Syncopation-Rubato Trade-Off (with Kelly Francis)Subjects: Doctoral students in piano, Eastman School of MusicProcedure: Subjects were given short passages and asked to perform them (after practicing). The variance in tempo within each performance was then analyzed.

40. Materials: Four pairs of musical phrases, with the same pitches in the same approximate positions; one pattern in each pair is unsyncopated (1A), the other is highly syncopated (1B). (Deadpan MIDI performances)

41. Results • Pianists played unsyncopated excerpts with significantly more • variance in tempo than syncopated excerpts (F(1, 9)=8.17, p=.02). • The effect varied greatly across excerpt pairs:

42. THE BIG QUESTION: The syncopation-rubato trade-off—is it clarification of structure, OR modulation of information flow?

43. THE BIG QUESTION: The syncopation-rubato trade-off—is it clarification of structure, OR modulation of information flow? Clarification of structure: Could we argue that the combination of rubato and syncopation obscures the metrical structure more than either one alone?

44. The syncopation rubato trade-off as clarification of structure A. A perfectly regular pattern B. Syncopation: one event occurs slightly before the beat C. Rubato: intervals between beats are slightly irregular D. Syncopation and rubato combined: the last beat is slightly late, the last event slightly anticipates it (now the onset pattern is the same as in A!) intended rhythm beats note onsets 600 ms 600 ms 600 ms 600 ms 600 ms 600 ms 450 ms 600 ms 600 ms 800 ms 600 ms 600 ms 800 ms 600 ms

45. The syncopation rubato trade-off as clarification of structure A. A perfectly regular pattern B. Syncopation: one event occurs slightly before the beat C. Rubato: intervals between beats are slightly irregular D. Syncopation and rubato combined: the last beat is slightly late, the last event slightly anticipates it (now the onset pattern is the same as in A!) intended rhythm beats note onsets 600 ms 600 ms 600 ms 600 ms 600 ms 600 ms 450 ms 600 ms 600 ms 800 ms So...syncopation and rubato in combination obscures the meter. 600 ms 600 ms 800 ms 600 ms

46. While there are cases where the combination of syncopation and rubato obscures the meter, it is difficult to make a general argument to this effect. What about modulation of information flow?

47. The Syncopation-Rubato Trade-Off as Modulation of Information Flow Imagine a one-second segment of a melody. The listenermust predict when the next note will occur. P q q

48. The Syncopation-Rubato Trade-Off as Modulation of Information Flow Imagine a one-second segment of a melody. The listenermust predict when the next note will occur. P q q P High information—all locations are equally likely P Low information—one location is much more likely than all others

49. In a style with no syncopation or rubato, the location of each note is highly predictable. (Low information) q q

50. In a style with rubato, the beat location itself is unpredictable so the note location is as well. (Moderate information) P q q