Listening to sound patterns as a dynamic activity

1. Listening to sound patterns as a dynamic activity By Mari Riess Jones Symposium talk; Psychonomics, November 2001

2. Thanks to Ralph Barnes Sue Holleran Devin McAuley Heather Moynihan Noah MacKenzie Amandine Penel Jennifer Puente Nate Vaughan Special Thanks to Edward Large

3. Overview • Theoretical background. • Dynamic attending. • Entrainment. • Time judgments • Targeting attending to points in time. • Expectancy profiles. 3. Pitch judgmentsRhythmic influences on pitch identifications.

4. Theoretical Background 1.

5. General Assumptions Temporal structure Is essential to living things and the way they connect to their environment. Living things Animate creatures possess internal periodicities that can synchronize to events. Environment Events afford time structure that is often quasi-periodic.

6. Internal Periodicities: Biological Oscillations Many internal periodicities exist; periods of 1 ms, several minutes, years !. Most studied are circadian rhythms; oscillatory periods of ca. 24 hours.

7. Connecting “How do internal oscillations of an organism connect with event time? “ One answer: By synchronizing. More precisely: By entrainment

8. Biological Entrainment Pittendrigh & Daan (1976) . Circadian rhythms of 24 hours synchronize to light/dark cycles. Daan et al. (2001) recently updated this two oscillator model.

9. Example: Two Entraining Oscillations Winter Summer From Daan et al. J.Biological Rhythms (2001)

10. Shorter Periods? In species with refined use of fast sound patterns, Can we postulate, internal oscillations with periods less than the circadian period ?

11. Generalizing….. Alternatively Other Oscillators Most Biological Oscillators Long natural periods Short natural periods Respond to light Respond to sound changes changes

12. Background:Rhythmic (periodic) Expectancies Quasi-periodicities in Rhythmical sound patterns Induce Periodic expectancies Do expectancies reflect entrainments to brief periods?

13. Back to Synchrony: Asimple premise about attending When we attend, an internal activity is elicited … that co-occurs with the attended object. This activity involves Internal oscillations (Jones, 1976).

14. Attending as Synchronous Oscillations In… dynamic arrays Between internal attending oscillations and environmental periodicities. static scenes Among distributed cortical-neural oscillations, in response to a common object.

15. I will be discussing Attending to Dynamic Arrays Where elements appear… then disappear Time Attention must occur in synchrony with elements.

16. Attentional Synchrony to dynamic events can be achieved by: • Sustaining attending uniformly in time. • OR 2. By variably allocating attending in time.

17. A Concern… Simple sustained attending: No attending to selected points in time. So, consider how attending varies in time….

18. Dynamic Attending Postulates that : Selective attending in time is paced by stimulus timing. Attentional focus in time becomes phase-locked to elements.

19. Attending oscillates in time Resources are periodically distributed as internal oscillations. Inter-onset time interval IOI Time Pulses of attention energy

20. Formalized as an Oscillator Model : by Ed Large (Large & Jones, 1999) Stimulus IOI Expected point in time Phase Period Periodic Expectancies Stimulus timing paces the oscillator. Oscillator period and phase determine this entrainment .

21. Formalized as an Oscillator Model : by Ed Large (Large & Jones, 1999) Stimulus IOI Expected point in time Phase Period Periodic Expectancies Stimulus timing paces the oscillator. Oscillator period and phase determine this entrainment .

22. Two routes to Synchrony 1. Anticipatory attending: Attending heightens prior to an element. Oscillator period is important. • Reactive attending: • If anticipation fails, • Attending shifts after an element occurs. • Oscillator phase is important.

23. The adaptive oscillator Tones Period Phase Period and Phase change: They draw attending rhythms to “fit” a time pattern. Period changes Phase changes

24. A simple dynamical system. The entrained oscillator gravitates to an attractor state of: • 1. Phase Synchrony:A zero time difference between an element onset and a pulse peak. • 2. Period matching:A zero time difference between an IOI and oscillator period.

25. 2. Time Judgments A few experiments….

26. Preview: Our Experimental Designs Involve a context rhythm using tones: C Followed by a comparison, C, (of some sort) Time JudgmentComparison IOI Pitch JudgmentComparison tone

27. Time Judgment TasksRationale If temporal arrays entrain attending, then a rhythmic context should Shape temporal expectancies Affect time judgments.

28. Example Study: Judgment Task: Is a comparison time interval “shorter”, “same” or “ longer” ? +/-T Standard IOI Comparison IOI Context IOI “Ignore context IOIs“ Context rhythm should synchronize attending.

29. Standard Ending Varies: Expected Standard Ending: = 600 ms Unexpected Standard Endings: Early = 579 ms Late = 621 ms Very Early = 524 ms Very Late = 676 ms A comparison IOI is always yoked to standard IOI 23

30. Modeling Oscillatory Attending Context IOIs STANDARD IOI COMPARISON IOI Expected Ending: “Same” Expected Ending 600 ms 600 Period Phase When a standard ends as expected, the oscillator’s period is accurately preserved ….as a memory of the standard IOI.

31. The Reason: The Attentional Pulse Expected time Pulse location Pulse location depends on rhythmic context 15

32. Prediction: A quadratic accuracy Expectancy Profile: Attentional Pulse

33. Predicted and ObservedExpectancy Profiles (Large & Jones 1999) P < .01

34. A Pitch Judgment Task Task: Is a comparison pitch: “same”, “higher”, ”lower” than a standard pitch? “Ignore distracting pitches!” Critical IOI “higher” same” “lower” Standard pitch Comparison pitch Independent Variable: Critical IOI 21

35. Subsequent Experiments Over a dozen experiments later (e.g., Barnes & Jones, 2000; McAuley & Jones, 2003) • We Find: • The profile vanishes with no context; sharpens with longer sequences . • Persists through a silent gap of twice the IOI. • Holds for different rates. • Reveals harmonic scaling of timing.

36. Different Induction Rates Isochronous sequences Three conditions: different induction rates. Condition Induction IOIs Standard durations 1. 300 ms 300 ms 524, 600, 676 ms 2. 500 ms 500 ms 524, 600, 676 ms 3. 600 ms 600 ms 524, 600, 676 ms This task required same vs different duration judgments

37. Observed Expectancy Profiles Three Time Judgment Conditions

38. Conclude Back to Relative timing 1. Ratios of induction IOIs to expected standards seem influential. 2. Harmonic ratios of 1:1 and 1:2 yield different profiles than the 500 ms with a more complex ratio.

39. Pitch Judgments We generalized this approach to attending to pitch. Does rhythmic context affect pitch judgments?

40. Rationale In a rhythmic context, attending is paced in time. Pitch judgments will be best for temporally expected tones: When critical IOI = context IOI.

41. Similar Predictions Aquadratic expectancy profile Attentional Pulse

42. P<.005

43. Related Experiments Found a: flattened expectancy profile with irregular context timing

44. Recent findings Suggest the importance of higher-order time structure … In focal attending

45. Focal Attending: Multiple Time Levels In 1989, Marilyn Boltz and I proposed: Focal Attending … In temporally coherent events, selective attending occurs over higher time levels (periodicities).

46. A Coherent Sound Event Accent Timing Higher time level Accented Tones Lower time level An Accent Rhythm 21

47. Accent Rhythm is Varied Regular Irregular Accented Tones 21

48. Same Pitch Judgment Task Critical IOI is Varied Standard pitch Comparison pitch 21

49. Back to Two Oscillators Remember those two circadian oscillators (Morning, Evening)? Well, these are different! Shorter periods Sensitive to sound

50. Two Different Oscillators Two oscillators an accent oscillator & a lower-order oscillator Which dominates? If the accent oscillator dominates, then only the Regular Rhythm will give an Expectancy Profile.