Categorical perception of speech: Task variations in infants and adults

1. Categorical perception of speech: Task variations in infants and adults Bob McMurray Jessica Maye Andrea Lathrop and Richard N. Aslin And a big thanks to Julie Markant

2. Categorical Perception & Task Variations • Overview • Previous work • Categorical perception and gradient sensitivity to subphonemic detail. • Categorical perception in infants • Reassessing this with HTPP & AEM • Infants show gradient sensitivity • A new methodology • Adult analogues

3. Categorical Perception Categorical Perception: Is subphonemic detail retained [and used] during speech perception? For a long time… NO! Subphonemic variation is discarded in favor of a discrete label. Categorical perception & gradiency

4. Non-categorical Perception A number of psychophysical-type results showed listeners’ sensitivity to within-category detail. 4AIX Task Pisoni & Lazarus (1974) Speeded Response Carney, Widen & Viemeister (1977) Training Samuel (1977) Pisoni, Aslin, Henessey & Perey (1982) Rating Task Miller (1997) Massaro & Cohen (1983)

5. Word Recognition These results did not reveal: Whether on-line word recognition is sensitive to such detail. Whether such sensitivity is useful during recognition.

6. Word Recognition • Mounting evidence that word-recognition is sensitive: • Lahiri & Marslen-Wilson (1991): vowel nasalization • Andruski, Blumstein & Burton (1994): VOT • Gow & Gordon (1995): word segmentation • Salverda, Dahan & McQueen (in press): embedded words and vowel length. • Dahan, Magnuson, Tanenhaus & Hogan (2001): coarticulatory cues in vowel.

7. Gradient Sensitivity 0.08 • Eye-movements to objects after hearing items from 9-step VOT continuum. 0.07 • Systematic relationship between VOT and looks to the competitor. 0.06 0.05 0.04 Response= Response= 0.03 Looks to 0.02 0 5 10 15 20 25 30 35 40 Competitor Fixations Looks to Category Boundary VOT (ms) McMurray, Tanenhaus & Aslin (2002) Bear

8. Gradient Sensitivity • If fine-grained detail is useful we might expect infants and children to • Show gradient sensitivity to variation • Tune their sensitivity to learning environment • ….BUT This systematic, gradient relationship between lexical activation and acoustic detail would allow the system take advantage of fine-grained regularities in the signal. • Gow, McMurray & Tanenhaus (Sat., 6:00 poster session) • Anticipate upcoming material. • Resolve Ambiguity

9. c Infant categorical perception Early findings of categorical perception for infants (e.g. Eimas, Siqueland, Jusczyk & Vigorito) have never been refuted. Most studies use: Habituation (many repetitions) Synthetic Speech Single continuum Perhaps a different method would be more sensitive? • Categorical perception in infants

10. Head-Turn Preference Procedure • Jusczyk & Aslin (1995) • Infants exposed to a chunk of language: • Words in running speech. • Stream of continuous speech (ala stat. learning) • Word list • After exposure, memory for exposed items (or abstractions) is assessed by comparing listening time to consistent items with inconsistent items.

11. How do we measure listening time? After exposure… Center Light blinks. Brings infant’s attention to center.

12. How do we measure listening time? When infant looks at center… One of the side-lights blinks.

13. Beach… Beach… Beach… How do we measure listening time? When infant looks at side-light… she hears a word.

14. How do we measure listening time? When infant looks at side-light… she hears a word. …as long as she keeps looking…

15. c Experiment 1: Gradiency in Infants 7.5 month old infants exposed to either 4 b-, or 4 p-words Bomb Bear Bail Beach Palm Pear Pail Peach 80 repetitions total Form a category of the exposed class of words. • Infants show gradient sensitivity Measure listening time on Bear Pear (Original word) Pear Bear (opposite) Bear* Pear* (VOT closer to boundary).

16. Experiment 1: Stimuli B: M= 3.6 ms VOT P: M= 40.7 ms VOT B*: M=11.9 ms VOT P*: M=30.2 ms VOT Both were judged /b/ or /p/ at least 90% consistently by adult listeners. B: 98.5% B*: 97% P: 99% P*: 96% Stimuli constructed by cross-splicing natural, recorded tokens of each end point.

17. Categorical Listening Time Gradient Bear Bear* Pear Measuring gradient sensitivity Looking time is an indication of interest. After hearing all of those B-words P sounds pretty interesting. So: infants should look longer for pear than bear. What about in between?

18. Individual Differences Novelty/Familiarity preference varies across infants and experiments. We’re only interested in the middle stimuli (b*, p*). Infants categorized as novelty or familiarity preferring by performance on the endpoints. Within each group will we see evidence for gradiency?

19. 10000 9000 8000 Listening Time (ms) 7000 6000 5000 4000 B B* P Novelty Results Novelty infants, Trained on B VOT: p=.001** Linear Trend: p=.001** .14 .004**

20. .001** 10000 .1 9000 8000 Listening Time (ms) 7000 6000 5000 4000 P P* B Novelty Results Novelty infants, Trained on P VOT: p=.001** Linear Trend: p=.001**

21. Familiarity Results Familiarity infants showed similar effects. B exposure Trend: p=.001 B vs B*: p=.19 B* vs P: p=.21 P exposure Trend: p=.009 P vs P*: p=.057 P* vs. B: p=.096 Trained on B Trained on P

22. Experiment 1: Conclusions • 7.5 month old infants show gradient sensitivity to subphonemic detail. • Individual differences in familiarity/novelty preferences. Why? • Length of exposure? • Individual factors? • Limitations of paradigm may hinder further study: • More repeated measures • Better understanding of “task” • Wider age-range.

23. Anticipatory Eye-Movements An ideal methodology would Yield an arbitrary, identification response. Yield a response to a single stimuli Yield many repeated measures Much like a forced-choice identification A new methodology Anticipatory Eye-Movements (AEM): Train Infants to look left or right in response to a single auditory stimulus

24. Anticipatory Eye-Movements Visual stimulus moves under occluder. Reemergence serves as “reinforcer” Concurrent auditory stimulus predicts endpoint of occluded trajectory. Subjects make anticipatory eye-movements to the expected location—before the stimulus appears. Teak Lamb

25. Anticipatory Eye-Movements After training on original stimuli, infants are tested on a mixture of • original, trained stimuli (reinforced) • Maintain interest in experiment. • Provide objective criterion for inclusion • new, generalization stimuli (unreinforced) • Examine category structure/similarity relative to trained stimuli.

26. Experiment 2: Pitch and Duration Goals: Use AEM to assess auditory categorization. Assess infants’ abilities to “normalize” for variations in pitch and duration… or… Are infants’ sensitive to acoustic-detail during a lexical identification task...

27. “teak!” “lamb!” Experiment 2: Pitch and Duration Training: “Teak” -> rightward trajectory. “Lamb” -> leftward trajectory. Test: Lamb & Teak with changes in: Duration: 33% and 66% longer. Pitch: 20% and 40% higher If infants ignore irrelevant variation in pitch or duration, performance should be good for generalization stimuli. If infants’ lexical representations are sensitive to this variation, performance will degrade.

28. The Stimuli Training stimulus

29. The Stimuli Testing stimulus

30. Results Each trials is scored as correct: longer looking time to the correct side. incorrect: longer looking time to incorrect side. Binary DV—similar to 2AFC. On trained stimuli: 11 of 29 infants performed better than chance–this is a tough tasks for infants. Perhaps more training.

31. 0.9 Pitch p>.1 0.8 0.7 0.6 0.5 Duration p=.002 0.4 0.3 0.2 0.1 0 Results On generalization stimuli: Proportion Correct Trials Duration Pitch Training Stimuli D1 / P1 D2 / P2 Stimulus

32. Experiment 2: Conclusions Infants’ developing lexical categories show graded sensitive to variation in duration. Possibly not to pitch Might be an effect of “task relevance” AEM yields more repeated measurements better understood task: 2AFC Could it yield a picture of the entire developmental time course? Is AEM applicable to a wider age range?

33. Treating undergraduates like babies Extreme case: Adult perception. Adults generally won’t Look at blinking lights… Suck on pacifiers… Kick their feet at mobiles… Result: few infant methodologies allow direct analogues to adults. They do make eye-movements… …could AEM be adapted?

34. Treating undergraduates like babies Pilot study. 5 adults exposed to AEM stimuli. Training: “Ba” left “Pa” right Test Ba – Pa (0-40 ms) VOT continuum.

35. Second group of subjects run in an explicit 2AFC. Same category boundary. Steeper slope: less sensitivity to VOT. Results 1 0.9 0.8 0.7 0.6 2AFC % /p/ 0.5 AEM 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 VOT (ms)

36. Adult AEM: Conclusions AEM paradigm can be used unchanged for adults. Should work with older children as well. Results show same category boundary as traditional 2AFC tasks, perhaps more sensitivity to fine-grained acoustic detail. Potentially useful for speech categorization when categories are not: nameable pictureable immediately obvious

37. Conclusions Like adults,7.5-month-old infants show gradient sensitivity to subphonemic detail. VOT Duration Perhaps not pitch (w.r.t. lexical categories)

38. Conclusions Task makes the difference: Moving to HTPP from habituation revealed subphonemic sensitivity. Taking into account individual differences crucial. Moving to AEM yields Better ability to examine tuning over time. Ability to assess perception across lifespan with a single paradigm.

39. Categorical perception of speech: Task variations in infants and adults Bob McMurray Jessica Maye Andrea Lathrop and Richard N. Aslin And a big thanks to Julie Markant

40. Palm Bomb Natural Stimuli Infants may show more sensitivity to natural speech Stimuli constructed from natural tokens of actual words with progressive cross-splicing.

41. 6 m/o 8 m/o Listening Time 10 m/o Experiment 1: Reprise Difficult to examine how sensitivity might be tuned to environmental factors in head-turn-preference procedure. • High variance/individual differences—can’t predict novelty/familiarity. • Only a single point to look at. • Between-subject comparison. • Difficult interaction to obtain Bear Bear* Pear

42. 10 m/o 8 m/o 6 m/o Experiment 1: Reprise AEM presents a potential solution: 1) Looking at whole continuum would yield more power. Bear Pear 2) Is AEM applicable to a wider age range?

43. The Stimuli Training stimulus

44. Data analysis Data coded by naive coders from video containing pupil & scene monitors.

45. Left-out, Right-out, center & start treated as “neither”. Left-in, Left treated as anticipation to left. Right-in, Right treated as anticipation to right. Left Left Right Right Center Left Left-In - In Right-In Right Left Left-out - out Right-out Right - out Start Off Data analysis Eye-movements coded from maximal size of stimulus to first appearance (or end of trial).