1 / 138

Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.

Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon. Bob McMurray University of Iowa Dept. of Psychology. Peter Ladefoged (1925 – 2006). He taught us all phonetics. Collaborators. Richard Aslin Michael Tanenhaus David Gow. Joe Toscano

teige
Download Presentation

Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Back to the future: Where we’re going, we don’t need phonemes. Implications of a gradient lexicon. Bob McMurray University of Iowa Dept. of Psychology

  2. Peter Ladefoged (1925 – 2006) He taught us all phonetics

  3. Collaborators Richard Aslin Michael Tanenhaus David Gow Joe Toscano Cheyenne Munson Meghan Clayards Dana Subik The students of the MACLab

  4. In language, information arrives sequentially. • Partial syntactic and semantic representations are formed as words arrive. The cowboys chased the linguists away… • Words are identified over • sequential phonemes.

  5. Spoken Word Recognitionis an ideal arena in which to study these issues because: • Speech production gives us a lot of rich temporal information to use in this way. • We have a clear understanding of the input (from phonetics). • The output is easy to measure online

  6. Online Comprehension • Listeners form hypotheses as the input unfolds. • Need measurements of how listeners interpret speech, moment-by-moment. • May reveal how information is integrated: • Discreteness vs. Gradiency • Combinatorial Units

  7. Mechanisms of Temporal Integration • Stimuli do not change arbitrarily. • Perceptual cues reveal something about the change itself. • Active integration: • Anticipating future events • Retain partial present representations. • Resolve prior ambiguity.

  8. Overview • Speech perception and Spoken Word Recognition. 2) Lexical activation is sensitive to fine-grained detail in speech. 3) Where we’re going, we don’t need phonemes: evidence for continuous information integration. 4) Back in time: staying off the garden-path. 5) Forward to the future: coping with (and benefiting from) with phonological modification.

  9. X basic bakery bakery X ba… kery barrier X X bait barricade X baby • Online Word Recognition • Information arrives sequentially • At early points in time, signal is temporarily ambiguous. • Later arriving information disambiguates the word.

  10. Current models of spoken word recognition • Immediacy: Hypotheses formed from the earliest moments of input. • Activation Based: Lexical candidates (words) receive activation to the degree they match the input. • Parallel Processing: Multiple items are active in parallel. • Competition: Items compete with each other for recognition.

  11. Input: b... u… tt… e… r time beach butter bump putter dog

  12. These processes have been well defined for a phonemic representation of the input. l   gw  d  But considerably less ambiguity if we consider subphonemic information. Example: subphonemic effects of motor processes.

  13. Coarticulation n n ee t c k Any action reflects future actions as it unfolds. Example:Coarticulation Articulation (lips, tongue…) reflectscurrent, futureandpastevents. Subtle subphonemic variation in speech reflects temporal organization. Sensitivity to theseperceptualdetails might yield earlier disambiguation.

  14. These processes have largely been ignored because of a history of evidence that perceptual variability gets discarded. Example:Categorical Perception

  15. Categorical Perception B 100 100 Discrimination % /p/ Discrimination ID (%/pa/) 0 0 B VOT P • Sharp identification of tokens on a continuum. P • Discrimination poor within a phonetic category. Subphonemic variation in VOT is discarded in favor of adiscretesymbol (phoneme).

  16. Sense Categorical Perception (CP) • Defined fundamental computational problems. • CP is output of • Speech perception • Input to • Phonology • Word recognition. Words Phonology Phonemes Sound

  17. Evidence against the strong form of Categorical Perception from psychophysical-type tasks: • Discrimination Tasks • Pisoni and Tash (1974) • Pisoni & Lazarus (1974) • Carney, Widin & Viemeister (1977) Classic explanation: Auditory tasks: non-categorical Phonological tasks: categorical Paradigmatic CP: within-category variation is noise. Not important to higher language. • Training • Samuel (1977) • Pisoni, Aslin, Perey & Hennessy (1982) • Goodness Ratings • Miller (1997) • Massaro & Cohen (1983)

  18. Sense Categorical Perception (CP) Enables a divide-and-conquer approach. Words Cont. cues (non-CP) Phonology Phonemes Sound

  19. Sense Categorical Perception (CP) Enables a divide-and-conquer approach. Words • But, assumes that • Speech tasks tap phonemes (or something like them) • Phonemes (or something like them) are legitimate processing units. Phonology Phonemes Sound

  20. Minimal computational problem: Computing meaning. Words Phonology Phonemes Sound

  21. ? CP Minimal computational problem: Computing meaning. CP tasks don’t necessarily tap a stage of this problem. Words Phonology Phonemes Sound

  22. Minimal computational problem: Computing meaning. CP tasks don’t necessarily tap a stage of this problem. Words Phonology Phonemes Lexical representation: clearly a component. Sound Goal: Reassess continuous sensitivity (non-CP) w.r.t. words

  23. Experiment 1 ? Does within-category acoustic detail systematically affect higher level language? Is there a gradient effect of continusuous acoustic detail on lexical activation?

  24. Experiment 1 A gradient relationshipwould yield systematic effects of subphonemic information on lexical activation. If this gradiency is useful for temporal integration, it must be preserved over time. Need a design sensitive to bothacoustic detailand detailedtemporal dynamicsof lexical activation. McMurray, Aslin & Tanenhaus (2002)

  25. Acoustic Detail Use a speech continuum—more steps yields a better picture acoustic mapping. KlattWorks:generate synthetic continua from natural speech. • 9-step VOT continua (0-40 ms) • 6 pairs of words. • beach/peach bale/pale bear/pear • bump/pump bomb/palm butter/putter • 6 fillers. • lamp leg lock ladder lip leaf • shark shell shoe ship sheep shirt

  26. Temporal Dynamics How do we tap on-line recognition? With an on-line task:Eye-movements Subjects hear spoken language and manipulate objects in a visual world. Visual world includes set of objects with interesting linguistic properties. abeach, apeachand some unrelated items. Eye-movements to each object are monitored throughout the task. Tanenhaus, Spivey-Knowlton, Eberhart & Sedivy, 1995

  27. Why use eye-movements and visual world paradigm? • Relatively naturaltask. • Eye-movements generated veryfast(within 200ms of first bit of information). • Eye movementstime-lockedto speech. • Subjectsaren’t awareof eye-movements. • Fixation probability maps ontolexical activation..

  28. Task A moment to view the items

  29. Task Bear Repeat 1080 times

  30. Identification Results 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 5 10 15 20 25 30 35 40 High agreement across subjects and items for category boundary. proportion /p/ B VOT (ms) P By subject:17.25 +/- 1.33ms By item: 17.24 +/- 1.24ms

  31. Task 200 ms Trials 1 2 3 4 5 % fixations Time Target = Bear Competitor = Pear Unrelated = Lamp, Ship

  32. Task 0.9 VOT=0 Response= VOT=40 Response= 0.8 0.7 0.6 0.5 Fixation proportion 0.4 0.3 0.2 0.1 0 0 400 800 1200 1600 2000 0 400 800 1200 1600 Time (ms) More looks to competitor than unrelated items.

  33. Task target Fixation proportion Fixation proportion time time • Given that • the subject heard bear • clicked on “bear”… How often was the subject looking at the “pear”? Categorical Results Gradient Effect target target competitor competitor competitor competitor

  34. Results 20 ms 25 ms 30 ms 10 ms 15 ms 35 ms 40 ms 0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 0 400 800 1200 1600 0 400 800 1200 1600 2000 Response= Response= VOT VOT 0 ms 5 ms Competitor Fixations Time since word onset (ms) Long-lasting gradient effect: seen throughout the timecourse of processing.

  35. 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Area under the curve: Clear effects of VOT B: p=.017* P: p<.001*** Linear Trend B: p=.023* P: p=.002*** Response= Response= Looks to Competitor Fixations Looks to Category Boundary VOT (ms)

  36. 0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 5 10 15 20 25 30 35 40 Unambiguous Stimuli Only Clear effects of VOT B: p=.014* P: p=.001*** Linear Trend B: p=.009** P: p=.007** Response= Response= Looks to Competitor Fixations Looks to Category Boundary VOT (ms)

  37. Summary Subphonemic acoustic differences in VOT have gradient effect on lexical activation. • Gradient effect of VOT on looks to the competitor. • Effect holds even for unambiguous stimuli. • Seems to be long-lasting. Consistent with growing body of work using priming (Andruski, Blumstein & Burton, 1994; Utman, Blumstein & Burton, 2000; Gow, 2001, 2002).

  38. An alternative framework Word recognition is systematically sensitive to subphonemic acoustic detail. 2) Continuous acoustic detail is represented as gradations in activation across the lexicon. 3) This can do the work of sublexical units like phonemes. 4) Gradient sensitivity coupled to normal word recognition processes enables the system to take advantage of subphonemic regularities for temporal integration.

  39. Lexical Sensitivity Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories

  40. Extensions B Sh L P Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories  Metalinguistic Tasks

  41. Extensions 0.1 Response=P Looks to B 0.08 0.06 Competitor Fixations Response=B Looks to B 0.04 Category Boundary 0.02 0 0 5 10 15 20 25 30 35 40 VOT (ms) Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories  Metalinguistic Tasks

  42. Extensions 0.1 Response=P Looks to B 0.08 0.06 Competitor Fixations Response=B Looks to B 0.04 Category Boundary 0.02 0 0 5 10 15 20 25 30 35 40 VOT (ms) Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories  Metalinguistic Tasks

  43. Lexical Sensitivity Word recognition is systematically sensitive to subphonemic acoustic detail. • Voicing • Laterality, Manner, Place • Natural Speech • Vowel Quality • Infant voicing categories  Metalinguistic Tasks ? Non minimal pairs • ? Duration of effect (Exp 3-4)

  44. bump pump dump bun bumper bomb 2) Continuous acoustic detail is represented as gradations in activation across the lexicon. Input: b... u… m… p… time

  45. 3) This can do the work of sublexical units like phonemes. If lexical processes can represent speech detail, do we need sublexical processes? Perhaps: How are multiple cues (to the same phoneme) integrated? (Exp 2)

  46. Temporal Integration 4) Gradient sensitivity coupled to normal word recognition processes enables the system to take advantage of subphonemic regularities for temporal integration. • Regressive ambiguity resolution (exp 3-5): • Ambiguity retained until more information arrives. • Progressive expectation building (exp 5-6): • Phonetic distinctions are spread over time • Anticipate upcoming material.

  47. Overview • Speech perception and Spoken Word Recognition. 2) Lexical activation is sensitive to fine-grained detail in speech. 3) Where we’re going, we don’t need phonemes: evidence for continuous information integration. 4) Back in time: staying off the garden-path. 5) Forward in time: coping with (and benefiting from) with phonological modification.

  48. words phonemes   Substitute your favorite sublexical unit here (syllables, diphones, etc)… Traditional speech chain: signal-> phonemes -> words

More Related