1 / 62

Behavioural Science Institute

Behavioural Science Institute. Towards an Embodied, Embedded Account of: Speech Perception, Speech Production, and... Reading? Fred Hasselman Ralf Cox Anna Bosman Dynamic Systems Group. Embodied Cognition Seminar, 18-07-05.

jovan
Download Presentation

Behavioural Science Institute

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Behavioural Science Institute Towards an Embodied, Embedded Account of: Speech Perception,Speech Production,and... Reading?Fred HasselmanRalf CoxAnna BosmanDynamic Systems Group

  2. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Warning: Work in progress!

  3. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? “Speech is rather a set of movements made audible than a set of sounds produced by movements” (Stetson 1951)

  4. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Overview • - Speech Perception and Speech Production: • - (Very) general overview • - Speech Perception & Production: The basics • - Models: ARTPHONE / DIVA • - Developmental Dyslexia as a Speech Perception deficit: • - What if there are no Critical Phoneme Boundaries? • Speech Perception, Speech Production (and Reading): • - Acquisition & Development • - Wild claims, bold statements

  5. Kent, R. (2000). Research on speech motor control and its disorders: A review and a prospective. Journal of Communication Disorders, 33, 391-428.

  6. BRAIN BODY ENVIRONMENT Kent, R. (2000). Research on speech motor control and its disorders: A review and a prospective. Journal of Communication Disorders, 33, 391-428.

  7. SOUL/ HOMUNCULUS / EVIL SCIENTIST / GOD BRAIN BODY ENVIRONMENT Kent, R. (2000). Research on speech motor control and its disorders: A review and a prospective. Journal of Communication Disorders, 33, 391-428.

  8. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? (Our) Problems with these kinds of models: • Cognition is achieved by a disembodied, logical, symbol manipulating, reasoning device (the brain) • Cognitive Internalism - Cowley, S., & Spurrett, D. (2003). ‘Putting apes (body and language) together again’, Language Sciences, 25, 289-318. • ”Brain in a Vat” view of cognition (Hillary Putnam). • The modules & filing cabinets: • Lexicon decision task • In Speech Production: The “mint” problem

  9. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? WEAVER ++ Levelt, W.J.M., Roelofs, A., Meyer, A.S.(1999). A theory of lexical access in speech production. BBS, 22, 1-75

  10. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? (Our) Problems with these kinds of models: • Cognition is achieved by a disembodied, logical, symbol manipulating, reasoning device (the brain) • Cognitive Internalism - Cowley, S., & Spurrett, D. (2003). ‘Putting apes (body and language) together again’, Language Sciences, 25, 289-318. • Brain in a Vat view of cognition (Hillary Putnam). • The modules & filing cabinets: • Lexicon decision task • In Speech Production: The “mint” problem • What about development?

  11. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? What about development? - Most models & theories are explaining acquisition and development of cognitive skills from models of the “end-product”. It is like figuring out how to bake a cake when you only have finished cakes. • Most models & theories are (implicitly) nativist: Cognitive Skill -> Brain Maturation -> Genes Genes -> Brain Maturation -> Cognitive Skill

  12. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? What about development? Most models & theories are (implicitly) nativist: - Origin of phoneme boundaries / categories: “… infants’ language-specific phonetic categories may initially emerge from an underlying cognitive capacity and proclivity to store in memory biologically important stimuli.” Kuhl, P. K., Williams, K. A., Lacerda, F., Stevens, K. N., et al. (1992). Linguistic experience alters phonetic Perception in infants by 6 months of age. Science, 255, 606-608. • Infants’ ability to extract words from the speech stream (Jusczyk & Aslyn, 1995; Houston, Jusczyk, Kuijpers, Coolen & Cutler, 2000) • Humans’ ability to extract linguistic information from the acoustically diverse speech signal (Traunmüller, 1994) - Speech module. Vouloumanos, A. & Werker, J. F. (2004). Tuned to the signal: the privileged status of speech for young infants.Developmental Science 7, 270–276. • Use of grammar and syntax only in human languages: language as ‘instinct’ or special purpose module (Chomsky, 1988; Fodor, 1981; Pinker, 1994) • The poverty of stimulus argument: Ability of infants to figure out the grammatical principles of their native tongue from just a small subset of linguistic input. (Hornstein & Lightfoot, 1981)

  13. Bates, E., Thal, D., Finlay, B. L., Clancy, B. (2002). Early language development and its neural correlates. In: F. Boller & J. Grafman (Series Eds.) & S.J. Segalowitz & I. Rapin (Vol. Eds.), Handbook of neuropsychology, Vol. 8: Child neurology (2nd ed., pp. 109–176). Amsterdam: Elsevier Science.

  14. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? The “007-Principle” “In general, evolved creatures will neither store, nor process information in costly ways when they can use the the structure of the environment and their operations upon it as a convenient stand-in for the information-processing operations concerned. That is, know only as much as you need to know to get the job done.” (Andy Clark, 1997, p. 46)

  15. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? BRAIN BODY ENVIRONMENT Beer, R.D. (2003). The dynamics of active categorical perception in an evolved model agent. Adaptive Behavior,11, 209-243.

  16. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? RT ENVIRONMENT BRAIN Beer, R.D. (2003). The dynamics of active categorical perception in an evolved model agent. Adaptive Behavior,11, 209-243.

  17. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Perception: The Basics BAK DAK Amplitude

  18. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Perception: The Basics B b>A K D d>A K Formant Transition Formant Transition

  19. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Perception: The Basics It is possible to create a continuum from BAK to DAK by slowly increasing F2 onset

  20. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Perception: The Basics It is possible to create a continuum from BAK to DAK by slowly increasing F2 onset

  21. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Perception: The Basics Phoneme Boundaries in Acoustic Space: Categorical Perception F3 F2

  22. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Perception: The ARTPHONE / SWEEP model • Based on the Adaptive Resonance Theory (Grossberg 1976, 1980) • Set of four coupled (linear) differential equations, mimicking • activation of items in working memory and lexical short term • memory through transmitter dynamics. • Accounts for categorical perception of VOT-continuum and speech rate. The SWEEP model is an extension of ARTPHONE capable of detecting transients in speech: formant transitions (Been & Zwarts, 1999) • - ARTWORD, ARTSTREAM, ARTMAP, fuzzy ART etc. also exist.

  23. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  24. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  25. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  26. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  27. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  28. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  29. (Our) Concerns • Body & Environment? -> Just brain. • Modules & filing cabinets • What about acquisition & development? • What about speech production?

  30. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Speech Production: The Basics Approximate number of muscle pairs that move: – Tongue: 9 – Velum: 3 – Lips: 12 – Mandible: 7 – Hyoid bone: 10 – Larynx: 8 – Pharynx: 4 NB: The respiratory system Perkell, J. S. Sensorimotor Control of Speech Production: Models and Data. Paper presented at ASHA.

  31. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Relation between Perception and Production • Speech Production of Said, shed, Sod and shod were recorded • Contact time of tongue to alveolar ridge was recorded • Speech Perception: Discrimination and labelling of a 7-step • ∫aid to shed continuum Perkell, J.S. et al. (2004). The distinctness of speakers' /s/-/S/ contrast is related to their auditory discrimination and use of an articulatory saturation effect. Journal of Speech, Language and Hearing Research ,47, 1259-1269.

  32. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Produced contrast distance is related to: • Ability to discriminate the contrast • Use of contact difference Interactions: • Speakers with good discrimination and use of contact difference: best contrasts • Speakers with one or the other factor: intermediate contrasts • Speakers with neither factor: poorest contrasts Perkell, J.S. et al. (2004). The distinctness of speakers' /s/-/S/ contrast is related to their auditory discrimination and use of an articulatory saturation effect. Journal of Speech, Language and Hearing Research ,47, 1259-1269.

  33. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Relation between Perception and Production • Same found for vowels: cod – cud / who’d - hood • Perkell, J.S., Guenther, F.H., Lane, H., Matthies, M.L. Stockmann, E., Tiede, M., & Zandipourf, M. (2004). • The distinctness of speakers’ productions of vowel contrasts is related to their discrimination of the contrasts • Journal of the Acoustical Society of America, 116, 2338-2344. • Different goals influence Speech Production: • Auditory & Somatosensory • Economy of Effort / Biomechanical Saturation / Clarity • Rub – GrubGuenther, F.H. et al. (1999). Articulatory tradeoffs reduce acoustic variability during American English /r/ • production. Journal of the Acoustical Society of America, 105, 2854-2865.

  34. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Relation between Perception and Production Formants begin to recover 60-90 ms after perturbation, jaw does not Evidence of within-movement, closed-loop error correction Perkell, J & Ostry, D.

  35. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? The DIVA (Directions Into Velocities) model

  36. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Neural Correlates of the DIVA model

  37. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? “While recognizing the value of theory-driven models, it is fair to ask whether the models offered by Plaut and Kello (1999) and Guenther (1994; Guenther, 1995; Guenther et al., 1998) would scale to handle all of the complexities inherent in the development and processing of real speech. The simulations are meant to serve as evidence for theories of speech acquisition and speech production. However, it is unclear how the models would perform when implemented with more veridical representations of speech articulations and acoustics. If the models were to fail under more veridical conditions, one would have to ask whether the theories were fundamentally flawed in some or way, or whether the failures were only due to shortcomings in the computational machinery.”

  38. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Strange Loops? • - Where do the speech sound maps and corresponding auditory and somatosensory goals in DIVA come from? • Auditory goals are tuned while “listening” to native language • phonemes and syllables or correct self-productions. • Somatosensory goals are tuned by many successful production • attempts. Who decides what is correct and successful? Pre- decided, hence nativist (Gustavson, 2005) • Auditory feedback in DIVA consists of learned goals/expectations. Kello & Plaut (2004) use formant frequencies in the speech signal, neither use the perceived signal. • Assumption: Speech signals with a particular pattern of formant frequencies always result in the same percept. • Probably works within individuals, but tells you nothing about the • validity of the model due to illocutionary intentions.

  39. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? • Origin of the Perceptual Boundaries: Three Models • Activation-deactivation of predispositions for feature perception: only • some of the innate perceptual boundaries are taken to form a phonological decoding level (Werker & Tees, 1984, Infant Behavior & Development) • 2. Perceptual magnet effects creates new boundaries from language- specific prototypes (Kuhl, 1991, Perception & Psychophyics) • 3. Coupling between predispositions creates new boundaries • (Serniclaes, 1987, PhD http://www.vjf.cnrs.fr/umr8606) Serniclaes, W. (2004). Speech perception: psychoacoustic,productive and linguistic factors. Paper pressented at the German-French Summerschool, Lubmin, Germany, 19th -24th September

  40. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Serniclaes, W. (2004). Speech perception: psychoacoustic,productive and linguistic factors. Paper pressented at the German-French Summerschool, Lubmin, Germany, 19th -24th September

  41. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Nonlinear Dynamics of Speech Categorization Porter, J. S., & Hogue, D. M. (1998). Nonlinear Dynamical Systems in Speech Perception and Production.Nonlinear Dynamics, Psychology, and Life Sciences, 2, 95-131.

  42. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? F3 F2

  43. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  44. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  45. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading?

  46. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Allophonic perception?

  47. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Nonlinear dynamics of speech perception in dyslexia

  48. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Urang Utang = (Wo)Man of the JungleBukit Lawang, Sumatera, Indonesia

  49. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Urang Utang = (Wo)Man of the JungleBukit Lawang, Sumatera, Indonesia

  50. Embodied Cognition Seminar, 18-07-05 An Embodied, Embedded account of Speech Perception, Speech Production and... Reading? Kanzi Picture from The Language Research Centre @ GSU

More Related