1 / 40

Brain, Psycholinguistics, & Cognitive Science

Brain, Psycholinguistics, & Cognitive Science. Outline. How does psycholinguistics fit within the umbrella of cognitive science? What do we know about language and the brain?. Inter-relationships. What do cognitive psychologists do?.

batista
Download Presentation

Brain, Psycholinguistics, & Cognitive Science

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Brain, Psycholinguistics, & Cognitive Science

  2. Outline • How does psycholinguistics fit within the umbrella of cognitive science? • What do we know about language and the brain?

  3. Inter-relationships

  4. What do cognitive psychologists do? Construct theories that describe how mental processing works in real time: • Within some domain, what are the mental representations and how are they manipulated? • How does information flow through the mind? • How is processing impacted by memory constraints, stimulus quality, mode of input, tasks, etc? Levels of processing Serial vs. parallel processing Top-down/bottom-up Automatic vs. strategic processing

  5. What do (formal theoretical) linguists do? • Construct formal theories of our linguistic knowledge • Sets of rules/principles/assumptions for generating utterances • Criteria for a good theory • The rules generate all and only grammatical outputs (intuitions) • In the simplest way

  6. Derivational Theory of Complexity • Miller’s clause-processing model of syntactic processing, motivated by Chomsky’s Transformational Grammar. • Example of taking a linguistic theory and trying to implement it directly as a psychological theory of processing.

  7. Transformational Grammar • Two levels of syntactic structure, related by transformational rules; accounts for similarity in meaning across different strings. • Surface structure ((John) (picked __ (the box) up)). • Transformations Particle Movement • Deep Structure ((John) (picked up (the box))). • Phrase structure rules and lexical insertion rules used to construct DS

  8. Diff SS, Same DS Transformations can move, delete, or add words. Complex SS’s require a sequence of transformations • Wh-movement & Subj/Aux Inversion SS: Which book did Jim buy __? SS Jim did buy a book. DS: Jim did buy a/which book • Passivization SS: The beer was drunk by Jim. SS: Jim drank the beer. DS: Jim drank the beer.

  9. DTC • Determine the surface structure of the sentence • Reverse the transformations one by one • Recover the deep structure • Map DS to meaning

  10. Processing Unit is entire Sentence • Wh-movement & Subj/Aux Inversion SS: Which book did Jim buy __? SS: Jim did buy a book. DS: Jim did buy a/which book • Passivization SS: The beer was drunk by Jim. SS: Jim drunk the beer. DS: Jim drunk the beer.

  11. How would you test the DTC? • Early evidence seemed to support it. • Later evidence was problematic • No longer a viable theory of sentence comprehension.

  12. Memory & Language • Semantic Memory: LTM storehouse of conceptual knowledge • What is a cow? What is truth? • Lexicon: LTM storehouse of knowledge about words • Lexical Entry: For each word/morpheme, Spelling, pronunciation, syntactic category, pointer to semantic memory • Episodic Memory: LTM storehouse of our experiences

  13. uses is a is Spreading Activation Model (Collins & Loftus, 1975) • Length codes typicality • Link codes type of relation • Some redundancy

  14. Commmon Current Assumptions about Semantic Memory & the Lexicon • Semantic Memory is a network of concepts, organized by semantic similarity • Lexicon is a network of words, organized by phonological similarity • Interconnections link meanings to words

  15. Cognitive Neuroscience • Cognitive psychologists tend to talk about the architecture of the mind in terms of functionality • E.g., what is the input to word recognition? What is the output? • Ultimately, the mental operations described by cognitive psychology occur in the brain • In some cases, neuroscience can inform cognitive psychology • E.g., we may be able to learn about how words/concepts are represented by investigating activation patterns in response to different classes of words: action verbs activate motor cortex; perception verbs activate visual cortex

  16. Brain Anatomy (& Language) • Is language localized in the brain? • Is language lateralized?

  17. LH lobes Parietal lobe Frontal lobe Occipital lobe Temporal lobe Cerebellum

  18. Some Lg-relevant areas

  19. Video Clip (15 min) Brain Story: First among Equals • The first segment has been comparing human and chimp abilities to plan. They conclude that chimps can make and execute plans, but not as far into the future as humans. • We’ll watch a segment on LG • Aphasia patient • Mapping the brain prior to surgery

  20. What did you learn from the clip? • What does it mean to have aphasia? • Do aphasics recover language function? How? • What is the current view on the role of Broca’s and Wernicke’s areas?

  21. Schiff et al.(2005) • Do minimally conscious patients process speech?

  22. Averaged healthy data Figure 1. Functional maps obtained during listening to narratives Yellow= forward; Blue = backward Red = both Schiff, N. D. et al. Neurology 2005;64:514-523

  23. Figure 2. Volumes of activation during the passive listening tasks. The 2 patients are in blue and red; Averaged healthy activation in black. Schiff, N. D. et al. Neurology 2005;64:514-523

  24. Schiff et al. • paper was published shortly after the Terry Schiavo media/political frenzy. • How do we decide whether a minimally conscious patient is experiencing a life worth living? • Is language comprehension relevant? • Why don’t we care as much about how their brain responds to smells?

  25. Speech & Spoken Word Recognition

  26. Outline • Why is speech perception difficult for computers to do? • Problem of Invariance • How do humans do it so easily? • Bottom-up information (acoustic signal) • Top-down information (higher level context)

  27. What I said bookmark it Motorolla modem port a procedure and then stick it in the mail movie clips I might add Inscrutable the right or left What was transcribed book market motor roll a mode import upper seizure and dense thicket in the mail move eclipse I my dad in screw double the writer left Same sequences of phonemes form different words Different phonemes, at underlying level Automatic Speech Recognition Follies (David Pogue, NYTimes, 8/15/02)

  28. The Problem of Invariance • Individual phonemes do not have invariant acoustic cues. • There’s a lot of variability in the acoustic signal! • Variance in the acoustic signal has many sources: • Coarticulation • Differences among speakers • Differences within speakers: yelling/whispering, phone/in-person, etc. • If the bottom-up acoustic signal doesn’t provide consistent cues, how do we recognize phonemes/words?

  29. Reading a spectrogram formants Can you see invariant cues associated with /i/ ?

  30. If we can’t/don’t rely solely on bottom-up input, how do we recognize speech? • Perceive speech as (intended?) articulatory gestures, not as acoustic signal? • McGurk Effect • Sine Wave Speech: There are no essential acoustic properties that enable speech perception. Rather second-order changes in frequency and amplitude over time are responsible. sine wave speech demos • Use top-down information (word and sentence context) to complement bottom-up information • If so, when and how?

  31. Is Speech Special? Modularity Thesis (Fodor, 1983) • The mind is not a unified whole. In addition to central processes, there are specialized input-output modules • Central: decision making • Input: color perception, voice recognition • Output: throwing, touch-typing, articulation • Modules are fast, informationally-encapsulated, mandatory, exhibit characteristic breakdowns, and have shallow outputs. • Speech perception may be handled by a specialized input module

  32. The Essence of Motor Theory • Speech perception is grounded in our knowledge of speech production. We recognize phonemes by covertly re-creating the articulatory gestures. (Lieberman et al., 1967) • Consistent with philosophy that performance & perception are inextricably linked. • Assume innate, encapsulated phonetic module

  33. Spoken Word Recognition • Overcoming the problem of invariance in speech perception • Motor Theory • Top-Down Feedback: Word to Phoneme • TRACE • Cohort theory of Spoken Word Recognition

  34. Evidence for Top-Down influence on speech perception • Phoneme Restoration Effect (Warren, 1970) • Lexical bias in categorical perception task, e.g. dype vs. type (Clifton & Connine, 1987)

  35. TRACE(McClelland & Elman, 1986) • Interactive connectionist model • Nodes in network represent phonetic features, segments, & words • Feature nodes activated by consistent input • Activation spreads up through network & back down again • Predicts top-down effects

  36. Example: initial phoneme in “pick” is ambiguous betwn /b/ & /p/.(Lexical Bias)

  37. Is word recognition Automatic & Modular? Automatic Processes • Fast • Do not require attention • Feed-forward (can’t be guided, controlled, or stopped midstream) • Not subject to top-down feedback (informational encapsulation)

  38. Stroop Effect Name font color RED GREEN BLUE YELLOW GREEN What happens if you have to name word?

  39. Differences between spoken and written word recognition • For relatively short words, letters in a written word are processed in parallel • Eye movement data • Word superiority effect • Letter-Search Task • Spoken word unfolds across time • Can recognize some words before they are completely pronounced.

  40. Research on the Lexicon (Outline) • How are lexical entries accessed? (Word Recognition) What is the input? • Speaking (Ashcraft) • Reading • Listening • How is lexical ambiguity resolved?

More Related