420 likes | 545 Views
Brain, Psycholinguistics, & Cognitive Science. Outline. How does psycholinguistics fit within the umbrella of cognitive science? What do we know about language and the brain?. Inter-relationships. What do cognitive psychologists do?.
E N D
Outline • How does psycholinguistics fit within the umbrella of cognitive science? • What do we know about language and the brain?
What do cognitive psychologists do? Construct theories that describe how mental processing works in real time: • Within some domain, what are the mental representations and how are they manipulated? • How does information flow through the mind? • How is processing impacted by memory constraints, stimulus quality, mode of input, tasks, etc? Levels of processing Serial vs. parallel processing Top-down/bottom-up Automatic vs. strategic processing
What do (formal theoretical) linguists do? • Construct formal theories of our linguistic knowledge • Sets of rules/principles/assumptions for generating utterances • Criteria for a good theory • The rules generate all and only grammatical outputs (intuitions) • In the simplest way
Derivational Theory of Complexity • Miller’s clause-processing model of syntactic processing, motivated by Chomsky’s Transformational Grammar. • Example of taking a linguistic theory and trying to implement it directly as a psychological theory of processing.
Transformational Grammar • Two levels of syntactic structure, related by transformational rules; accounts for similarity in meaning across different strings. • Surface structure ((John) (picked __ (the box) up)). • Transformations Particle Movement • Deep Structure ((John) (picked up (the box))). • Phrase structure rules and lexical insertion rules used to construct DS
Diff SS, Same DS Transformations can move, delete, or add words. Complex SS’s require a sequence of transformations • Wh-movement & Subj/Aux Inversion SS: Which book did Jim buy __? SS Jim did buy a book. DS: Jim did buy a/which book • Passivization SS: The beer was drunk by Jim. SS: Jim drank the beer. DS: Jim drank the beer.
DTC • Determine the surface structure of the sentence • Reverse the transformations one by one • Recover the deep structure • Map DS to meaning
Processing Unit is entire Sentence • Wh-movement & Subj/Aux Inversion SS: Which book did Jim buy __? SS: Jim did buy a book. DS: Jim did buy a/which book • Passivization SS: The beer was drunk by Jim. SS: Jim drunk the beer. DS: Jim drunk the beer.
How would you test the DTC? • Early evidence seemed to support it. • Later evidence was problematic • No longer a viable theory of sentence comprehension.
Memory & Language • Semantic Memory: LTM storehouse of conceptual knowledge • What is a cow? What is truth? • Lexicon: LTM storehouse of knowledge about words • Lexical Entry: For each word/morpheme, Spelling, pronunciation, syntactic category, pointer to semantic memory • Episodic Memory: LTM storehouse of our experiences
uses is a is Spreading Activation Model (Collins & Loftus, 1975) • Length codes typicality • Link codes type of relation • Some redundancy
Commmon Current Assumptions about Semantic Memory & the Lexicon • Semantic Memory is a network of concepts, organized by semantic similarity • Lexicon is a network of words, organized by phonological similarity • Interconnections link meanings to words
Cognitive Neuroscience • Cognitive psychologists tend to talk about the architecture of the mind in terms of functionality • E.g., what is the input to word recognition? What is the output? • Ultimately, the mental operations described by cognitive psychology occur in the brain • In some cases, neuroscience can inform cognitive psychology • E.g., we may be able to learn about how words/concepts are represented by investigating activation patterns in response to different classes of words: action verbs activate motor cortex; perception verbs activate visual cortex
Brain Anatomy (& Language) • Is language localized in the brain? • Is language lateralized?
LH lobes Parietal lobe Frontal lobe Occipital lobe Temporal lobe Cerebellum
Video Clip (15 min) Brain Story: First among Equals • The first segment has been comparing human and chimp abilities to plan. They conclude that chimps can make and execute plans, but not as far into the future as humans. • We’ll watch a segment on LG • Aphasia patient • Mapping the brain prior to surgery
What did you learn from the clip? • What does it mean to have aphasia? • Do aphasics recover language function? How? • What is the current view on the role of Broca’s and Wernicke’s areas?
Schiff et al.(2005) • Do minimally conscious patients process speech?
Averaged healthy data Figure 1. Functional maps obtained during listening to narratives Yellow= forward; Blue = backward Red = both Schiff, N. D. et al. Neurology 2005;64:514-523
Figure 2. Volumes of activation during the passive listening tasks. The 2 patients are in blue and red; Averaged healthy activation in black. Schiff, N. D. et al. Neurology 2005;64:514-523
Schiff et al. • paper was published shortly after the Terry Schiavo media/political frenzy. • How do we decide whether a minimally conscious patient is experiencing a life worth living? • Is language comprehension relevant? • Why don’t we care as much about how their brain responds to smells?
Outline • Why is speech perception difficult for computers to do? • Problem of Invariance • How do humans do it so easily? • Bottom-up information (acoustic signal) • Top-down information (higher level context)
What I said bookmark it Motorolla modem port a procedure and then stick it in the mail movie clips I might add Inscrutable the right or left What was transcribed book market motor roll a mode import upper seizure and dense thicket in the mail move eclipse I my dad in screw double the writer left Same sequences of phonemes form different words Different phonemes, at underlying level Automatic Speech Recognition Follies (David Pogue, NYTimes, 8/15/02)
The Problem of Invariance • Individual phonemes do not have invariant acoustic cues. • There’s a lot of variability in the acoustic signal! • Variance in the acoustic signal has many sources: • Coarticulation • Differences among speakers • Differences within speakers: yelling/whispering, phone/in-person, etc. • If the bottom-up acoustic signal doesn’t provide consistent cues, how do we recognize phonemes/words?
Reading a spectrogram formants Can you see invariant cues associated with /i/ ?
If we can’t/don’t rely solely on bottom-up input, how do we recognize speech? • Perceive speech as (intended?) articulatory gestures, not as acoustic signal? • McGurk Effect • Sine Wave Speech: There are no essential acoustic properties that enable speech perception. Rather second-order changes in frequency and amplitude over time are responsible. sine wave speech demos • Use top-down information (word and sentence context) to complement bottom-up information • If so, when and how?
Is Speech Special? Modularity Thesis (Fodor, 1983) • The mind is not a unified whole. In addition to central processes, there are specialized input-output modules • Central: decision making • Input: color perception, voice recognition • Output: throwing, touch-typing, articulation • Modules are fast, informationally-encapsulated, mandatory, exhibit characteristic breakdowns, and have shallow outputs. • Speech perception may be handled by a specialized input module
The Essence of Motor Theory • Speech perception is grounded in our knowledge of speech production. We recognize phonemes by covertly re-creating the articulatory gestures. (Lieberman et al., 1967) • Consistent with philosophy that performance & perception are inextricably linked. • Assume innate, encapsulated phonetic module
Spoken Word Recognition • Overcoming the problem of invariance in speech perception • Motor Theory • Top-Down Feedback: Word to Phoneme • TRACE • Cohort theory of Spoken Word Recognition
Evidence for Top-Down influence on speech perception • Phoneme Restoration Effect (Warren, 1970) • Lexical bias in categorical perception task, e.g. dype vs. type (Clifton & Connine, 1987)
TRACE(McClelland & Elman, 1986) • Interactive connectionist model • Nodes in network represent phonetic features, segments, & words • Feature nodes activated by consistent input • Activation spreads up through network & back down again • Predicts top-down effects
Example: initial phoneme in “pick” is ambiguous betwn /b/ & /p/.(Lexical Bias)
Is word recognition Automatic & Modular? Automatic Processes • Fast • Do not require attention • Feed-forward (can’t be guided, controlled, or stopped midstream) • Not subject to top-down feedback (informational encapsulation)
Stroop Effect Name font color RED GREEN BLUE YELLOW GREEN What happens if you have to name word?
Differences between spoken and written word recognition • For relatively short words, letters in a written word are processed in parallel • Eye movement data • Word superiority effect • Letter-Search Task • Spoken word unfolds across time • Can recognize some words before they are completely pronounced.
Research on the Lexicon (Outline) • How are lexical entries accessed? (Word Recognition) What is the input? • Speaking (Ashcraft) • Reading • Listening • How is lexical ambiguity resolved?