What do we need to do, to understand an utterance?

1. What do we need to do, to understand an utterance? Cartoon-head figures from Jackendoff (1994), Patterns in the Mind

2. Speech Perception • What are the phonemes? • Where are the word boundaries? Segment the auditory stream into words, made up of particular phonemes

3. Word Recognition Recognize individual words, resolving ambiguities • Meaning Chris walked near the bank.  • Syntactic category (noun, verb, etc.) She saw her duck. Buffalo buffalo buffalo buffalo.

4. Syntactic Analysis • Determine the structure of the sentence • Constituents • Hierarchical structure • Buffalo buffalo buffalo buffalo. • Put the ball in the box on the table.(ball in the box) on (the table) (ball) in (the box on the table)

5. Semantic & Pragmatic Analysis • What does the sentence mean? • Who did what to whom? • Truth conditions • What is the speaker trying to convey? • Can you pass the salt? 

6. The Big Question: How Do We Accomplish Linguistic Communication? Jackendoff (1994), Patterns in the Mind

7. Map sounds to stored representations? • A bird was in the tree yesterday. • Are there any birds in the tree? • A bird might be in the tree. • That tree looks like a bird. What do we need?

8. Linguistic Knowledge as Categorical Rules for structure-building Jackendoff (1994), Patterns in the Mind

9. Related Questions • How do those categorical rules map onto cognitive processes? • Are the same cognitive processes used to produce speech as to understand it? • Does modality (reading/listening) matter? • Are our brains specialized for language? • If I form a hypothesis about language understanding/production, how would I test it? • What would count as data? • Let’s give it a shot…

10. History • Philosophical interest in language processing & language acquisition goes back to ancient times • E.g., Aristotle on relations among thought, language, & external world • Modern experimental approach quite new (most paradigms developed within last 50 years) • Modern theories reflect contributions from modern linguistic theory, cognitive psychology, computer science, & cognitive neuroscience • Human language is unique among animal kingdom

11. Early Psychological study of Language • Wundt (1832-1920) • One of psychology’s founding fathers • Primarily used introspection for studying mental behavior, but one of first to use RT • Published on language in 1911

12. Wundt’s (1911) hypotheses about Language • Sentence (defined intuitively) is the primary unit of language • “Leave!” is one; “Days of the week” is not • “I filled the water with bottle.” • Production converts a thought into a sequential string of sounds • Comprehension is simply the reverse

13. What’s wrong with Wundt’s Approach? • Is introspection a reliable, replicable, objective tool for science? • Do Wundt’s hypotheses lead to any clear predictions about behavior? • Constructs (e.g., sentence) not carefully defined • The stimuli/inputs for production and the response/outputs for comprehension are neither well-defined nor directly observable. • How can one develop a hypothesis and test it experimentally?

14. Would a different approach be more productive? • Need to develop hypotheses that lead to clear predictions about behavior • Link environmental conditions to observable behavior • Need experimental techniques that can be clearly described and replicated in different laboratories.

15. Behaviorism Dominant paradigm in psychology 1927-1960. (Skinner, Pavlov) • How often does a behavior occur and with what intensity? • All behavior shaped by the environment using classical and operant conditioning. • No mental representations • Introspection devalued

16. Skinner’s (1957) Verbal Behavior • Language is a difficult phenomena for a behaviorist account. • In 1934, at a dinner party, philosopher A. N. Whitehead challenged Skinner to “account for my behavior as I sit here saying ‘No black scorpion is falling upon this table.’” • Skinner began the book the next morning, and spent 20+ years working on it. • Skinner often called this book his most important work

17. Skinner’s (1957) Verbal Behavior • Emphasis on production, rather than comprehension • A sentence is a chain of associative links, “like beads on a string” There – is – no – black – scorpion… • Speech is learned response to environmental stimuli (reinforcement, punishment)

18. Experimental Evidence • Speech is learned response to environmental stimuli (reinforcement, punishment) • Use of plural nouns increase if reinforced with “mmm-hmm” (Greenspoon, 1954, 1955) • Proportion of opinion statements increase if paraphrased/agreed (Verplank, 1955)

19. Language research during the Reign of Behaviorism • Operant studies (e.g., Verplanck) • Classical Conditioning experiments (e.g., Stroop) • Practical research, much of which was funded by the defense department • George Miller: understanding speech in noisy radio transmissions

20. George Miller’s Lab • Interested in speech and hearing • Trained as a behaviorist in the 1940s • In 1950’s investigated radio-based communication • How high does the signal-to-noise ratio need to be, for adequate transmission of the message? • Amount of noise • Characteristics of the message • Characteristics of the speaker

21. How do we (the military) insure adequate transmission of message? One strategy: Limit the vocabulary/possible messages • Digits are easy: 0-9 have 8 different nuclear vowels (only 5 and 9 share their vowel) • Nonsense syllables are opposite extreme—need to hear each phoneme clearly

22. George’s Ground-Breaking Findings Miller et al. (1951) Miller & Selfridge (1950) demonstrated analogous pattern in free-recall test. Why are words in sentences easier to perceive and easier to remember than words in lists?

23. What does “sentence-advantage” mean? Miller et al. (1951) maintain that sentences effectively restrict the number of alternative words, similarly to small vocabularies. “In 1951, I apparently still hoped to gain scientific respectability by swearing allegiance to behaviorism. Five years later, inspired by such colleagues as Noam Chomsky and Jerry Bruner [a social psychologist], I had stopped pretending to be a behaviorist.” (Miller, 2003)

24. Miller (1962) re-examines Miller et al. (1951) • Words in a sentence are not as distinct as words in isolation… • Less carefully pronounced (splice-test) • Words run together • Why is there no extra cost for these? • Speech rate of 2-3 words per second leaves little time for deducing set of alternatives after each word • “Reduction of alternatives” explanation is inadequate

26. Miller (1962)

27. Miller (1962)

29. Evolution of Speech and Language August 4, 2009

30. How did human vocal tract evolve? All mammals produce vocal sounds in essentially the same way… Source – Oscillator (voicing) – Filter (formants) YouTube - vocal tract model synthesis YouTube - Vocal formants In human speech, formants are the most informative parameter. They make speech intelligible. • E.g., whispered speech lacks voicing and pitch, but has normal formants Human speech requires fine, rapid motor control during articulation

31. Role of formants in animal communication Primates & birds perceive formants as accurately as humans • Individual identification via vocal signature • Provide cues to body size of “speaker”

32. Diff’s btwn ape & human vocal tract 1. Human larynx lowers in throat during 1st yr of life ·Allows more tongue movement, for broad range of discriminable formant patterns ·Lowers formant freq-- impression of larger size 2. Human oral cavity shorter, nasal cavity bigger 3. Humans lack laryngeal air sacs • Little known about function

33. Vocal Imitation • Except for humans, primates are poor at this. • Apes raised like human kids • Monkeys raised with other species • Little evidence for learned vocal behavior • Humans clearly learn language(s) • Human whistling • Human bird calls • Human imitation of animal noises

34. Vocal Imitation • Whales, seals and dolphins are somewhat better than most primates. • Whales learn their songs • Passerine Birds are terrific at this, even cross-species • Songbirds learn their songs • Mockingbirds learn other species songs, as well as environmental sounds (insects, car alarms, etc.) • Parrots can mimic human speech and even specific voices • Irene Pepperberg has trained African Grey Parrots to use human speech communicatively

35. Primates are poor candidates for production of spoken language • Lack of rapid, fine motor control of vocal articulation • Structure of the vocal tract • Limited ability for vocal imitation

36. Ape Language studies (1950’s to present) • No luck training chimpanzees to produce spoken language • Some success with manual/visual “languages.” • Chimpanzees, gorillas, & bonobos approximate linguistic skill of a 3-yr old human • Ceiling on lg potential?

37. Koko (gorilla.org) Gorilla trained in sign language by Penny Patterson Video Clip Has Koko acquired a language? What evidence is necessary to answer this question?

38. Summing Up: How is human language special? • Vocal tract anatomy • Vocal imitation • Rapid, fine motor control of ariculators • Creative recombination of phonemes, morphemes, words for expression of nearly any thought • But what about Koko? • Compare Koko to Nicaraguan deaf kids

39. Spontaneous emergence of Nicaraguan Sign Language • Clip from “Birth of a Language” • How is this signed communication the same/different from Koko’s? • What kind of tests would you need to conduct to compare them?

40. Special Features of Human Speech • Specialized vocal tract: Broad range of formants for producing many distinct sounds • Vocal imitation/Social Learning • Rapid, fine articulation • Hierarchical structure • Rule Learning

41. Hierarchical Structure of Lg Human speech has hierarchical structure, which is necessary to produce utterances of arbitrary complexity. Structure is distinct from content (specific phonemes). Syllable = (onset) + rhyme Rhyme = nucleus + (coda) Onset = one or more consonants Nucleus = one or more vowels Coda = one or more consonants

42. Compositionality and the Rate of Data Transmission • Small set of phonemes can be recombined very productively (but in a constrained way) to form morphemes. • Morpheme = one or more syllables (meaning unit) • Signed language morphemes are also made up of “phonological” constituents (e.g., hand shape, movement, location) • Morphemes can be combined productively (but constrained) to create words. • (prefix) + stem + (suffix) • Stem = (prefix) + stem + (suffix) • Words can be combined productively to create utterances.

43. Hierarchical Structure of Lg Syntactic Hierarchies & Center-embedding • The man read Chaucer. • The man who the woman despised read Chaucer. • The man who the woman the children loved despised read Chaucer.

44. What are the preconditions for learning hierarchical structure? 1. Fixed sequences (linear order): • Idioms & stock phrases (once upon a time) are fixed word sequences • Words are fixed phoneme sequences 2. Statistical Learning is probably important for: • discovering words in speech stream • identifying syntactic category and subcategory of words • resolving lexical and syntactic ambiguity • Within phrases (e.g. NP), there is a predictable ordering of categories (e.g., the predicts a noun in the next word or two) Predictive constraints on sequences may allow us to learn hierarchical relationships

45. Coding the probabilities of sequences aids word segmentation prior to lexical knowledge Jenny Saffron & colleagues Marc Hauser & colleagues

46. Shared neural underpinnings to syntax & sequence learning? • Broca’s aphasics who have severe syntactic deficits also exhibit deficits in sequence learning (Christiansen et al., 2001 unpubl) • Incongruent musical sequences elicit P600’s, just like syntactic anomalies (Patel et al., 1998) • MEG shows that Broca’s area is involved in processing music sequences (Maess et al., 2001) • All higher organisms must learn about sequential events. How does human sequential learning compare with that of other primates?

47. “The rat the cat the dog bit chased died.” What limits the kinds of rules that Tamarins can learn about sequences? Fitch & Hauser (2004) suggest that Tamarin’s can master Finite State Grammars, but not Phrase Structure Grammars

48. Types of Grammars All human languages allow for an infinite number of different utterances. What kinds of grammars allow this? Finite State Grammars: A finite number of states (e.g., words, calls, syntactic categories), with rules for getting from one state to the next.

49. boring sucks class the me disappoints easy satisfies an A+ really FSG’s provide rules for concatenation

50. Types of Grammars A phrase structure grammar allows for long distance dependencies. (Last year, (Demi Moore (took (that cute dumb guy who’s about 20 years old from “That 70’s Show”) out for a while))). NP took out NP. NP took NP out. NP = NP + PP NP = NP + S The intervening NP can have an arbitrary amount of internal complexity