Talking heads!4/10

1. Talking heads!4/10 Easy and hard problems Artificial intelligence Turing test Loebner prize The real thing: human language comprehension

2. Overview of comprehension

3. Overview of comprehension Vast database lexicon • (production runs the arrows the other way- creating a tree that determines movements) Local context Semantic Representation (labeled phrase “tree” with Morphemes) interpretation parsing

4. detailed sketch

5. The waveform of the utterance • Continuous movements convey discrete thoughts.

6. Lexicon • function morphemes • Content words (nouns, verbs, adjectives…) • Word formation rules • Example: (adj -->(prefix)+V+ -able) unfixable • Other info on morphemes: phonology, syntax, meaning and use

7. Sketch of the lexicon

8. Parsing -- computing a structural description (labeled tree) using: • Acoustic cues in time-pressure wave • Function words and inflections • Lexical guidance • Word and phrase order cues • Determines grammatical relations and input to semantics (recall the aphasic woman given “The bird that the cat watched was hungry.”

9. Lexical guidance, inflections and parsing -1 • In languages with I. E. inflections, subject and object NPs would be indicated directly and thus the syntactic role of an NP is clear. In English only the pronouns retain these inflections. Compare the following she/her/her and he/his/him. • In the following examples, we see that the English loss of inflections and optional that puts an extra load on a word-order dependent parser. Notice that if that -- especially unstressed -- follows directly after the verb, it predicts a complement clause.

10. What is “lexical guidance?” • A general term for the role lexical information plays in building trees containing that morpheme. • Lexicon gives possible or required role players for verbs. • Recall discussions of “eat” and “dine” from Pinker. • “Leave” from “The dog barked at the girl that Otto wanted to ()leave()().”

11. Lexical guidance, inflections and parsing -2 • Some verbs take only simple NP objects (a) • 11a- The doctor visited the child/her/*she. • others take complement clause NPs (b) • 11b- The doctor insisted the child/*her/she/take the pill.

12. Lexical guidance, inflections and parsing -3 • Some take either (c). • 11c- 1. The doctor remembered the child/her.) • 11c- 2. The doctor remembered the child/she/ had an allergy.

13. The semantic representation of the utterance • Much of the conventional meaning of an utterance derives from its hierarchical structure and morpheme meanings or “senses.” • This is the assumption of combinatorial or compositional semantics.

14. The listener's interpretation of the utterance is based on: • A Semantic representation of the sentence • Local context • This includes determining referents of referring expressions, antecedents to anaphors and definite descriptions, nonverbal cues, etc. • The "vast database” • Anything else!

15. Reference and local context Reference and co-reference Significant aspects of meaning come from the context of the utterance. These include deictic words (e.g. now, this, here) and other pronouns, as well immediately preceding conversation in “working memory.” Gaps and traces (Pinker, p.219;482) The policeman saw the boy that the crowd at the party accused (TRACE) of the crime.

16. Complex sentences - ambiguity Example 1 can be interpreted as a relative (2) or complement clause (3) 1. “ The fact that Otto knew was surprising.” 2. “The fact that Otto knew () was surprising.” (Otto knew some fact that was surprising. Note the gap in 2 but not in 3 below.) 3.”(The fact) that Otto knew was surprising.” (A complement clause names explicitly the surprising fact, namely “that Otto knew (something) was surprising.”

17. Complex sentences - ambiguity Multiple gaps allow multiple interpretations “The girl that Bill wanted () to leave () wore a blue dress.” Is “that” coreferent with the first or second gap -- the subject or object of leave? Compare “The girl that Bill wanted () to leave Sam wore a blue dress.”

18. Acquisition video examples When did the boy say () that he hurt () himself? What do you think Cookie Monster eats ()? You think Cookie Monster eats (what)? *What do you think what's in here?

19. Complex sentences - “idea density” Packaging several propositions (sentences) into one sentence increases complexity of processing. The most extreme case is: The player kicked the ball kicked him. The player (that was) thrown the ball kicked him. The player kicked the ball (that was) thrown him.

20. Many aphasics cope by using several simple reduced complexity sentences. Kemper et al (1997) report low idea density predicts Alzheimers disease decades before other symptoms!. Complex sentences “idea density” 2

21. “Vast database” contributions to meaning Inference -- going beyond the given propositions -- is part of comprehension. Pinker: She: “I’m leaving.” He: “Who is he?” “In 1950, I was the tallest kid in sixth grade.”

22. “Vast database” 2 Some inferences are “presuppositions” -- implicit statements assumed true by speaker and inferred by listener. “Bill knows that the world is flat.” “When did you stop drinking?”

23. “The earthquake destroyed all the buildings in the town except the mint.” “Her gift is too tall for her bedroom.” She fell off the first/top rung of the ladder. (primes ok/dead) Our brain tends to activate likely consequences as it comprehends L Vast database 2.5 - inferences from anywhere based on the semantic interpretation of the sentence

24. “vast database” 3 “sportugese” - Sportsfans predict scores from verbs better than non-fans? No kidding! “Lakers crush Celtics” “Celtics hold off Timberwolves” What is the margin of victory?

25. Read and summarize: "The procedure is actually quite simple. First you arrange things into different groups depending on their makeup. Of course one pile may be sufficient depending on how much there is to do. If you have to go somewhere else due to lack of facilities that is the next step, otherwise you are pretty well set. It is important not to overdo any particular endeavor. That is, it may not seem important, but complications from doing too many can easily arise. A mistake can be expensive as well....." “vast database” 3.5

26. “vast database” 4 For most of us, this is a parsable text, using known words, but with little overall meaning unless we were given a title "Washing Clothes." Then it becomes meaningful and much more memorable as Bransford and Johnson, (1972) demonstrated. Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior, 61, 717-726.

27. Do hemispheres comprehend differently? The LH does much of the language processing in comprehension. However there is some suggestion the RH gives a unique perspective.

28. RH contribution to comprehension? Using similar materials, with and without titles, St George et al (1999) show differential hemisphere involvement -- the RH working hardest when there are no titles.

29. RH solves the puzzle better • St George, M., Kutas, M., Martinez, A., & Sereno, M. I. (1999). Semantic integration in reading: engagement of the right hemisphere during discourse processing. Brain, 122(7), 1317-1325.

30. “vast database” 5 The listener’s perception of the purpose of the utterance influences one’s interpretation of it. "I now see that my husband was simply engaging the world in a way that many men do: as an individual in a hierarchical social order in which he was either one-up or one-down… conversations are negotiations. Life is a contest, a struggle to preserve independence and avoid failure.. I, on the other hand, was approaching the world as many women do: as an individual in a network of connections.. conversations are negotiations for closeness… they try to protect themselves from others' attempts to push them away.." p.331 712 notes. (Deborah Tannen, 1990)