Announcements

1. Announcements • Next few lectures • Require some syntactic knowledge • Review Chapter 2’s Syntax Section • Readings • Original Articles  Greater difficulty level • Read in order as stated in syllabus. • Statistics knowledge? • Sample exam questions • This week (Friday): I will post a few Qs in our discussion forum. • Next week (Thursday): You will submit your Qs into dropbox

2. Psy1302 Psychology of Language Lecture 9Models of Speech Recognition

3. Continuation of Last Lecture…Outline • We are fast at speech recognition. • How do we achieve speed? • Parallel Activation • Constrained by contextual Effects • Terminologies and Ideas • Two Classic Models • Cohort Model • TRACE Model [and many experimental paradigms and findings]

4. Top-down Example 1Last time: Shadowing and Corrections Intended Mispronunciation Feature • narrow marrow place • detrimental tetrimental voicing • perfectionistic berfectionistic voicing • lives rives place • back mack manner • hampered kampered place & manner • take nake manner • self zelf voicing • comfort vomfort all three

5. Terminologies Bottom-Up vs. Top-Down Processing • Bottom-up: Processing that is stimulus or data-driven. • Top-down: Processing that involves the use of knowledge obtained from higher-level sources

6. Top-down Examples 2Lexical Influence on Phoneme Perception • Ganong (1980) • Splice speech waves • /d/ to /t/ + /æsk/ dask-task • /d/ to /t/ + /æš/ dash-tash • Obtained % of /d/ identification • Two possible outcomes: • No Effect of Lexical Knowledge • Effect of Lexical Knowledge

7. Top-down Examples 2Lexical Influence on Phoneme Perception nonword-word: dask-task word-nonword: dash-tash 100 % identification as /d/ 0 short VOT (d) long VOT (t)

8. nonword-word: dask-task word-nonword: dash-tash 100 % identification as /d/ 0 short VOT (d) long VOT (t) Top-down Examples 2Lexical Influence on Phoneme Perception • Ganong (1980) • Lexical knowledge influence perception • Only able to shift AMBIGUOUS phones and not those at the ends of continuum

9. Top-down Examples 3Phoneme Restoration Effect Warren (1970) & Warren & Warren (1970): • “The state governors met with their respective legiSlatures convening in the capital city” • S replaced with cough or noise and played to listeners • Then asked listener to figure out where the sound was replaced. • What happened?

10. Top-down Examples 3Phoneme Restoration Effect Warren (1970) & Warren & Warren (1970): It was found that the *eel was on the orange. It was found that the *eel was on the axle. It was found that the *eel was on the shoe. It was found that the *eel was on the table. It was found that the *eel was on the fishing-rod. http://www.asj.gr.jp/2006/data/kashi/index.html http://www.acsu.buffalo.edu/~bmb/Courses/Old-Courses/PSY341-Fa2003/Exercises/Phon-rest/phon-rest.html

11. Gating Task(Grosjean 1980) • Cumulative fragment of speech played. • Measure how much from the onset of word participants need to hear before identifying it. • RECOGNITION POINT = earliest “gate” at which the participant picks the correct response and maintains it for the rest of the trials. 367 ms 300 ms 50 ms 100 ms 150 ms 200 ms 250 ms

12. Top-down Examaple 4Gating Task (Grosjean 1980) • Compare word in isolation and in context. • In isolation: “camel” • In context: “The kids went to the zoo and rode on the camel” • Recognition Point: In Isolation ~333 ms In context ~199 ms Isolation 367 ms 300 ms 50 ms 100 ms 150 ms 200 ms 250 ms Context 300 ms 50 ms 100 ms 150 ms 200 ms 250 ms

13. Top-down Example 5Word Monitoring(Marslen-Wilson, Brown, & Tyler, 1988) • Listening to sentences & monitoring for specific words • Word in isolation: ~300 ms • Normal: The boy held the guitar. ~ 240 ms. • Discourse Incongruence: ~235 ms. • Pragmatic Anomalous: The boy buried the guitar. ~ 268 ms • Semantic Anomalous: The boy drank the guitar. ~291 ms • Categorical Anomalous: The boy slept the guitar. ~320 ms

14. Speech Recognition • How do we achieve speed? • Parallel search • I.e. Activation of potential candidates in parallel • Consult contextual information • Use of contextual information to select or weed out candidates!

15. Models that consider contextual information • Examine 2 influential models of speech processing (evolved from Forster & Morton’s) • Cohort Model • TRACE Model • Currently other existing models in the literature.

16. Subtext • How might psychology experiments • inform us of our mental processes • help us create models of our mental representations and of how our mind process information? • be designed to help us distinguish between models or help us revise an existing one?

17. Subtext • In evaluating any model, consider: • How well does the model account for existing experimental findings? • Is the representation depicted in the model an intuitively plausible one? • Does the model make predictions that are not in fact borne out by available empirical (i.e. observational and/or experimental) evidence?

18. Cohort ModelMarslen-Wilson and Welsh (1978) INTEGRATION STAGE (in which the semantic and syntactic properties of the chosen words are utilized) SELECTION STAGE (the most likely candidate is chosen from cohort) ACCESS STAGE (perceptual representation used to activate lexical items, thus generating a candidate set of items – the cohort) Input

19. S song story sparrow saunter slow secret sentry Cohort Model – Access StageMarslen-Wilson and Welsh (1978) ... ... (i.e., words beginning w/ the sound heard so far)

20. SP spice spoke spare spin splendid spelling spread Cohort Model – Access StageMarslen-Wilson and Welsh (1978) ... (candidates that no longer fit the incoming stream, are eliminated)

21. SPI spit spigot spill spiffy spinaker spirit spin Cohort Model – Access StageMarslen-Wilson and Welsh (1978) ...

22. SPIN spin spinach spinster spinaker spindle Cohort Model – Access StageMarslen-Wilson and Welsh (1978)

23. SPINA spinach Cohort Model – Access StageMarslen-Wilson and Welsh (1978)

24. SPINA spinach Cohort Model – Access StageMarslen-Wilson and Welsh (1978) word uniqueness point • Note: Some words have no uniqueness point (e.g., “spin”)

25. Cohort Model – Access StageMarslen-Wilson and Welsh (1978) • Uniqueness point • Recognition point • Highly Correlated. • Support idea of cohort.

26. Cohort Model • Auditory Lexical Decision. • Uniqueness point + 450 ms constant for responding “NO, It’s not a word.”

27. Cohort ModelMarslen-Wilson and Welsh (1978) INTEGRATION STAGE (in which the semantic and syntactic properties of the chosen words are utilized) SELECTION STAGE (the most likely candidate is chosen from cohort) ACCESS STAGE (perceptual representation used to activate lexical items, thus generating a candidate set of items – the cohort) Input

28. Cohort Model – Access StageMarslen-Wilson and Welsh (1978) • Selection stage: Making use of contextual effects to achieve speed. Contexts: • All the information not in the immediate sensory signal. • E.g., Information from previous sensory input (prior context) to higher knowledge sources (e.g., lexical, syntactic, semantic, and pragmatic info). • One big Q: • Which contextual effects are helpful?

29. Cohort Model – Access StageMarslen-Wilson and Welsh (1978) Another BIG Q : • When do/can we consider contextual information? • Generation vs. Selection • Proposal vs. Disposal • Pre-lexical or Post-lexical • How do we address the when Q experimentally?

30. Zwitserlood (1989) • Crazy complicated classic experiment. • Involves 3 separate groups of participants • Involves Sentence Completion Task. • Determines the Strength of Contextual Information • Involves Gating Task. • Determines Probe Positions on the PRIME word. • Involves Cross-Modal Priming. • Determines whether CAPITAIN primes BOAT and MONEY (semantically related to CAPITAL) at various probe positions (i.e. points in time).

31. BOOT GELD Cross-Modal Priming KAPITAAL KAPITEIN KAPITEIN Hear Prime: Varying position of when to do lexical decision Lexical Decision: or “BOAT” “MONEY”

32. What is the strength of the context? (sentence completion) What’s a good continuation for: • They mourned the loss of their _______. • With dampened spirits the men stood around the grave. They mourned the loss of their _______. • Classify Responses of Participants into: • Biasing contexts: • 16%-33% said the prime word and 0% said prime competitor. • Neutral contexts: • 0% said prime word, and 0% said prime competitor.

33. Where to Probe for Activation? (Gating Task) • Isolation Point: 1st time 50% of the participants pick the correct word and sticks with it to the end. PROBE POSITIONS • Position 0: Onset of word • Position 1: Isolation Point with Biasing Context • (ave. 130 ms after onset) • Position 2: Isolation Point with Neutral Context • (ave. 199 ms after onset) • Position 3: Isolation Point in Carrier Phrase • The next word is ____. (ave. 278 ms after onset) • Position 4: Recognition Point w/ Carrier Phrase • (ave. 410 ms after onset)

34. TASK Hear: CAPTAIN Lexical Decision: BOAT or MONEY GRAPH LEGEND BOAT – solid line MONEY – dashed line When does context play a role? (Four Possible Outcomes) Before word spoken During lexical access During selection phase At post-lexical integration stage

35. Context plays a roleBEFORE word spoken BOAT MONEY C A P T A I N 

36. Context plays a role DURING lexical access BOAT MONEY C A P T A I N 

37. Context plays a role DURING selection phase BOAT MONEY C A P T A I N 

38. Context plays a roleAT POST-LEXICAL integration BOAT MONEY C A P T A I N 

39. Comparing Data to Predictions • Zwitserlood’s prediction slides plots level of activation vs. time. • Her data is in terms of reaction time vs. probe positions (~time). • How do we compare the two? • Assumption: Faster reaction = higher level of activation

40. Results BOAT Reaction Time (ms) MONEY C A P T A I N 

42. Cohort ModelMarslen-Wilson and Welsh (1978) INTEGRATION STAGE (in which the semantic and syntactic properties of the chosen words are utilized) Interactive SELECTION STAGE (the most likely candidate is chosen from cohort) Interactive ACCESS STAGE (perceptual representation used to activate lexical items, thus generating a candidate set of items; the cohort) Autonomous Input

43. Terminologies Some Terminologies • Serial vs. Parallel • Bottom-up vs. Top-down • Autonomous vs. Interactive • Autonomous: stage of processing proceeds independently of information from other processing modules • Interactive: stage of processing quickly considers information from other processing modules as info comes in • Incremental: structuring and interpreting information as it comes in

44. Problem for Cohort Model • If you set up the wrong cohort, how do you recover? • e.g. dragedy for tragedy • Misalignment problem ThesKyisfalling! The skyis falling! or Thisguyis falling!

45. Revised Cohort Model(Marslen-Wilson (1987) • Still set up an initial cohort of candidates. • Elimination process is no longer all-or nothing. Items that do not receive further positive information decay in activation rather than being eliminated • Allows backtracking for misheard/distorted words • Context loses some of its power, as it cannot be used to influence the items that form the initial cohort. • A recognized word has a higher relative activation than other words in the cohort.

46. TRACE Model(McClelland, Elman, Rumelhart’86) • Model used for other things…

47. Digression: Connectionist Networks Connectionist Models A NEURON NETWORK OF NEURONS Connections can be either inhibitory or excitatory. http://www.cheshireeng.com/Neuralyst/nnbg.htm

48. Digression: Connectionist Networks Properties of Connectionist Unit Activation Level = w1*A1 + w2*A2 + ...... + w8*A8 where -1 £ wn£ +1

49. Digression: Connectionist Networks Squashing/Threshold Function If Activation Level < 0.5 Output = 0 If Activation Level  0.5 Output = 1

50. Digression: Connectionist Networks Network of Connectionist Units