Yuki Kamide, Gerry T.M. Altman, and Sarah L. Haywood (2003)

1. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements Yuki Kamide, Gerry T.M. Altman, and Sarah L. Haywood (2003)

2. Incremental processing • The incoming inputs are processed incrementally on a word-by-word basis; relevant constraints are applied in parallel to the analysis of the input as it unfolds.

3. Prediction of thematic roles • The processor can anticipate thematic role assignments drawing on different sources of information: • Lexical information about the verb coupled with discourse-based information about available entities (Boland et al., 1995; Altmann, 1999) • E.g., Which preschool nursery/military base did Hank deliver the machine guns to _ last week? • Slower reading times in the ‘preschool nursery’ condition

4. Prediction of thematic roles • Selectional restrictions (Altmann & Kamide, 1999) • E.g., The boy will eat the cake. The boy will move the cake. • More anticipatory eye-movements to the target in the selective condition.

5. Overview of the study • The present study explores the extent to which the incremental analysis of a sentence can lead to the assignment of thematic roles in advance of linguistic input at which that assignment is unambiguously signaled. • Verb-based information (in combination with a pre-verbal argument) in English (Experiments 1 and 2) • In the absence of the verb, morphosyntactic and semantic constraints extracted from pre-verbal arguments in Japanese (Experiment 3)

6. Experiment 1 • Does the processor anticipate information pertaining to the Goal argument? • Can anticipatory eye movements be found during an expression that refers to a different object in the scene? • Can anticipatory eye movements be obtained even if there is no explicit task other than to look and listen?

7. Inanimate goal Animate goal Experiment 1 Animate Goal Agent Theme Inanimate Goal Distractor

8. Methods • 64 subjects • 18 experimental pictures each paired with the animate condition and the inanimate condition • SMI EyeLink head-mounted eye-tracker

9. Region 2: More anticipatory eye-movements towards the appropriate Goal Results Region 1: No evidence of anticipatory eye movements towards the Goal

10. Discussion • The processor can anticipate Goal arguments even during reference to some other object in the scene in a ‘look and listen’ task. • Some 3-place Vs with an animate Goal allowed an alternating constituent order casting doubt on whether the processor anticipated the Goal to be referred in the 1st or in 2nd post-verbal position. • It is not sure whether the verb alone, or the combination of the verb with its direct object led to the anticipatory eye movements.

11. Experiment 2 • Can the arguments of a verb be predicted on the basis of combinatory information derived from the semantics of the Agent in combination with the verbs’ selectional restrictions? • The boy ate … vs. The cat ate …

12. Experiment 2

13. Method • 64 subjects • 24 scenes with 4 sentential conditions

14. Analyses

15. Results Combinatory effects: More looks to the motorbike in the man ride condition than in the girl ride condition (Regions 1 and 2) and in the man ride condition than in the man taste condition (Region 2)

16. Discussion • The semantic properties of a forthcoming Theme are predicted on the basis of the combination of information about the Agent and about the verb. • Relatively small number of looks to the target objects  overt shifts in attention as evidenced by eye movements may underestimate the true extent of attentional shifts • Does the prediction of a verb’s arguments have to be based on the verb itself?

17. Experiment 3 • In Japanese, all arguments of the verb appear prior to the verb and each argument is case-marked. • E.g., syoojo-ga neko-ni sakana-o yatta. girl-nom cat-dat fish-acc gave • Is there pre-head (pre-verb) prediction in Japanese?

18. Experiment 3 Dative condition Accusative condition Prediction: More anticipatory looks towards the hamburger in the Dative condition than in the Accusative condition.

19. Method • 24 native speakers of Japanese • 16 experimental scenes

20. Results Significantly more looks to the Target in the Dative condition than in the Accusative condition during the adverb

21. Discussion • Prediction of forthcoming arguments is possible even in the absence of the grammatical head. • This prediction was in part based on syntactic information regarding case-structure in Japanese. • ?? Combinatory information derived across different NPs • Evidence against head-driven parsing accounts; support for incremental pre-head attachment. • Preference for analyzing ‘NP-dat’ as the Goal of a 3-place verb over analyzing it as the Theme or Goal of a monotransitive verb • Sensitivity to the statistical distributions of particular verb types • The availability in the concurrent visual scene of an object that could plausibly take on the Theme role.

22. General discussion • Experiment 1: verb-based info is not limited to anticipating Themes, but can also anticipate Goals. • Experiment 2: In combination with info conveyed by the verb, the Agent can constrain the anticipation of a subsequent Theme. • Experiment 3: In a head-final construction in Japanese, properties of the Theme can be anticipated on the basis of the combination of lexico-semantic and case-marking information.

23. General discussion • Is it the linguistic structure or the visual context that triggers a predictive process? • The processor uses accruing constraints to compute the likely thematic relationships amongst the entities already referred to in the linguistic input and amongst the entities concurrently available in the visual context. • Incremental, probabilistic processing drawing on different sources of information at the earliest possible opportunity to establish the fullest possible interpretation of the input at each moment in time.