Top-down influences on the time-course of visual attention:

1. Top-down influences on the time-course of visual attention: When the attentional blink meets word superiority effect Maria V. Falikman Lomonosov Moscow University Research supported by the Russian Fund for Basic Researches, grant # 03-06-80191

2. Where cognitive science was born… “… I went away from the symposium with the strong conviction, more intuitive than rational, that human experimental psychology, theoretical linguistics, and computer simulation of cognitive processes were all part of a larger whole...” (George Miller on the 1956 MIT Symposium)

3. Task, Experience Conscious percept Stimulus Types of processing (information flow) Top-down (conceptually-driven) Bottom-up (data-driven)

4. Top-down influences on visual attention Unconscious (implicit) Conscious (explicit) Context-based or memory-based Goal-directed or task-dependent Contextual cueing Priming (+/-) Schemata ... Strategic regulation of perceptual task accomplishment

5. Rapid Serial Visual Presentation (RSVP) R A P L E R P S A U I E T S N O I V T N A I D S E R I A L

6. The Attentional Blink (AB) under RSVP conditions R A P I D S A L

7. The Attentional Blink (AB) under RSVP conditions R A P I D S E R I A L

8. The Attentional Blink (AB) under RSVP conditions R A P I D S A L

9. The Attentional Blink (AB) under RSVP conditions R A P I D S E R I A L

10. The Attentional Blink (AB) under RSVP conditions R A P I D S A L

11. The Attentional Blink (AB) under RSVP conditions R A P I D S E R I A L

12. The Attentional Blink (AB) under RSVP conditions R A P I D S A L

13. The Attentional Blink (AB) under RSVP conditions R A P I D S E R I A L

14. The Attentional Blink (AB) under RSVP conditions R A P I D S A L

15. The Attentional Blink (AB) under RSVP conditions R A P I D S E R I A L

16. The AB effect replicated: Results of the standard dual-task RSVP experiment (2000)

17. Blindness! Amnesia! Attentional gate model Attentional dwell-time theory Two-stage model Interference model Object substitution theory Central interference theory Perceptual system reconfiguration Hybrid models Why a blink?

18. Models of selective attention and models of its blink Attentional gate (Raymond et al, 1992) Early selection / filter (Broadbent, 1958) Late selection (Deutsch & Deutsch, 1963) Pertinence model (Norman, 1968) Interference model (Shapiro et al, 1994) Two-stage model (Chun & Potter, 1995) Multiple selection (Johnston & Heinz, 1979) Flexible selection (Yantis & Johnston, 1990) “Smart selection” (Shapiro & Luck, 1999) “Intelligent filtering system”(Enns et al, 2001) Perceptual cycle (Neisser, 1976) Reentrant model (Di Lollo et al, 2000)

19. Quo vadis? Attention for action Attention as an aspect of action D.A. Allport O. Neumann A.H.C. van der Heijden Gippenreiter, 1983 [rus] 1986 or a set of “task-related mechanisms” (e.g.Deacon & Shelley-Tremblay, 2000)

20. Early stage LC-stage T1 T1 report T2 Early stage . . . LC-stage T2 The prototypical “bottleneck” model

21. The AB effect: my old results again

22. The AB effect might be due to the inability of perceptual system to obtain further information until the previous goal-directed act (conscious target identification) is completed. Then, the AB could be modulated through the change of the size of the observer’s activity “units”: if we incorporate the 1st target into a larger perceptual “object”, the AB will probably not interrupt perception until the end of the act. “The difficulty is not to combine stimuli, but rather to deal with them independently at the same time... capacity limits occur where stimuli have to be kept apart, not where they have to be combined…” (Neumann, 1987, p.363)

23. “Attentional units” larger than letters? W O R D S

24. PRINTED WRITTEN Stimuli examples: Russian letters

25. The word superiority effect • Better performance on letters within a word • than on random letters: • faster recognition • more letters perceived within a brief (10 ms) presentation “I find it takes about twice as long to read (aloud, as fast as possible) words which have no connexion as words which make sentences, and letters which have no connexions as letters which make words. When the words make sentences and the letters words, not only do the processes of seeing and naming overlap, but by one mental effort the subject can recognize a whole group of words or letters, and by one will-act choose the motions to be made in naming them, so that the rate at which the words and letters are read is really only limited by the maximum rapidity at which the speech organs can be moved...” (Cattell, 1886, p.64).

26. The word superiority effect Further explorations: Reicher, 1969 Wheeler, 1970 McClelland, 1976 ...

27. The word superiority effect R

28. W O D R The word superiority effect

29. W O D R The word superiority effect

30. I ...SP R I T E # …WA T E R # L …FA V O R # ...PI L O T # G …BR I D E # ...BR I D G E # Some examples of “mutable” words (main experimental session)

31. Hypothesis IF the AB is due to a certain bottom-up controlled limitation (“a bottleneck”), it will leave its signature on the observers’ performance on mutable words. IF it is rather shaped by top-down processes in the information processing system, it will either disappear or be diminished in the word-reading task.

32. Comparison of correct reports on separate letters in mutable words and on a probe X under “standard” AB conditions(2000) Word identification point?

33. There is a blink on mutable words! Individual differences.

34. And what about the overall result?

35. The ACTUAL experimental design: Preceding control session: identify the type of a white letter and name three letters following it (after Weichselgartner & Sperling, 1987) Concluding control session: identify the type of a white letter and name three letters following it (after Weichselgartner & Sperling, 1987) Main experimental session: identify the type of a white letter and read a word beginning with this letter

36. Strategic change: Results of framing sessions

37. There could be no strategic changes! Individual differences again. Subject 2 Subject 16

38. No words noticed between! Control experiment: No practice effects

39. Significant! (p<0.0001) Significant! (p<0.0001) Word superiority in written vs. printed T1 task T1 type identification in the preceding control session: 89  3.9% T1 type identification in the word-reading session: 96.2  2.4 % T1 type identification in the concluding control session: 90.4  2.9%

40. Subjective dual-task estimation through the experimental sessions (% of subjects) Preceding session Main session (word-reading) Concluding session Two separate tasks A single whole

41. The AB seems to depend on the task and on subjective strategies. But strategies serve for achieving a goal, and the goal is to report on T1 etc. For that, verbal working memory consolidation or encoding is essential!

42. Working memory consolidation: neurophysiological data P300 - a late ERP component, a correlate of working memory encoding and attentional load T2: suppressed for missed probes during the AB (Vogel et al, 1998; see also Rolke et al, 2001) T1: repeats the dynamics of the AB after T1 identification (McArthur et al, 1999)

43. Report P ... ... H P H P H T1 + “word” WM encoding R A Time to start WM encoding: - I got the word! - My VSTM buffer is full! E R A possible model: two-stage compatible ... P P T1 WM encoding... Not yet! H R A S

44. Conclusions • Perceptual task accomplishment can in principle be organized in such a way, that, given a sufficient load of that limited-capacity block or process, the AB would either disappear or diminish and shift from its critical temporal interval. • These changes can be determined both externally (by task conditions and requirements) and internally (by subjective strategy), that is characteristic for any human activity.

45. Hurrah! Recent results working for this hypothesis: if T1 is an addition task (e.g. 2+3) and T2 is an answer (seems to be… 5!) there is no blink! (Kunar & Shapiro, in preparation)

46. Conclusions • Our results directly suggest that the attentional blink deficit does not represent merely the workings of a fixed capacity-limited mechanism for the perception of events occurring over time, but rather the structure of actions a subject has to perform in order to accomplish a given task.

47. Missing controls? • Pronounceable pseudowords - in progress • Random strings, instruct to read words - in progress • Words with ambiguous endings? - Thanks to Molly Potter for the brilliant idea! To be performed. • More ideas? - Please keep them in mind until the discussion...

48. Prospect: • Visual search? • Attentional switching (gating paradigm)? • Metacontrast masking? • Unconscious perceptual priming? • Crowding? - YES (Fine, in press)

49. Special thanks to Dr. Valeriy Romanov, Dr. Yuriy Dormashev (Moscow University) for inspiration of the AB research Ekaterina Pechenkova (Moscow University) for collaboration and invaluable discussions Our programmers: Suren Sagiyan George Kouryachy

50. Thank you for your attention that hopefully didn’t blink (owing to the top-down regulation!)