Roger Johansson, Jana Holsanova, Kenneth Holmqvist

1. Pictures and spoken descriptions elicit similar eye movements during mental imagery, both in light and in complete darkness Eye movements as a window to the mind Jörg Brunstein Roger Johansson, Jana Holsanova, Kenneth Holmqvist Department of Cognitive Science, Lund University, Sweden

3. Theoretical background • Mental images: Do they exist? And if yes, how do they work? • Relevance: Where do people look when reflecting, remembering the last display etc.?

4. Hypotheses • Similar EM when seeing pictures, remembering pictures, and retelling verbal descriptions. • EM indicate locations of objects • Equally strong for retelling and remembering • In light and in complete darkness

5. Experiments • Listen and retell in light • Look and retell in light • Listen and retell in complete darkness • Look (in light) and retell in complete darkness • Data: EM during encoding (see/listen) and retrieval (describe/retell)

6. Methods • 12 + 12 + 28 + 28 participants (50% (fe)male) • SMI iView X at 50 Hz (bicycle helmet) • Glasses and lenses no problem, but mascara • White board 657 x 960 mm (about 36 x 38 inches) in 150 cm distance (about 59 inches)

7. Welcome to our lab!

8. Calibration

9. Phase 2 • Look: “You will soon see a picture. We want you to study the picture as thoroughly as possible. While you study the picture we will measure you pupil size.” (30 sec.) • Listen: “Imagine a two-dimensional picture. There is…” (2 min. 6 sec.)

10. Look!

11. Retell!

12. Interview • What do you think the objective of this study was? • Rate the vividness of your visualization during the description phase on a scale ranging from 1(not very vivid) to 5 (extremely vivid). • Access whether you usually think in pictures or words.

13. Data

14. Experiments 1 - 4 • Experiment 1: Listen and retell in light (2 min + 1-2 min) • Experiment 2: Look and describe in light (30 sec + 1-2 min) • Experiment 3: Listen and retell in complete darkness (2 min + 1-2 min) • Experiment 4: Look (in light) and describe (in complete darkness) (30 sec + 1-2 min)

15. Data analysis • Temporal reference: • Holsanova (2001): eye-voice latencies typically between 2 and 4 sec. • listening: looking after hearing (2.1 sec) • retelling: looking before or after telling (0.29 sec.) • 5 sec. before and after onset

16. Data analysis • Spatial reference: • Global correspondence: looking to correct position relative to complete scene (direction + distance) (up, down, left, right; full/half distance; stand still) • Local correspondence: looking into correct direction • No correspondence: neither location nor direction within time window

17. Results

18. Discussion: Content • Eye movements reflect positions of objects • Retelling = describing from memory • Functional role of eye movements for mental images as spatial indexes (working memory, simulated vision, utterance planning)

19. Discussion: Methods • Complex pictures and images • Spatial arrangements instead directions only • Relative ROIs instead of absolute ones for scaling effects • Mental images instead of visual percepts • Tracking in light and in complete darkness • Minor: don’t say that they track directions but calibrate participants