Eye Movements

1. Eye Movements From Ennis (2004)

2. Chapter 9 – Mental Imagery A case study of the value of neuroscience to the study of human cognition

3. Perception vs. Cognition • What is the relationship between Perception and Cognition? • Are they separate categories of mental functioning? • Perception: Seeing, detecting, recognizing • Cognition: Thinking, recalling, surmising • Or, do they form a continuum?

4. Perception – Cognition Continuum • Two criteria for identifying a given representation as more “perceptual” or more “cognitive • More peripheral representations largely “hardwired” (e.g., wiring of rods and cones in retina), but more cognitive representations largely dependent upon experience (e.g., our ability to represent and distinguish faces of racial minorities) • Peripheral representations are more closely linked to retinal activity than cognitive representations. • Mental imagery: Activates visual representations independent of retinal input

5. What is mental imagery? Using visual imagery as a mental laboratory • Which is greener? A pea or a spruce tree? • Which is larger? A crow or a goose? • Do the tips of the ears of Newfoundland dogs protrude above the top of their heads? • Which is rounder? An apple or a pear? • How many black stripes are on the sides of a school bus? • Can you rearrange the furniture in your bedroom to make room for a new dresser? • What route to school will avoid new road construction?

6. Mental Imagery: A short history • 1889 – 1913. Psychology originally defined as the study of the mind; studies of mental imagery were in vogue. • 1910s. Good techniques for studying mental imagery did not exist. Non-productive debate between those who believed that thought was based in imagery and those who argued for imageless thought led to the downfall of the mind and the experimental study of imagery. • 1920s. Psychology redefined as the study of behavior; in the behaviorist era, there was no place for concepts such as “mind” or “imagery”. • 1970s. New techniques led to renewed study of the role of mental imagery in cognition. • 1973 – 1981. The Imagery Debate: Pylyshyn vs. Kosslyn (and others) • Does mental imagery involve some of the structures and processes normally used during visual perception or does it involve only more abstract post-perceptual representations? • Are mental images array-like (retinotopic) or are they propositional (symbolic) in format? • The present: Mental imagery is a central cognitive concept..

7. The Classics: Image Scanning (Kosslyn, Ball & Reiser, 1978) • Participants learned locations on map of a fictional island. • Scanning trials: Participants were to mentally image the map and focus on the first named location; if the second named location was on the map, they were to scan to it, pressing a key upon arrival. They were to image a small black speck zipping (as fast as possible, while remaining visible) in the shortest distance between the two points • More time is required to scan across longer distances. • Visual images preserve metric spatial information

8. The classics: Mental Rotation (Shepard & Metzler, 1971) • Stimuli: Connected 3D cubes. Presented in pairs. • Task: Make a rapid same-different response indicating whether the two figures are the same or different. • Results: Time to make a correct same decision is a linear function of the angular difference in orientation between the two figures. • Mental rotation corresponds to the way physical objects might be rotated.

11. The Imagery Debate: The Tacit Knowledge Account (Pylyshyn, 1973, 1978) • Cognition is distinct from perception and involves symbolic (propositional) processing. • Participants in imagery experiments use tacit knowledge of how their visual systems work to model their task performance according to how they think they should do it (e.g., scanning task: they know it takes longer to scan longer distances…) • Imagery is not functional; it is an epiphenomenon.

12. The Imagery Debate: Experimenter Expectancy Effects (Intons-Peterson, 1983) • Is mental imagery performance sensitive to experimenter beliefs as conveyed by unintentional cues? • Used a version of the Kosslyn et al (1978) scanning task. • Experimenter beliefs can influence performance.

13. The Imagery Debate: Overcoming the Impasse (Anderson, 1978) • Anderson (1978) argued that cognitive data could not be used to determine whether imagery used array-like or symbolic representations. • Brain-based data, however, could be used to overcome problems of tacit knowledge and experimenter expectancy effects. • That is, few, if any, participants know (1) what parts of their brain are active during visual processing or (2) how to control blood flow or brain activity in those regions.

15. Imaging Imagery: ERP studies Farah et al (1988) • Goal: To show that imagery and perception share representations • Demonstrated content-specific effects of imagery on perception: perceiving a letter (T) facilitated by imaging the same letter. • ACCURACY: Match > No match (77% > 72%) • ERP: Traced time course and location of maximum brain activity: Within the first 200 ms (N1) in occipital and temporal-occipital areas. Maximum activation in occipital areas.

16. Imaging images: PET studies (Kosslyn et al, 1995) • Research Problem: To localize the area of the brain involved when using visual imagery • Research Hypotheses • If visual imagery and visual perception share processes and structures, using visual imagery should be associated with activation in primary visual cortex. • Activation associated with different sized images should be organized topologically • Methodological: • Resting baseline is not an appropriate control condition for image generation (because participants might be visualizing while “resting”). • Listening baseline (wherein participants hear the same auditory instructions given in the visual imagery conditions) is appropriate

17. Kosslyn et al (1995) continued

18. Imaging images: fMRI. Le Bihan et al (1993). • fMRI activation in response to actual visual stimulation (two flashing red squares) compared with activation when participants imaged the same visual input. • All participants showed significantly increased activation in V1 and V2 during perception and 5 of 7 during imagery relative to no stimulus conditions.

20. Using neuropsychological data to dissociate visual & spatial imagery (Farah et al., 1988) • Visual Imagery tests (color of objects, size comparison, animal tails, state shapes). • Spatial Imagery tests (Image transformations: rotation, scanning; Image location: matrix memory, corner classification). • Compared the performance of LH (associative agnosia, prosopagnosia) with 12 men in 30s with Master’s degrees.

21. Imaging imagery: Some conclusions • Most neuroimaging studies show that visual perception and visual imagery activate the same areas of the brain. • Most, but not all, patient data are consistent with the hypothesis that visual perception and visual imagery share representations. The exceptions can generally be explained in other ways. • Primary visual cortex (V1, Area 17, striate cortex) is activated during visual imagery in well designed studies, suggesting that imagery is array-like, not propositional. • These results cannot easily be explained away by recourse to the effects of tacit knowledge or experimenter expectancy.

22. How do Imagery and Perception Differ? • Do they both involve the same processes, such that visual imagery is a form of top-down driven visual perception that is running backwards? • One difference: Although we virtually always recognize objects, we do not always generate images. • Generating images is often intentional and effortful. • Perhaps the process of generating images is a separate component, specific to imagery. • If so, where might it be localized?

23. Impaired on tasks requiring image generation Verifying “Grapefruit is larger than a cantaloupe” Visual memory tests (imagery) Drawing a object (e.g., fish) or completing an object from memory Given B-W drawing of an object, select correct color from three choices Normal on tasks not requiring image generation Verifying “Canada has two major political parties” Visual memory tests (perception) Given two drawings of an object (e.g., fish), select the correct shape Given three colored drawings of an object, select the correct one. Dissociating perceiving and generating images (RM, Farah et al, 1988)

24. Generating images: Interpreting the evidence • Farah & colleagues: RM has a selective impairment in image generation. • Goldenberg & colleagues: RM has a “subtle impairment of long-term visual memory knowledge”. Some of his patients have problems with image generation and subtle visual memory. • Global form okay; supports object recognition • Details, color, surface properties impaired; needed for image generation

25. Generating images: A part of perception • Perception is interactive, involving both bottom-up and top-down processing. When making difficult discriminations (determining the correct appearance of a visual stimulus), we may need active top-down processing in the form of image generation from visual memory.

26. Localizing the image generation process • Brain damaged patients • Impairments in image generation usually associated with a lesion in the posterior left hemisphere. • Hemispheric lateralization • Studies of normal subjects yield small effects that are difficult to replicate. • Split brain patients • All cases show a left hemisphere superiority for image generation that diminishes or disappears with practice, depending upon the patient and task. • Conclusion • “The left hemisphere plays a dominant role in image generation for most people…” (p. 289)

27. A model of visual imagery (Kosslyn, 1994) • Key Phenomenon to be explained include our ability to: • Generate images • Maintain images • Inspect images • Transform images • The Model: Imagery is a multi-component system that draws on the processes and structures used in visual perception, including: • A visual buffer • Long-term associative memory • Processes and structures to generate, maintain, inspect, and transform images

28. The Protomodel: Visual Perception • Visual Buffer: The image is a retinotopically mapped pattern of activation (V1) • Attention Window: Can select a set of contiguous points in visual buffer for detailed processing. • Object Properties: Encoding of shape, color, texture (Ventral stream, IT). • Spatial Properties: Encoding of location, orientation, size (Dorsal stream, parietal lobe). From Kosslyn (1994)

29. The Protomodel: Visual Perception • Associative memory: Convergence of output from object and spatial properties encoding and knowledge from other modalities (parietal-occipital junction?) • Information look-up: When need more information to complete recognition, the partially activated information from associative memory guides further encoding; e.g., top-down processing (Dorsolateral prefrontal cortex). • Attention-shifting: Mechanism to shift attention (several areas)

30. The Protomodel in Visual Imagery • A visual image is a pattern of activation in the visual buffer that does not reflect sensory input. • A pattern of activity in the visual buffer is processed in the same way, whether it originated in perception or imagery. • Three differences between images and percepts: • Visual images fade rapidly and effort is needed to maintain them. • Visual images are created from stored information. They are generated by the same mechanisms that prime perception. • Visual images are malleable.

31. Kosslyn (1994) Model of Visual Imagery