Memory and Cognition

1. Memory and Cognition PSY 324 Topic: Visual Imagery Dr. Ellen Campana Arizona State University

2. Visual Imagery

3. What is Imagery? • Think about the following questions…. • How many windows are on front of where you live? • How is the furniture arranged in your bedroom? • Are an elephant’s ears rounded or pointy? • What does a pumpkin pie smell like? • You use mental imagery to answer these • Mental imagery = experiencing a sensory impression in the absence of sensory input

4. Uses of Imagery • Creative process of music • Paul McCartney’s “Yesterday” • Conductors use “inner audition” to practice without orchestras • Imagine sounds • Imagine locations of instruments • Sports Psychology • Swimming – imagining all of the strokes • Time to mentally rehearse linked to real times • Jack Nicklaus, golfing, discovered an error

5. Uses of Imagery • Visual Imagery (mental imagery of visual info) useful in scientific discoveries • Einstein’s theory of relativity came from imagining himself riding along a beam of light

6. The Study of Imagery • Early ideas • Aristotle: thought is impossible without an image • Wundt: images are one of three basic elements of consciousness (others: sensations, feelings) • Strong link between images and thought • Gave rise to imageless-thought debate • Can people think without images? • Galton said yes: people who have difficulty forming images can still think • Wundt still said no

7. The Study of Imagery • Behaviorism • Rise of behaviorism shut down the imageless-thought debate • Study of imagery is unproductive because visual images are invisible to everyone else • Watson (1928) called visual imagery “unproven” and “mythological” • 1920’s-1950’s: no study of imagery in mainstream psychology

8. The Study of Imagery • The Cognitive Revolution • Cognitive psychologists developed ways to measure behavior to infer cognitive processes (like visual imagery) • Memory (Paivio, 1963, 1965) • Mental chronometry (Shepard & Meltzer, 1971) • Study of imagery returned to mainstream psychology • Research continues today

9. Visual Imagery Research

10. Demonstrating that Imagery Exists • Paivio (1963, 1965): concrete vs. abstract nouns • Method: paired-associate learning • Training: participants learn pairs of words • Testing: participants presented with first word of pair, task is to recall the second (recognition task) • Two types of word pairs (within-Ss design) • Concrete: truck – tree (both easy to form image of) • Abstract: truth – idea (both hard to form image of) • Result: concrete pairs were easier to recall • Interpretation: Conceptual-peg hypothesis

11. Demonstrating that Imagery Exists • Shepard and Metzler (1971) • Mental chronometry – general idea of measuring how long it takes to carry out various cognitive tasks • Task: participants saw two objects, had to indicate quickly whether the two objects were the same or different • Results: Time it took to indicate that they were the same object was directly related to how far the object had to be rotated • Interpretation: imagery and perception share some of the same mechanisms

12. Shepard & Metzler (1971)

13. Imagery and Perception • Major question: do imagery and perception share the same mechanisms? • Kosslyn (1973): mental scanning • Task: Memorize an image, then answer questions about whether certain parts appear in the image • Time it takes to say yes is related to distance between initial focus and correct part

14. Kosslyn (1973)

15. Imagery and Perception • Major question: do imagery and perception share the same mechanisms? • Kosslyn (1973): mental scanning • Task: Memorize an image, then answer questions about whether certain parts appear in the image • Time it takes to say yes is related to distance between initial focus and correct part • Lea (1975): alternative explanation for result • Perhaps as people scan they are distracted by other parts • Kosslyn (1978): ruled out alternative explanation

16. Imagery and Perception

17. Imagery and Perception • Kosslyn’s experiments were convincing, but there was another alternative explanation • Pylyshyn (1973) – results are based on propositional mechanisms, not on spatial representation • Ushered in the imagery debate (still going on)

18. Propositional vs. Spatial • Basic idea: just because the experience is spatial doesn’t mean the underlying representation is • Spatial experience of mental images could be an epiphenomenon (side effect) • Epiphenomenon: related to / co-occurring with the mechanism, but not actually part of the mechanism • Information could be encoded with language (propositional representation) or with images (depective representations) – can’t tell • Propositional rep. is like a semantic network

19. Propositional vs Spatial

20. How Does Phlyshyn Explain Kosslyn’s First Set of Results? • Tacit-knowledge explanation • People in the mental scanning task behave based on what happens in a real scene (tacit knowledge) • In the real world it takes longer to travel greater distances • Simulate this behavior in the experiment • This explanation seems overly complicated and doesn’t seem to scale to complex scenes • Phylyshyn still makes these arguments, so they have to be addressed

21. Propositional vs Spatial

22. Against Propositional Representation • Finke and Pinker (1982) • Short presentation of simple display with 4 random dots, followed (after 2-sec delay) by an arrow • Participants had to say whether the arrow pointed to one of the dots in the first display (gone now)

23. Finke and Pinker (1982)

24. Against Propositional Representation • Finke and Pinker (1982) • Short presentation of simple display with 4 random dots, followed (after 2-sec delay) by an arrow • Participants had to say whether the arrow pointed to one of the dots in the first display (gone now) • Results: time it took to respond “yes” was directly related to distance from arrow • Why couldn’t this be propositional? • No time to convert to propositions, no meaning in the dots (except spatial relations)

25. Propositional vs. Spatial • Phylyshyn refuses to cede, responding to each study with tacit knowledge explanations • He is in the minority • Most researchers think there’s overwhelming evidence that mental imagery is spatial • Let’s assume (like most people) that mental imagery is spatial, and go on talking about how it relates to perception

26. Imagery and Perception

27. Size in the Visual Field • As you move closer to objects in the real world • Object fills more of your visual field • Details are easier to see • Kosslyn (1978) used these facts to investigate mental imagery

28. Size in Visual Field

29. Kosslyn (1978) • Task • Imagine two objects, moving close enough so that the bigger object fills most of visual field • Answer questions about one of the animals • Conditions (within-subjects) • Question about the animal when it was bigger than the other animal • Question about the animal when it was smaller than the other animal • Result: questions answered faster when it was bigger • Same as with a similar perceptual task

30. Kosslyn (1978)

31. Kosslyn (1978) • In addition to the question-answering experiment, participants did a mental walk task • Imagine an animal • Walk toward it until it “overflows” the visual field • How far away is it? (estimate distance) • Let’s try: • Cat • This building • Result: Greater distances for larger objects

32. Interactions Between Perception and Imagery • Visual images can interact with perception • Perky (1910) – subliminal presentation • Participants imagined an object and described their mental image to the experimenter • While they did this, an image of the same object was projected very dimly • No participants reported seeing the image • Descriptions of mental images matched details of real image • Farah (1985) – priming task • Mental images prime performance in a perception task

33. Farah (1985) • Imagine the letter ___ • Indicate whether it was in the first or second white square H • Results: participants were more accurate when the letter that appeared matched the letter they had imagined

34. Imagery and the Brain

35. Brain’s Response to Imagery • Imagery neurons (Kreiman & coworkers, 2000) • Respond to both perceiving and imagining an object • Single cell recording, but in humans • fMRI & PET • most studies show overlap for perceiving and imagining (visual cortex, frontal lobes) • More activity toward the back of the brain for perception than imagery • Nonvisual areas deactivated (Amedi & coworkers, 2005)

36. Brain’s Response to Imagery • Transcranial Magnetic Stimuluation • Method for temporarily disrupting function in part of the brain • Lets researchers demonstrate causality, not just correlation (if you disrupt it and people can still do the task it cannot be causal) • Kosslyn and coworkers (1999) used this method to show that areas involved in both perception and imagery were causally linked to imagery (not an epiphenomenon)

37. Brain’s Response to Imagery • Neuropsychological case studies • Some evidence of related mechanism • MGS: mental imagery changed after removal of occipital lobe tissue • Perceptual problems accompanied by problems with imagery • Unilateral neglect (Map of part of Milan study) • Some evidence of independence • Dissociations between imagery and perception • R.M. Could see and draw but not imagine and draw, etc. • C.K. Visual agnosia (could not name pictures) but OK drawing from imagination

38. Putting it all Together • Some evidence for common mechanism BUT also some evidence for separate mechanisms • Behrmann & Coworkers (1994) have an account that brings them together (top-down vs. bottom-up) • No true “answer,” just continuing debate

39. Other stuff…

40. Using Images to Improve Memory • Method of Loci • “Placing” objects at places within a familiar location to remember them – walking through that location when remembering • Peg-word Technique • Learning a rhyme (one is a bun, two is a shoe…) and then imagining things you want to remember asssociated with those things

41. Mental Representation of Mechanical Systems • People can use visual imagery to solve problems • Five-gear problem • Water-pouring problem • Pulley problem • It has also been important for scientific discovery • Einstien’s theory of relativity

42. The End