Visual Working Memory

1. Visual Working Memory From Perception to Memory Hans Supèr Department Visual System Analysis AMC/NORI

2. Types of Memory MEMORY (Long & short term) DECLARATIVE (Explicit, Episodic) NON-DECLARATIVE (Implicit, Procedural) PRIMING FACTS EVENTS • facilitation by repetition SKILLS/HABITS - dates, names - experience - routine NON-ASSOCIATIVE LEARNING SIMPLE CLASSICAL CONDITIONING - habituation - Pavlov’s dog

3. Types of Memory MEMORY (Long & short term) DECLARATIVE (Explicit, Episodic) NON-DECLARATIVE (Implicit, Procedural) PRIMING FACTS EVENTS • facilitation by repetition SKILLS/HABITS - dates, names - experience - routine NON-ASSOCIATIVE LEARNING SIMPLE CLASSICAL CONDITIONING - habituation - Pavlov’s dog

4. Concept of Working Memory Central Executive Visual sketchpad Phonological loop Long-term memory Baddely, Trends Cognit Sci, 2000

5. Functions of Working Memory Physiology • Learning. • Planning. • Reasoning. • To combine different modalities (e.g. visual and auditory). • For a correct internal representation of the object. • To bridge the gap between perception and action. • For correct behavioral responses. Psychology

6. Visual Working Memory • Maintaining a mental representation • Two components • Storage of information • Retrieval of information • Manipulate this information • Limited capacity • Spatial and object based working memory • Can lead to long-term memory

7. Visual System

8. Visual Perception Perception of object

9. Memory Storage Storage of object

10. Memory Retrieval Retrieval of object

11. Definition of Visual Working Memory • Storage of a perceptual representation for a short period. • Requires first the sensory/perceptual processing of the stimulus. • Subsequent maintenance of its representation after stimulus removal. • This information is required for later use. • It is only stored when needed and is thus an active process.

12. Localization of Visual Working Memory Frontal lobe (Fuster; Goldman Rakic) Parietal lobe (Goldberg, Haxby) Occipital lobe (Jonides, Supèr) Temporal lobe (Squire, Desimone)

13. Specialization of Visual Working Memory Frontal lobe Parietal lobe Spatial based Domain specific - Spatial cognition in Dorsolateral PFC - Object cognition in Ventrolateral PFC Occipital lobe Temporal lobe Sensory based Object based

14. Medial Temporal Lobe System Hippocampus Rhinal areas

15. Measuring Visual Working Memory • Object matching (identity of objects) • Spatial memory(position of objects) Screen 1 Screen 2 Screen 3

16. How Is Working Memory Implemented in the Brain? time Spike rate Low (tonic) High (burst) Spike interval Neuron 1 Neuron 2 Synchronized De-synchronized

17. Coherent (synchronized) activity of neurons. Neurons are active at the same moment. Low and high frequency range. Between neurons in different cortical areas. Neural Correlates of Working Memory Fell et al., Nat. Neurosci. 2001

18. Delay activity, I.e. Enhanced responses during memory period. This is found in parietal and frontal areas. Neural Correlates of Working Memory ( Chafee & Goldman-Rakic, J. Neurophysiol, 1998)

19. Synaptic Plasticity • LTP/LTD • Mainly NMDA dependent • Observed in several cortical areas • Ionotropic/Metabotropic glutamate receptors • Calcium, Calcium Binding Proteins • Kinases (PKC, CAMKII) • Transcription factors • Early genes (c-fos, zif-268) • Neuro-modulators (stress hormones) • Protein synthesis • Pre & post synaptic membrane proteins • Morphological changes Synapse spine

20. Summary • Working memory is a cognitive (i.e. internal) process. • Neural correlate of working memory is found in (poly-) sensory and association, and limbic areas. • Different domains of working memory in the prefrontal cortex. • During memory coding neurons show coherent activity (spike activity and EEG). • During memory period some neurons (delay cells) have an enhanced firing rate. • Synaptic plasticity; strengthening and weakening of synapses.

21. !How is the perceptual information transferred into a working memory component? Storage of object Perception of object

22. The Visual System Feedfoward Feedback

23. Experimental Setup: Perception Average neural response

24. Experimental Setup: Perception Average neural response

25. Figure Positions Positions relative to the receptive fields of the recorded neurons

26. Eye Positions and Reaction Time Window of analysis Start Saccade Eye position

27. Probing Perception

28. Perceptual Correlate Object (figure) is represented by enhanced activity (colored area) (Lamme & Roelfsema, TINS 2000) (Supèr et al, Nat. Neurosci, 2001)

29. Performance

30. Modulation Strength and Figure-ground Contrast

31. Reaction Times Window of analysis Start Saccade Eye position

32. Response Strength

33. Experimental Setup: Working Memory Average neural response

34. Experimental Setup: Working Memory Average neural response

35. Experimental Setup: Working Memory Average neural response

36. Probing Working Memory

37. Visual Stimulation

38. Performance of Memory Task

39. Results Memory Task Figure Response Memory Correlate Ground Response Supèr et al, Science, 2001

40. Remembered Vs Forgotten

41. Task Probing Active Storage Figure = Target Figure = Distractor End of trial; t = 1500 ms. End of trial; t = 1500 ms. Saccade Saccade Cue on; t = 1000 ms. Cue on; t = 1000 ms. Distractor on; t = 500 ms. Target on; t = 500 ms. Fixation Fixation Target off; t = 28 ms. Distractor off; t = 28 ms. Target on; t = 0 ms. Distractor on; t = 0 ms. Start fixation Start fixation

42. Results Active Storage Figure = Target Figure = Distractor Modulation Index Memory trace is maintained Memory trace disappears

43. Transformation of Memory to Motor Activity.

44. Conclusions • Perception/Visual working memory is represented by enhanced firing rate and coherent activity. • Visual working memory is the active storage of perceptual information. • Visual working memory is a distributed process throughout the brain. • Sequence of correlate of perception, memory, and motor-related activity.