The neuroscience of memory

1. The neuroscience of memory

2. The standard model of memory • Sensory store (iconic memory) • Short-term memory • Long-term memory

3. Problems with the standard model • Is the sensory store really “memory”? • Is the short-term store really so passive? • What defines an “item?” • Levels of Processing

4. Working memory • There is clear evidence that the short-term store is more active than just a temporary warehouse. It is where we actually process the information we’re using. • Information enters working memory from both perception and from LTM. • Baddeley: • Central Executive • Phonological loop • Visuo-spatial scratchpad

5. Neurological evidence of WM

6. Information encoding • Propositional Hypothesis: All information is stored in an abstract, amodal format called “propositions” • Based on the mathematics of propositional logic • Propositions represent facts about the world, and can capture properties of objects or relationships between two objects. • Dual-Code Theory: We store information in two types of codes: • Verbal codes are abstract and amodal. Could be similar to propositions • Analog/Imaginal codes are tied to the particular sensory modalities. For example, when we imagine something we are actually loading a picture-based representation into working memory.

7. Mental Rotation • Shepard & Metzler (1967) did the classic mental rotation experiment. • Show people two arbitrary, 3-D objects at different orientations. • Ask people to judge whether the objects are mirror images of each other or not. • The amount of time it takes people to answer is directly proportional to the degrees of rotation by which the objects differ.

9. Relative size & Image scaling • When asked to imagine two objects, it takes longer to make judgments about the smaller object • It also takes longer to make judgments about smaller features of objects

11. Image scanning • Give people a picture of something, such as a map of a simplistic island and have them memorize it. • Ask people to scan the image from one point on the map to another. • The time it takes to do this is proportional to how far apart the two points are.

12. The Mind’s Eye hypothesis • Kosslyn proposed that mental imagery is functionally equivalent to vision. • What he means by this is that mental imagery uses the same internal representations and processes that vision does, but without visual input.

13. Neurological evidence for mental imagery • fMRI studies have shown that when subjects perform imagery tasks, early visual areas of the brain become active. • It is also the case that when subjects are asked to think about particular properties, such as color or action, the parts of the brain that process those properties (V4 and MT) become active.

14. Unitary Content Hypotheses (UCH)(Caramazza et al., 1990; Caramazza & Shelton, 1998; Riddoch et al., 1988; Pylyshyn, 1973)

15. Multiple Semantics Hypotheses (MSH)(Paivio, 1971; Beauvois, 1982; Shallice, 1987, 1988; McCarthy & Warrington, 1988)

16. The Evidence • Category-specific deficits (e.g., Warrington & McCarthy, 1983, 1987; Warrington & Shallice, 1984; Gainotti & Silveri, 1996) • Patients show impairments in processing living things vs. man-made objects and vice versa. • MSH explanation: the Sensory-Functional Theory (Warrington & Shallice, 1984; Farah & McClelland, 1991) • Modality-specific deficits • Patients are unable to name visually presented objects, but can name them from other modalities and can access other semantic information about visually presented stimuli (Beauvois, 1982) • UCH explanation: priviliged access and privileged relationships.

17. More evidence • Brain-imaging (Martin et al., 1995; Martin et al., 1996) • Brain areas associated with visual processing more active when animals are being named; • Areas associated with generating action words or imagining actions active when naming man-made objects • Areas associated with color perception are active when color words are generated, even for man-made objects • Areas associated with motion perception are active when action words are generated for man-made objects • UCH: Are they just tapping into color or action concepts?

18. Evidence (cont.) • Naming vs. Categorization (Potter & Faulconer, 1976; Guenther et al., 1980, Seifert, 1997) • Faster to name words vs. pictures • Faster to categorize pictures vs. words • Stroop-like effects (Glaser, 1992; Glaser & Glaser, 1989) • Incongruent words interfere with pictures more than pictures interfere with words in a naming task • Incongruent pictures interfere with words more than words interfere with pictures in a categorization task

19. Conclusions While both the UCH and MSH can be modified to account for any one piece of evidence, the preponderance of evidence seems to favor a system organized more closely to the MSH.

20. Computational models:Graded specialization • Multiple input and output dimensions • Encodes the notion of privileged access and relationships in the systematicity of the environment • Further presumes that neurons like short connections • As a result, neurons close to modality-specific inputs and outputs respond better to those inputs and outputs. (From Plaut, 2002)

21. Future Directions • Most existing models were designed and tailored to account for only one of the many phenomena mentioned • e.g., Farah & McClelland, 1991 - SFT; Caramazza et al., 1990 - optic aphasia, etc.

22. Graded specialization: Accounting for evidence • Optic aphasia - damage to the pathways between visual inputs and naming outputs • Category specific deficits - systematic relationship between information from a given category leads to vertical organization of category information that is orthogonal to modality-based organization. • Experiment 1 explained in terms of different inputs accessing different category representations • Pictures have systematic relationships to each other, and thus to categories. • Words don’t. • Experiment 3 simply implies that words have more systematic relationships to their functions than to their categories.