Learning and Memory in Hippocampus and Neocortex: A Complementary Learning Systems Approach

1. Learning and Memory in Hippocampus and Neocortex: A Complementary Learning Systems Approach Psychology 209 Feb 11&13, 2013

2. Effect of a HippocampalLesion • Intact performance on tests of general intelligence, world knowledge, language, digit span, … • Dramatic deficits in formation of some types of new memories: • Explicit memories for episodes and events • Paired associate learning • Arbitrary new factual information • Largely spared implicit learning: • Skills such as mirror reading and mirror copying • Spared item-specific priming effects in some studies • Temporally graded retrograde amnesia

3. Implicit learningin Amnesia • Mirror reading skill (also mirror tracing) is acquired at normal rate in amnesics (see fig.), but sensitivity to repeated items is impaired. In the Cohen and Squire (1981) study (results at left) amnesics showed normal learning to read mirror-reflected word triples, although there was a deficit in reading of repeated triples. • Stem (e.g., win___) and fragment completion is intact for words and pictures. • Repetition priming is intact for words and pseudowords (e.g., marnot), but stem completion is impaired for pseudowords (mar___). Example of one of Cohen and Squire’s stimuli:

4. Performance Based on Memories Acquired at Different Times Prior to Hippocampal Lesion or (for humans) ECT Task/species used in studies at left: • Choice of once-experienced food in rats. • Contextual fear conditioning in rats. • Arbitrary food-object association learning in monkeys. • Memory for TV shows in humans. • Notes: • There is a possible confound in the Squire and Cohen human study. • Replicability of animal results is also an issue. Time of experience prior to lesion/ECT in days

5. language An Account of the Findings • Processing takes place through the propagation of activation among neurons in ‘the neocortical system’ and other non-hippocampal areas. • Implicit learning results from small changes to connections among neurons active in each episode of processing. • Single processing episodes produce item-specific effects. • Skills arise through accumulation over many such episodes. • New arbitrary associations are based on concomitant learning that takes place within the hippocampal system. • Bi-directional connections between cortex and hippocampus carry information in and out. • Large changes to connection strengths in the hippocampal system bind the elements of the incoming pattern together. • Recall depends on pattern completion within the hippocampus. • Over time, memories initially stored in the hippocampus may be repeatedly reactivated, resulting in gradual transfer to cortex via accumulation of small adjustments in the neocortex. language

6. Supporting Neurophysiological Evidence • Anatomy: • The necessary bi-directional pathways exist • Hippocampus very rich in plasticity-related NMDA receptors • Slice physiology: • Long-term potentiation in hippocampal slices • One study has suggested that Hippocampal LTP is easier to induce but saturates easily, while neocortical LTP cumulates gradually. • Single neuron recording in intact animals: • Memory reinstatement during sleep

7. Bi-directional connectivity between regions overlying the hippocampus and various other regions. There are dense connections to association cortex, but not to primary sensory cortices.

8. Long-term potentiation is most easily done in hippocampal slices. Some studies support the suggestion that hippocampal LTP quickly reaches maximum, while LTP in cortex cumulates gradually with over repeated induction episodes.

9. Reactivation during sleep • In a study by Skaggs and McNaughton (1996), animals were trained on a triangular track, so that cells with neighboring place fields were activated at successive times. • Reactivation with temporal asynchrony occurred during sleep after experience in maze but was not seen in sleep before. • This is evidence of the reactivation of very recent experience; there is as yet no evidence of reactivation of memories acquired many days earlier. • It is also worth noting the time-compressed character of the reactivation effect.

10. Why Should the System be Organized this Way? • Why do we need the hippocampal system at all, if all kinds of knowledge can eventually be acquired in the neocortical system? • Why aren’t the changes made directly in the cortex in the first place? • Why is consolidation (transfer to neocortex) so slow? • Why can’t we integrate what has been learned in the hippocampus into the neocortex more rapidly, for example (as Marr suggested) overnight?

11. Successes and Failures of Connectionist Models • Gradual discovery of shared structure through gradual interleaved learning in multiplayer networks. • Illustrated in the network of Rumelhart and Todd, as well as many other models of reading, language processing, etc. • Catastrophic interference with “sequential learning”. • As illustrated in the case of paired associate learning by McCloskey and Cohen (1989) • And for the case of learning new semantic knowledge in the Rumelhart and Todd network.

12. The Rumelhart Model The QuillianModel

13. Early Later LaterStill Experie nce

14. Phenomena in Development • Progressive differentiation • Overgeneralization of • Typical properties • Frequent names • Emergent domain-specificity of representation • Basic level advantage • Expertise and frequency effects • Conceptual reorganization

15. McCloskey and Cohen’s Network Training Experiment • Trained network with two lists: List 1: set of A-B pairs. List 2: set of A-C pairs. • Items were random patterns. • A fixed context pattern was used for List 1, and another for List 2. • Training on List 1 continued until performance was perfect, then training on List 2 commenced.

16. Interference Effects in Humans and in McCloskey and Cohen’s Network

17. Adding New, Partially Arbitrary, Information to the Neocortical Representation • Penguin is a bird • Penguin can swim, but cannot fly

18. Catastrophic interference, and solving the problem with interleaved learning

19. Proposed Answers to the Two Questions • Why do we need the hippocampal system at all, if all kinds of knowledge can eventually be acquired in the neocortical system? • We need the hippocampus to allow rapid learning of new arbitrary or inconsistent information, without at the same time corrupting the connections that store our knowledge of the shared structure of experience. • Why is consolidation (transfer to neocortex) so slow? Why can’t we integrate what has been learned in the hippocampus into the neocortex more rapidly? • Consolidation is slow to allow new material to be gradually interleaved with ongoing exposure to other events and experiences that embody the shared structure. • New Results: What about new material that is consistent with what you already know?

20. Tse et al (Science, 2007, 2011) Additional tests after surgery for old and new associations. Then train and test asecond pair of newassociations. During training, 2 wells uncovered on each trial

22. Schemata and Schema Consistent Information • What is a ‘schema’? • An organized knowledge structure into which new items could be added. • What is schema consistent information? • Information consistent with the existing schema. • Possible examples: • TroutCardinal • What about a penguin? • Partially consistent • Partially inconsistent • What about previously unfamiliar odors paired with previously unvisited locations in a familiar environment?

23. New Simulations • Initial training with eight items and their properties as indicated at left. • Added one new input unit fully connected to representation layer to train network on one of: • penguin-isa & penguin-can • trout-isa & trout-can • cardinal-isa & cardinal-can • Used either focused or interleaved learning • Network was not required to generate item-specific name outputs.

24. New Learning of Consistent and Partially Inconsistent Information

25. What about the HippocampUS?

26. Why Doesn’t Catastrophic Interference Occur in the Hippocampus? • Interference is caused by overlap of patterns; what is learned about one pattern tends to bleed over into other patterns that overlap with it. • Hippocampus may use conjunctive coding to reduce pattern overlap, thereby reducing interference.

27. A simple pattern associator network illustrating the problem of overlap • Willshaw (1969) studied simple associative nets, in which connections could be either on or off. • Connections are switched on to store associations between designated input and output patterns. • If two patterns are chosen at random, there is a chance that they will overlap; when they do, what has been learned about one pattern will tend to show up in the output produced by the other. • The probability that the patterns will overlap goes down if they are sparser (smaller fraction of units active).

28. Conjunctive coding sparsifies and reduces overlap • Consider two different input patterns, each consisting of two active units, with one unit of overlap. • If the patterns are recoded, so that a separate unit is used to represent every possible pair of units, this tends to reduce the overlap. • In general, the higher the order of the conjunctive coding, the greater the sparsification, and the greater the reduction of the overlap. • Several studies showed that catastrophic interference can be much reduced using sparse, conjunctive representations instead of the highly overlapping distributed representations of McCloskey and Cohen.

29. Conjunctive coding sparsifies and reduces overlap • Consider two different input patterns, each consisting of two active units, with one unit of overlap. • If the patterns are recoded, so that a separate unit is used to represent every possible pair of units, this tends to reduce the overlap. • In general, the higher the order of the conjunctive coding, the greater the sparsification, and the greater the reduction of the overlap. • Several studies showed that catastrophic interference can be much reduced using sparse, conjunctive representations instead of the highly overlapping distributed representations of McCloskey and Cohen.

30. Examples of neurons found in entorhinal cortex and hippocampal area CA3, consistent with the idea that the hippocampus but not cortex uses sparse conjunctive coding Recording was made while animal traversed an eight-arm radial maze.

31. Collaboration of Hippocampus and Neocortex • The effect of prior association strength on memory in both normal and control subjects are consistent with the idea that hippocampus and neocortex work synergistically rather than simply providing two different sources of correct performance. • Even a damaged hippocampus can be helpful when the prior association is very strong.

32. Performance of Control and AmnesicPatients in Learning Word Pairs with Prior Associations Base rates man:woman hungry:thin city:ostrich

33. Hippocampus Relation Cue Context Response Neo-Cortex Kwok & McClelland Model • Model includes slow learning cortical system representing the content of an association and the context. • Hidden units in neo-cortex mediate associative learning. • Cortical network is pre-trained with several cue-relation-response triples for each of 20 different cues. • When tested just with ‘cue’ as probe it tends to produce different targets with different probabilities: • Dog (chews) bone (~.30) • Dog (chases) cat (~.05) • Then the network is shown cue-response-context triples. Hippo. learns fast and cortex learns (very) slowly. • Hippocampal and cortical networks work together at recall, so that even weak hippocampal learning can increase probability of settling to a very strong pre-existing association.

34. Data with Simulation Results From K&M Model