Two-Stage Model of Memory Formation: Neuromodulation

1. Neuromodulation: acetylcholine and memory consolidation Michael E. Hasselmo Trends in Cognitive Sciences – Vol. 3, No. 9, September 1999

2. Why „two-stage model“ of long-term memory storage? • traditional concept of „long-term“ memory can be divided into intermediate – term episodic memory  sensitive to lesions of hippocampus and long – term episodic and semantic memory  not sensitive to hipp. lesions1,2 • suggestion, that these properties of memory might result from a two- stage process for long-term memory formation4-7

3. { active waking EEG: theta { quiet waking, SWS EEG: SPWs Two-stage model of long-term memory formation (Hasselmo). information from environment rapid encoding intermediate-term episodic representation (excitatory connections between CA3 pyramids are strengthened that are activated by the event4,7-9) re-activation of the stored representations including spread of activity across the strengthened excitatory connections6,10 further strengthening of CA3-CA3 and CA3-CA1 connections4,9 activation of neurons representing the event in EC5 and association neocortex11 slower strengthening of the excitatory associative synaptic connections between neurons representing these features in neocortex  Formation of links between these neurons that could be described as semantic or long-term episodic memories in neocortex4,7,8

4. Neuropsychological evidence for the „two-stage model“(1) • Ribot‘s law: the most recently encoded episodic memories are the most sensitive • to hippocampal damage (process: „temporally graded amnesia“3) evidence:  in subjects with damage to hippocampal subregions the temporal extent of retrograde amnesia depends on number of regions involved2  subjects having been treated with electroconvulsive therapy1  monkeys trained on object recognition prior to a hippocampal lesion, impairment was - greater, when lesion followed training within 0 – 25 days - minor in monkeys with lesion 75 days after training12  in rats having been trained to associate a particular testing location (=context) with a tone-shock pairing, the ability to associate was - greater if hippocampal lesions are performed 0 – 2 days after tone-shock pairing - reduced if lesions were applied 5 – 10 days later13 (location [=context] learning depends on hippocampus; tone-shock pairing depends on amygdala)  neuropsychological data supporting the „two-stage model“

5. Neuropsychological evidence for the „two-stage model“(2) criticism on concept of hippocampal activity to guide the formation of neocortical associations14,15:  animal studies: extensive lesions: flat gradient of retrograde amnesia partial lesions: temporally graded retrograde amnesia14 • human data: hippocampal damage caused a very long-term retrograde amnesia15 (but: these lesions included subiculum and EC whereas very short-tem retrograde amnesia occurs when damage is restricted to specific hippocampal subfields2) • thus, consolidation does not neccessarily result in formation of links in association neocortex, but could instead result in strengthening of representations within hippocampus itself or within the EC • „multiple-trace“ hypothesis15possible also to be valid! (= reactivation of memories and formation of additional representations within the hippocampal formation; in this scenario memories are reactivated in CA3 and guide strengthening of associations in CA1 or the EC4) • but: also the process of forming multiple traces requires two stages with different dynamics: first for encoding of new sensory information second for reactivation of an old memory to form additional traces  whole process takes place within the hippocampus

6. Computational evidence for the „two-stage model • diverse computational models describing the interaction between hippocampus and • neocortex when reactivating representations for consolidation7,8,16-18 • key feature of these models: •  initial storage of an association in an „associative matrix“ (= component of hippocampus) • which requires capacity for rapid synaptic modification in the hippocampus • (LTP as a physiological correlate?9)  then, stored representations from this matrix are repetitively retrieved to activate units in association neocortex (this repetitive reactivation results in gradual strengthening of the connections between neurons in the associative neocortex) = slow process compared to synaptic modification within the hippocampus (experimentally supported by observation that LTP in rat association neocortex is enhanced when stimulation is spaced and repeated over a series of days20)

7. Computational evidence for the „two-stage model (2) • why two stages? 1. discussed in terms of modification of existing semantic representations in the neocortex3 example: network established that codes a range of items of semantic knowledge here: „birds can fly“ („robins are birds that fly… sparrows are birds that fly…“) if such a network were sequentially trained with a new item inconsistent with previous input: „penguins are birds; penguins swim but do not fly“  this new information could interfere with the previous learned knowledge and could lead to the belief that all birds swim but: if the new information is stored as an episodic memory in hippocampal formation and later inter- leaved with other examples during training of neocortical representations during consolidation „robins are birds that fly… sparrows are birds that fly… penguins are birds that swim… eagles are birds that fly“  new information can be incorporated as an exception in semantic networks  interleaved learning of multiple different examples is necessary for the formation of efficient representation in a number of different models (e.g. 21,22) 2. in models focussing on formation of representations in multi-layer hierachical networks23,24 alternate phases of dominant feedforward versus dominant feedback connection have been used  examples of computational models of memory consolidation favouring two-stage architecture

8. Electrophysiological evidence for the „two-stage model“  hippocampal networks appear to reactivate external stimuli they have responded to during active waking also during quiet waking and slow-wave sleep • (experimental evidence: • place cells that fired during waking have an enhanced probability to fire again in a • subsequent sleep episode25 • pairs of place cells that code adjacent positions during active waking show more • correlated firing during subsequent SWS6,10,11)  experiments demonstrating direction of flow of activity in the hippocampal formation: • during active waking (theta present): extensive neuronal activity in layer II of the • EC (= layer providing input to the hipocampus)5 • during quiet waking and slow-wave sleep, SPWs selectively activate hippocampal • target structures in the EC („deep“ layers V and VI) but not superficial layers5 • also demonstration of correlation between sharp wave-associating ripples an sleep • spindles of prefrontal cortex26 • suggests that hippocampus could be inducing co-activation of neurons in neocortical regions, which could form new crossmodal associations27

9. Neuromodulation during sleep-wake cycle  the two stage model requires different dynamic states during each stage  during active waking: predominant influence of EC on CA3 Feedback connections arising from CA3 are functional but do not dominate over the feedforward connection • during quiet waking and slow-wave sleep: predominant influence of CA3 on CA1 and EC  how are these different dynamic states modulated? • Hypothesis: modulation mediated by acetylcholine, which shows parallel fluctuations during different stages of waking and sleep

10. Changes in cortical neuromodulation during waking, slow-wave sleep and REM sleep hippocampus locus coeruleus frontal cortex Fig. 3. Histogram bars represent levels of neuromodulators (shown at left) in cortical structures relative to levels measured during waking (here normalized to be the same for all neuromodulators). Acetylcholine (Ach) levels are based on microdialysis measurements in hippocampus[32 and 33]. Levels of acetylcholine do not increase as much in the cortex during REM sleep [33]. For norepinephrine (NE), level of unit activity in the locus coeruleus is shown [75 and 85]. Serotonin (5HT) levels are based on microdialysis in the frontal cortex [77].

11. Neuromodulation during sleep-wake cycle - active waking (AW) • defined by behaviour (exploration, sniffing, extensive rearing-up) and • EEG (large-amplitude theta31 in hippoc. and EC) • microdialysis measurements: high levels of ACh release32,33 • influence of hippoc. on EC is weak during AW • (as shown by - low spiking activity in deep layers of EC5 • - small amplitude of EC field potentials arising • from hippoc. connections36,37) • effects of ACh could provide a specific mechanism for weakening the • hippocampal feedback during AW • (experimental evidence from brain slice or intact animal: • suppressive effect of ACh at excit. glut. synapt. transmission at CA3  EC • feedback connections24,38-45 • but weaker effects on feedforward connections to hippocampus42-44,46,47  Fig. 4) • specific effects of ACh (brain slices): • suppresses transmission at excitatory recurrent collaterals in CA344,48 • suppresses transmission at Schaffer collaterals38,39,43,49,50 • suppresses transmission at CA1  Subiculum connection45

12. Neuromodulation during sleep-wake cycle - active waking (AW) 2 • effects of ACh (in vivo data) • CA3  CA1 connection is weakened (reduced EPSPs) at • local ACh-infusion40-42 • stimulation of cholinergic innervation from the medial septum54 • sensory stimulation46 (effect blocked by atropine42) • Taken together, ACh-mediated suppression of excitatory feedback would act to reduce the influence of hippocampus on EC and other cortical areas during active waking But: no total suppression of feedback from hippocampus  Sufficient feedback for retrieval of relevant stored information ACh does not suppress most of feedforward connections to hippocampus - PP input to CA1 much weaker suppressed than CA3CA1 connection within the same slice43 - direct ECCA3 input also less suppressed48

13. Neuromodulation during sleep-wake cycle - active waking (AW) 3 • specific cholinergic effects on dentate gyrus: • effects of ACh on mossy fiber  CA3 synapse • cholinergic suppression also present 57, but weaker than associational inputs • to CA3 radiatum at comparable doses43 • no cholinergic suppression in the outer molecular layer of dentate gyrus47,58,59, • (PP input from lateral EC) • cholinergic suppression does appear • in the middle molecular layer47(input from medial EC) • in the inner molecular layer59 (commissural input from contralatral dentate gyrus) • cholinergic influence on PP input to dentate gyrus is weak (experimentally supported • by fact that angular bundle-evoked EPSPs in the dentate gyrus are larger during • high ACh-levels of active waking than during low levels of SWS)36,60 • medial septum-stimulation leads to increases in population spike activity in the dentate gyrus • while having no systematic effect on EPSPs61,62 • (possible explanation for this enhanced spiking: GABAergic input from MS; • absence of an effect on EPSPs suggests that feedforward input is not suppressed) • postsynaptic effects of ACh (which would enhance the response of neurons • to feedforward input64): • - depolarization of pyramidal cells65,66 • suppression of spike frequency adaptation67,68

14. Neuromodulation during sleep-wake cycle - active waking (AW) 4 functional purpose of suppressing the feedback to EC during active waking? • suppression is not total, as recently stored memories from hippocampus are still • accessible for retrieval • but: hazard that the strength of connections neccessary for the strong transmission of • stored memories back to CA1 and EC would dominate over afferent input  this could distort the initial perception of sensory information, causing interference during learning in temporal structures and – if retrieval activity is sufficiently strong – would cause hallucinations (as mediated by the influence of cholinergic antagonists at high doses) • thus: partial cholinergic suppression of excitatory feedback allows cued retrieval • without hallucinatory retrieval

15. Behavioural effects of cholinergic antagonists • during active waking, high levels of ACh could result in a predominant feedforward • flow of information, allowing normal interaction with the environment • what happens, when – during waking – reduction of the cholinergic effects is induced • by administration of a muscarinic receptor antagonist, e.g. scopolamine? • prediction 1: enhancement of feedback effects in the cortex and this will interfere • with the feedforward sensory input to hippocampus  low doses of scop. impair encoding of new information in human subjects but do not effect (even slightly enhance!) retrievala,b  higher doses would cause stronger feedback effects, which might be the reason for the described hallucinogenic effects of muscarinic antagonistsc-e • prediction 2: when cholinergic modulation is blocked after learning of paired associates, • this should interfere with learning of subsequent overlapping paired associates  preliminary supporting data (then still unpublished): - learning of related paired associates (A-B, lotion-bottle) - then: injection of scopolamine - new paired associates, either/or not overlapping (A-C, lotion-oil vs. D-E, kitchen-spoon) - then test: greater impairment after learning of overlapping paired associates

16. Neuromodulation during sleep-wake cycle - quiet waking • defined by behaviour (grooming, eating5,30,33) and EEG (Sharp waves in hippocampus5,37) • microdialysis measurements: decrease of levels of ACh • (~60% of those during active waking33) - this decrease would release glutamatergic synapses from cholinergic suppression (experimental evidence: evoked synaptic potentials are much larger in CA1 or in EC during quiet waking than during active waking60or when a rat is not being presented with sensory stimulation46) • possibly drop of cholinergic inhibition also explains generation of SPWs and - stronger feedback from CA3 to EC would then allow SPWs to spread from CA3  CA1  EC4,5,37 consistent: finding that „retrohippocampal“ structures (deep layers of EC) show much stronger spiking activity during SPWs of quiet waking than during theta activity of active waking5

17. Neuromodulation during sleep-wake cycle - slow-wave sleep (1) • defined by EEG pattern (delta waves in neocortex) • microdialysis measurements: decrease of levels of ACh • (~33% of those during active waking32,33) • this drop in ACh levels would further release glutamatergic synapses from • cholinergic suppression  even stronger excitatory feedback than during quiet waking • (= very large increase in the effect of excitatory recurrent connections within • CA3 and excitatory feedback connections from CA3 to CA1 and EC) • this drop could therefore underly the increase of SPW activity during SWS4,37 • the relese of suppression of excitatory transmission could contribute to the greater • tendency of cells to fire together during SWS if they fired in the previous waking period6,10,11 • loss of cholinergic suppression should also enhance the spread of activity in response • to stimulation – • shown for CA1 and EC • ( increased magnitude of evoked synaptic potentials during SWS36,30)

18. Neuromodulation during sleep-wake cycle - slow-wave sleep (2) • what functional role could this enhancement of excitatory feedback have? • provide the appropriate dynamics for the formation of additional traces within CA3 and CA1, and could allow the hippocampus to „train“ the EC or association neocortex on the basis of previously encoded associations • not all information for episodic memories is stored in the hippocampus but CA3 region provides a mechanism for linking together disparate information from multiple regions of association neocortex; • during waking, these links can be used for cued retrieval of recently stored information by neocortex, but they should not be able to strongly drive neocortical activity to the level of distorting sensory inputs (hazard of interference leading to hallucinations!) • during SWS, the associative links formed in the hippocampus need to be reactivated and need to influence other brain areas in a strong enough manner to drive the slower modifications of neocortical synapses • neurons coding an association in the hippocampus must be able to drive cells in EC and neocortex without any assistance of sensory input – reduction of cholinergic suppression might provide the opportunity for this strong feedback influence; additionally: physiological activity during SWS has been proposed to be appropriate for modification of synaptic components70

19. Neuromodulation during sleep-wake cycle - slow-wave sleep (3) • evidence for the importance of SWS for episodic memories: • subjects are better at retrieval of word lists if they learn the list before falling asleep and are tested on retrieval when awakened in the middle of the night, than if they learn the list after some hours of sleep and are tested in the morning71  this suggests greater importance of SWS for memory consolidation because most SWS occurs in the early part of the night • if semantic memory is constantly being reshaped during the consolidation process in SWS, then sustained disruption of SWS should impair semantic memory  certain types of epileptic syndromes display constant seizure activity during SWS72,73 which can cause severe impairments of memory function (also delusion, loss of language function)73

20. Neuromodulation during sleep-wake cycle - REM sleep (1) • defined by EEG pattern (similar to that observed during waking) and rapid eye movements, muscular • atonia, muscle twitches31, ponto-geniculo-occipital waves33,74 and theta oscillations • in the hippocampus • microdialysis measurements: ACh levels in the hippocampus increase to levels above those seen • during active waking32,33  levels of ACh resemble those seen during waking • contrasting: levels of other neuromodulators: • noradrenergic neurons projecting to neocortex decrease in activity to low levels during SWS and • show no activity during REM sleep75,76 • serotonergic neurons projecting to neocortex also decrease activity during REM (recordings from • raphe nuclei75 and microdialysis of serotonin in frontal cortex77) • Function of this change in neuromodulatory state? - norepinephrine suppresses feedback excitatory synaptic transmission in the piriform cortex, with a much weaker effect on feedforward transmission78 - norepinephrine does not suppress feedback within hippocampal formation80 • Hypothesis: - during waking: high levels of ACh and norepinephrine shut down recurrent connections in neocortex - during REM sleep: high levels of ACh in hippocampus might suppress feedback there, the lower levels of ACh and very low levels of norepinephrine in cortex might allow spread of activity in neocortex without strong influence from hippocampus

21. Neuromodulation during sleep-wake cycle - REM sleep (2) Theoretical framework: • waking involves initial encoding of episodic representations in the hippocampus  SWS would allow transmission of episodic representations from hippocampus to EC and association cortex  REM would allow neocortical structures to undergo a re-analysis, in which this episodic information would be re-interpreted in relation to previous semantic representations  REM could represent the development of new feedforward representations for behavior (prediction: REM would therefore also be important for procedural tasks  experiment: evidence, that performance in procedural memory task seems to depend on quantity of REM sleep81-84)  process of altering neocortical representations during REM sleep could utilize the information clamped on the neocortex by hippocampal feedback during SWS  indeed: perceptual learning task performance correlated most strongly with SWS during first quarter of the night and REM sleep during the last quarter of the night83,84

22. Two-stage model of long-term memory formation Fig. 1.. (A)During active waking, information coded by neocortical structures flows through the entorhinal cortex and dentate gyrus (DG) into hippocampal region CA3 (connections less sensitive to modulation by ACh; thick arrows). Here, synaptic modification forms an intermediate-term representation, binding together different elements of an episodic memory. Connections suppressed by ACh modulation (thin arrows) to region CA1, entorhinal cortex and association cortex are strong enough to mediate immediate retrieval, but do not overwhelm the feedforward connectivity. (B) During quiet waking or slow-wave sleep, memories are reactivated in region CA3 during EEG phenomena termed sharp waves. These waves of activity flow back through region CA1 to entorhinal cortex[5] and neocortex [26]. This will enable the slow consolidation (formation of separate traces) of long-term episodic memory in hippocampal region CA1, entorhinal cortex and association neocortex, and might underlie modification of semantic memory within circuits of association neocortex.

23. Two-stage model of long-term memory formation Fig. 1.. (A) During active waking, information coded by neocortical structures flows through the entorhinal cortex and dentate gyrus (DG) into hippocampal region CA3 (connections less sensitive to modulation by ACh; thick arrows). Here, synaptic modification forms an intermediate-term representation, binding together different elements of an episodic memory. Connections suppressed by ACh modulation (thin arrows) to region CA1, entorhinal cortex and association cortex are strong enough to mediate immediate retrieval, but do not overwhelm the feedforward connectivity. (B) During quiet waking or slow-wave sleep, memories are reactivated in region CA3 during EEG phenomena termed sharp waves. These waves of activity flow back through region CA1 to entorhinal cortex[5] and neocortex [26]. This will enable the slow consolidation (formation of separate traces) of long-term episodic memory in hippocampal region CA1, entorhinal cortex and association neocortex, and might underlie modification of semantic memory within circuits of association neocortex.