Tutorial: Plasticity Revisited - Motivating New Algorithms Based On Recent Neuroscience Research

1. Tutorial: Plasticity Revisited -Motivating New Algorithms Based On Recent Neuroscience Research TsviAchlerMD/PhD Approximate Outline and References for Tutorial Department of Computer Science University of Illinois at Urbana-Champaign, Urbana, IL 61801, U.S.A.

2. Intrinsic Plasticity: Synaptic Outline Homeostatic ‘Systems’ • Plasticity is observed in many forms. We review experiments and controversies. • Intrinsic ‘membrane plasticity’ • Synaptic • Homeostatic ‘feedback plasticity’ • System: in combination membrane and feedback can imitate synaptic • What does this mean for NN algorithms?

3. Synaptic Plasticity Algorithms: Lateral Inhibition Outline: NN Algorithms Feedback Inhibition Common computational Issues • Explosion in connectivity • Explosion in training • How can nature solve these problems with the plasticity mechanisms outlined?

4. 1. Plasticity

5. Intrinsic Intrinsic ‘Membrane’ Plasticity • Ion channels responsible for activity, spikes • ‘Plastic’ ion channels found in membrane • Voltage sensitive channel types: • (Ca++, Na+, K+) • Plasticity independent of synapse plasticity Review: G. Daoudal, D, Debanne, Long-Term Plasticity of Intrinsic Excitability: Learning Rules and Mechanisms, Learn. Mem. 2003 10: 456-465

6. Synaptic Synaptic Plasticity Hypothesis • Bulk of studies • Synapse changes with activation • Motivated by Hebb 1949 • Supported by Long Term Potentiation / Depression (LTP/LTD) experiments Review: Malenka, R. C. and M. F. Bear (2004). "LTP and LTD: an embarrassment of riches." Neuron 44(1): 5-21.

7. Synaptic LTP/LTD Experiment Protocol Pre-synaptic electrode 50% ? A50 A50 • Establish ‘pre-synaptic’ cell • Establish ‘post-synaptic’ cell • Raise pre-synaptic activity to amplitude to A50 where post-synaptic cell fires “50%” • Induction: high frequency high voltage spike train on both pre & post electrodes • Plasticity: any changes when A50 is applied Post-synaptic electrode Brain

8. Synaptic Plasticity: change in post with A50 • LTP : increased activity with A50 • LTD : decreased activity with A50 • Can last minutes hours days • Limited by how long recording is viable

9. Synaptic Strongest Evidence • Systems w/minimal feedback: • Motor, Musculature & tetanic stimulation • Sensory/muscle junction of Aplesia Gill Siphon Reflex • Early Development: Retina → Ocular Dominance Columns Variable Evidence: areas with feedback Cortex, Thalamus, Sensory System, Hippocampus • Basic relations between pre-post cells • Basic mechanisms of Synaptic Plasticity are Still controversial Why so difficult? Connections change with activity Strong evidence: Muscles, early development (occular dominance colums) Tetanic stimulation Applesia siphon responses Supported by Long Term Potentiation (LTP) experiments

10. Synaptic Variable Evidence Cortex, Thalamus, Sensory Systems & Hippocampus • Basic mechanisms still controversial 60 years and 13,000 papers in pubmed • It is difficult to establish/control when LTP or LTD occurs

11. Synaptic LTP vs LTD Criteria is Variable • Pre-Post spike timing: (Bi & Poo 1998; Markram et al. 1997) • Pre-synaptic spike before post  LTP • Post-synaptic spike before pre  LTD: • First spike in burst most important (Froemke & Dan 2002) • Last spike most important (Wang et al. 2005) • Frequency most important:  Freq  LTP (Sjöström et al. 2001; Tzounopoulos et al. 2004). • Spikes are not necessary (Golding et al. 2002; Lisman & Spruston 2005) • The criteria to induce LTP or LTD are also a current subject of debate (Froemke et al, 2006). Some studies find that if the presynaptic neuron is activated within tens of milliseconds before the postsynaptic neuron (pre-post), LTP is induced. The reversed order of firing (post-pre) results in LTD (ie Bi & Poo 1998; Markram et al. 1997). In other studies, timing of the first spike (Froemke & Dan 2002) or the last spike (Wang et al. 2005) in each burst is found to be dominant in determining the sign and magnitude of LTP. Yet other studies show that synaptic modification is frequency dependent and that high-frequency bursts of pre- and postsynaptic spikes lead to LTP, regardless of the relative spike timing (Sjöström et al. 2001; Tzounopoulos et al. 2004). Even other studies show that somatic spikes are not even necessary for the induction of LTP and LTD (Golding et al. 2002; Lisman & Spruston 2005). • In addition, it is unclear if these mechanisms drive single synapse changes as predicted by synaptic plasticity because physical changes in synapses show variability as well. • Synaptic Change • Activity dependent changes in synaptic spine morphology has been reported in the hippocampus (see Yuste et al. 2001 for review) including localized changes to single synapses with caged-glutamate sub-spike stimulation (Mazrahi et al., 2004). However changes in synapses can also occur with: estrus cycle (Woolley et al 1990), irradiation (Brizzee 1980), hibernation (Popov et al 1992), exercise (ie. Fordyce & Wehner 1993), epilepsy (Multani et al., 1994) and synaptic blockade using Mg+ (Kirkov & Harris, 1999). • Also the synaptic morphology may not coincide with synaptic/behavioral function (Yuste et al. 2001). • Furthermore brain regions responsible for recognition processing may display different characteristics. For example synaptic changes with experience in the mouse barrel cortex appear to be more variable than the hippocampus (Trachtenberg et al 2002). • The strongest evidence supporting the synaptic plasticity hypothesis has been reported in the gill withdrawal reflex of the marine mollusk aplysia (Antonov, Antonova & Kandel, 2003). However the changes occur between sensory and motor neurons, not between processes responsible for recognizing stimuli. It may be the case that motor learning occurs via synaptic plasticity while recognition processing occurs through recurrent feedback. Furthermore, even if synaptic plasticity is found in the sensory cortex, it may be specific to pre-motor processing. Such pre-motor processes may co-exist with recognition circuits in the same regions

12. Synaptic Many factors affect LTP & LTD • Voltage sensitive channels ie. NMDA • Cell signaling channels ie via Ca++ • Protein dependent components • Fast/slow • Synaptic tagging Review: Malenka, R. C. and M. F. Bear (2004). "LTP and LTD: an embarrassment of riches." Neuron 44(1): 5-21.

13. Synaptic Studies of Morphology Unclear Synapse Morphology and density studies: • Spine changes ≠ Function changes • Many other causes of changes in spines: • Estrus, Exercise, Hibernation, Epilepsy, Irradiation Review: Yuste, R. and T. Bonhoeffer (2001). "Morphological changes in dendritic spines associated with long-term synaptic plasticity." Annu Rev Neurosci 24: 1071-89.

14. Synaptic Many Components & Variability • Indicates a system is complex • involving more than just the recorded pre-synaptic and postsynaptic cells • Means NN learning algorithms are difficult to justify • But the system regulates itself Review of LTP & LTD variability: Froemke, Tsay, Raad, Long, Dan, Yet al. (2006) J Neurophysiol 95(3): 1620-9.

15. Homeostatic Homeostatic Plasticity Self-Regulating Plasticity Networks Adapt to: Channel Blockers Genetic Expression of Channels

16. Adaptation to Blockers Homeostatic Post-synaptic electrode Pre-synaptic electrode Pre-Synaptic Cell Post-Synaptic Cell • Establish baseline recording • Bathe culture in channel blocker (2 types) • Either ↑ or ↓ Firing Frequency • Observe System changes after ~1 day • Washing out blocker causes reverse phenomena Culture Dish

17. Homeostatic Adaptation to Blockers Pre-Synaptic Cell Post-Synaptic Cell Displays Feedback Inhibition Response ↑ Frequency → ↓ Frequency → Frequency x Strength = Baseline → ↓ Synaptic Strength → ↑ Synaptic Strength Turrigiano & Nelson (2004)

18. Homeostatic Homeostatic Adaptation to Expression Cell Channels Involved 1 2 3 Marder & Goaillard (2006) Cells with different numbers & types of channels show same electrical properties

19. Homeostatic Homeostatic Summary • Adapts networks to a homeostatic baseline • Utilizes feedback-inhibition (regulation)

20. Homeostatic Feedback Inhibition Pre-Synaptic Cell Post-Synaptic Cell Feedback Ubiquitously Throughout Brain

21. Homeostatic Feedback Throughout Brain Thalamus & Cortex Nice lnk but pictures poor quality http://images.google.com/imgres?imgurl=http://www.benbest.com/science/anatmind/FigVII6.gif&imgrefurl=http://www.benbest.com/science/anatmind/anatmd7.html&h=320&w=1193&sz=19&tbnid=dhilzMBJy4rbFM:&tbnh=40&tbnw=150&hl=en&start=5&prev=/images%3Fq%3Dthalamus%2Bfeedback%26svnum%3D10%26hl%3Den%26lr%3D%26client%3Dfirefox-a%26rls%3Dorg.mozilla:en-US:official_s%26sa%3DG http://arken.umb.no/~compneuro/figs/LGN-circuit.jpg LaBerge, D. (1997) "Attention, Awareness, and the Triangular Circuit". Consciousness and Cognition, 6, 149-181 http://psyche.cs.monash.edu.au/v4/psyche-4-07-laberge.html

22. Figure from Aroniadou-Anderjaska, Zhou, Priest, Ennis & Shipley 2000 Homeostatic Overwhelming Amount of Feedback Inhibition Feedback and Pre-Synaptic Inhibition found in Many Forms • Feedback loops • Tri-synaptic connections • Antidromic Activation • NO (nitric oxide) • Homeostatic Plasticity Regulatory Mechanisms Suggest Pre-Synaptic Feedback Modified from Chen, Xiong & Shepherd (2000). • Feedback loops • 3 cells • Antidromic • Adjustment of pre-synaptic processes in Homeostatic Plasticity Feedback & Pre-Synaptic Inhibition Evidence has Many Forms

23. Homeostatic Summary • Homeostatic Plasticity requires and maintains Feedback Inhibition

24. ‘Systems’ ‘Systems’ Plasticity Feedback Inhibition combined with Intrinsic Plasticity Can be Indistinguishable from Synaptic Plasticity

25. ‘Systems’ Many cells are always present in plasticity experiments • Pre & Post synaptic cells are never in isolation • Studies: • In Vivo • Brain slices • Cultures: only viable with 1000’s of cells Post-synaptic electrode Pre-synaptic electrode Culture Dish Changes in neuron resting activity is tolerated

26. ∆↓ ∆↓ ∆↓ ∆↑ ∆↓ ∆↓ ‘Systems’ Feedback Inhibition Network Then learning is induced artificially by activating both neurons together Increase pre-synaptic cell activity until Induction can affect all connected post-synaptic cells Immeasurable changes of all connected neurons Only the two recorded cells and the synapse between them are considered With Pre-Synaptic Inhibition recorded postsynaptic cell fires 50% Pre-synaptic cells connect to many post-synaptic cells but this is rarely considered Causes big change in the recorded neuron LTP protocol: find pre-synaptic and post-synaptic cells

27. All Neurons 0.01 Baseline ‘Systems’ Simulation: Up to 26 Cell Interaction Immeasurable changes of all connected neurons LTP LTD 1 Normalized Activity Scale (0-1) 0.9 Resting ∆ Value 0.8 0.7 0.6 Causes big change in the recorded neuron 0.5 0.4 LTP = bias recorded cell 0.3 Lternatively LTD = negatively bias recorded cell 0.2 0.1 0

28. ‘Systems’ Significance Experiments can not distinguish between synaptic plasticity and feedback inhibition • Membrane voltage Vm allowed Δ ~6mV • 0.01 = ~∆Vm of 0.3 mV • Thus not likely to see membrane affects • Presynaptic cells connect to >> 26 cells • Effect much more pronounced in real networks

29. ‘Systems’ Regulatory Feedback Plasticity • Feedback Inhibition + Intrinsic Plasticity are indistinguishable in current experiments from Synaptic Plasticity theory • Why have ‘apparent’ synaptic plasticity? • Feedback Inhibition is important for processing simultaneous patterns

30. Break

31. Synaptic Plasticity Lateral Inhibition 2. Algorithms Feedback Inhibition

32. Y2 Y2 I2 I1 I3 Challenges In Neural Network Understanding Regulatory Feedback Limited Cognitive Intuition Large Network Problems lw13 Y1 Y3 Y4 Y1 Lateral Connections: connectivity explosion Y2 Y3 lw12 lw23 w31 w21 w33 w32 w43 w22 w12 I4 w23 w11 w13 w42 w41 x4 x1 x2 x4 x3 x1 x2 x3 Input Feedback Neural Networks Weights w22 Strong evidence of feedback Replace with binary bidirectional connections. Y1 Y3 Y4 w23 Could feedback dynamics be necessary vital? Strong evidence of feedback Replace with binary bidirectional connections. w34 w31 w21 w24 w44 w32 w22 w12 w43 w33 w23 w11 w13 w42 w14 w41 x4 x1 x2 x3

33. Y1 Y2 Y3 Lateral Connectivity x2 x3 x4 x1 Millions of representations possible -> a connection required to logically relate between representations Every representation can not be connected to all others in the brain Combinatorial Explosion in Connectivity Can lead to an implausible number of connections and variables Symbolic Logic based on direct connections Combinatorial Explosion in Connectivity What would a weight variable between them mean? 0.8   ?

34. Y2 Y2 I2 I1 I3 Challenges In Neural Network Understanding Regulatory Feedback Large Network Problems lw13 Y1 Y3 Y4 Y1 Lateral Connections: connectivity explosion Y2 Y3 lw12 lw23 w31 w31 w21 w21 Weights: combinatorial training w33 w33 w32 w32 w43 w43 w22 w22 w12 w12 I4 w23 w23 w11 w11 w13 w13 w42 w42 w41 w41 x4 x1 x2 x4 x3 x1 x2 x3 Input Feedback Neural Networks Weights w22 Strong evidence of feedback Replace with binary bidirectional connections. Y1 Y3 Y4 w23 Could feedback dynamics be necessary vital? Strong evidence of feedback Replace with binary bidirectional connections. w34 w31 w21 w24 w44 w32 w22 w12 w43 w33 w23 w11 w13 w42 w14 w41 x4 x1 x2 x3

35. Y1 Y2 Y3 Weights: Training Difficulty Given Simultaneous Patterns Superposition Catastrophe x4 x3 x1 x2 • Teach A B C … Z separately • Test multiple simultaneous letters A D B D A B E A C G E w31 w21 Not Taught with simultaneous patterns: Will not recognize simultaneous patterns Teaching simultaneous patterns is a combinatorial problem w33 w32 w43 w12 w22 w23 w11 w13 w42 w41

36. Y1 Y2 Y3 Weights: Training Difficulty Given Simultaneous Patterns x4 x3 x1 x2 • Teach A B C … Z separately • Test multiple simultaneous letters A D G E w31 w21 w33 w32 w43 w12 w22 ‘Superposition Catastrophe’ (Rosenblatt 1962) Can try to avoid by this segmenting each pattern individually but it often requires recognition or not possible w23 w11 w13 w42 w41

37. Composites Common • Natural Scenarios (cluttered rainforest) • Scenes • Noisy ‘Cocktail Party’ Conversations • Odorant or Taste Mixes Segmentation not trivial Segmentation is not possible in most modalities (requires recognition?)

38. Y1 Y2 Y3 Segmenting Composites x4 x3 x1 x2 New Scenario: Learn: If can’t segment image must interpret composite Building 1 0 0 1 1 0 0 1 0 1 0 1 0 1 0 1 1 1 0 2 Feature Space Feature Space Letters learned individually w31 w21 Chick w33 w32 w43 w12 w22 + = w23 w11 w13 w42 w41 A B C D Chick & Frog Simultaneously B Frog A 0 1 0 0 1 Simultaneous 2 4 0 1 2 B 1 1 0 0 0 B 1 1 0 0 0 C 0 1 0 1 1 Feature Space Recognition Given Simultaneous: A, Bx2, and C + Train Test

39. Y2 Y2 I2 I1 I3 Challenges In Neural Network Understanding Regulatory Feedback Large Network Problems Y1 Y3 Y4 Y1 Lateral Connections: connectivity explosion Y2 Y3 w31 w21 Weights: combinatorial training w33 w32 w43 w22 w12 I4 w23 w11 w13 w42 w41 x4 x1 x2 x4 x3 x1 x2 x3 Feedback Inhibition: avoids combinatorial issues interprets composites Input Feedback Neural Networks Weights w22 Strong evidence of feedback Replace with binary bidirectional connections. Y1 Y3 Y4 w23 Could feedback dynamics be necessary vital? Strong evidence of feedback Replace with binary bidirectional connections. w34 w31 w21 w24 w44 w32 w22 w12 w43 w33 w23 w11 w13 w42 w14 w41 x4 x1 x2 x3

40. Self-Regulatory Feedback Feedback Inhibition Control Theory Perspective Feedback Inhibition Every output inhibits only its own inputs • Gain control mech for each input • Massive feedback to inputs • Iteratively evaluates input use • Avoids optimized weight parameters • Training establishes binary relationships • Testing iteratively evaluates input use Output Input Neuroscience Perspective ya yb Output Network I2 I1 Input x1 x2 x1 x2

41. Equations Used Feedback Inhibition X b Î j ya yb Y Xb Raw Input Activity a X b I2 I1 I b x1 x2 x1 x2 Q b Q shunting inhibition. Qb shunting inhibition at input b. C collection of all output cells Ca cell “a”. Na the set of input connections to cell Ca. na the number of processes in set Na of cell Ca. P primary inputs (not affected by shunting inhibition). I collection of all inputs Ibinput cell “b”. Mb the set of recurrent feedback connections to input Ib. mb the number of connections in set Mb

42. Equations Feedback Inhibition X = b I b Q b Î j ya yb Y Xb Raw Input Activity Ib Input after feedback Qb Feedback a X b I2 I1 I b x1 x2 x1 x2 Q b Q shunting inhibition. Qb shunting inhibition at input b. C collection of all output cells Ca cell “a”. Na the set of input connections to cell Ca. na the number of processes in set Na of cell Ca. P primary inputs (not affected by shunting inhibition). I collection of all inputs Ibinput cell “b”. Mb the set of recurrent feedback connections to input Ib. mb the number of connections in set Mb

43. Equations Feedback Inhibition Y ( t ) å + = Output D a Y ( t t ) I a i n Î a i Y a X = b I Inhibition b Q b å = Q Y ( t ) Feedback b j Î X j Î j b ya yb Y Ya Output Activity Xb Raw Input Activity Ib Input after feedback Qb Feedback na # connections of Ya a X W b I2 Q2=yb I1 = Q1=ya+yb = I b x1 x2 x1 x2 Q b Q shunting inhibition. Qb shunting inhibition at input b. C collection of all output cells Ca cell “a”. Na the set of input connections to cell Ca. na the number of processes in set Na of cell Ca. P primary inputs (not affected by shunting inhibition). I collection of all inputs Ibinput cell “b”. Mb the set of recurrent feedback connections to input Ib. mb the number of connections in set Mb

44. Equations Feedback Inhibition Y ( t ) å + = Output D a Y ( t t ) I a i n Î a i Y a X = b I Inhibition b Q b å = Q Y ( t ) Feedback b j Î X j Î j b ya yb No Oscillations No Chaos Repeat I2 x2 I1 = x1 = Q2=yb Q1=ya+yb x1 x2 C collection of all output cells Ca cell “a”. Na the set of input connections to cell Ca. na the number of processes in set Na of cell Ca. P primary inputs (not affected by shunting inhibition). I collection of all inputs Ibinput cell “b”. Mb the set of recurrent feedback connections to input Ib. mb the number of connections in set Mb Q shunting inhibition. Qb shunting inhibition at input b.

45. Y2 Y2 I2 I2 I1 I1 I3 I3 Feedback Inhibition Simple Connectivity Y1 Y3 Y4 Output Nodes W I4 Input Nodes x4 x1 x2 x3 Source of Training Problems All links have same strength New node only connects to its inputs Source of Connectivity Problems Inputs have positive real values indicating intensity

46. Y2 I2 I1 Feedback Inhibition Allows Modular Combinations ‘P’ ‘R’ Outputs Y1 1 0 1 1 Features Features I1 Inputs

47. Inputs Steady State: (C1, C2) (PA, PB) Inputs (PA, PB) Results (C1, C2) (½, ½) (x1 ≥ x2) (x1–x2, x2) (x1≤ x2) (0, (x1+x2)/2) (1, ¼) (⅓, 1) (0, ½) (¾, ¼) (0, ⅔) Algorithm Interprets Composite Patterns Steady State Network Configuration Inputs x1 , x2 Outputs y1 , y2 y1 y2 ( - ) Outputs 1 , 0 1 , 0 ‘P’ 1 , 1 0 , 1 ‘R’ x1 x2 0 , 2 2 , 2 2Rs Supports Non-Binary Inputs Behaves as if there is an inhibitory connection 2 , 1 1 , 1 P&R yet there is no direct connection between x2 & y1 Inputs simultaneously supporting both outputs A=1 and B = ½ ( (1, ¼) (¾, ¼) (⅓, 1) (0, ⅔) =1, =½)

48. 1 1 å å = = 1 1 wxy N N x • Forward Connections: How it Works Feedback Inhibition Algorithm Iterative Evaluation Outputs Y1 Y2 I2 I1 Inputs x1 x2 1 thus Wy=

49. How it Works Feedback Inhibition Algorithm Back Outputs Y1 Y2 I2 I1 Inputs x1 x2

50. How it Works Feedback Inhibition Algorithm Forward Outputs Y1 Y2 I2 I1 Inputs x1 x2