Biological motor behaviour (cont.)

1. Biological motor behaviour (cont.) Brain control Central Pattern Generating Neural Networks (CPGs): Small, relatively simple neural systems with well-defined units, well-defined circuitry, and well-defined function modulation feedback Central Pattern Generators reflexes control Muscles Such central pattern generators are believed to be responsible for practically all known muscle behaviour.

2. simple models for pattern generating networks Wilson‘s model of the locust flight CPG (1961) Delcomyn (1998)

3. Spinal Circuits for Generating Rhythm

4. simple models for pattern generating networks Postinhibitory rebound: tendency of a neuron to fire a brief series of APs upon being released from inhibition, without any excitatory input. Delcomyn (1998)

5. Rhythm generation in CPG circuits Understanding CPG circuits: Models of biological neural circuits generating self-sustained out-of-phase (or anti-phase) oscillations. Figure with permission: A. Ayali

6. The output pattern of the STG can be modulated The big question: How do neuromodulators and neuromodulatory neurons reconfigure circuits so that the same group of neurons can produce a variety of behaviorally relevant outputs?

7. Alteration of synaptic strength by neuromodulators There are probably only few synapses which are not subject of some kind of modulation. presynaptic facilitation presynaptic inhibition diffuse presynaptic modulation diffuse postsynaptic modulation Marder, Thirumalai (2002)

8. Development of circuits that generate simple rhythmicbehaviors in vertebratesMartyn Goulding1 and Samuel L Pfaff2

9. Mathematical Modeling

10. Mathematical Analysis and Simulations of the Neural Circuit for Locomotion in LampreysLi Zhaoping,1 Alex Lewis,1 and Silvia Scarpetta21University College, London, United Kingdom2INFM & Department of Physics, University of Salerno, Italy

11. Experimental data To motor neurons E C C E L L Two segments in CPG E C C E L L Spontaneous oscillations in decapitated sections with a minimum of 2-4 segments, from anywhere along the body. Three types of neurons: E (excitatory), C (inhibitory), and L (inhibitory). Connections as in diagram E ,C neurons: shorter range connections (a few segments), L: longer range Head-to-tail (rostral-to-caudal) descending connections stronger E andL oscillate in phase, C phase leads.

12. ( ) ( ) ( ( ) ) JWQ 0 -K g(EL) EL LL CL EL LL CL 0 -A g(LL) -H -B g(CR) Connection strengths Firing rates Neurons modeled as leaky integrators Contra-lateral connections from C neurons - + external inputs d/dt = + Membrane potentials Decay (leaky) term E E C C L L Left-right symmetry in connections E E C C L L

13. Neurons modeled as leaky integrators ( ) ( ) ( ( ) ) JWQ 0 -K g(EL) EL LL CL EL LL CL 0 -A g(LL) -H -B g(CR) Connection strengths E1L E2L E,L,C are vectors: EL= E3L J,K,etc are matrices: : . J11 J12 J13 . . J= J21 J22 . . . J31 - + external inputs d/dt = + Membrane potentials E E C C L L E E C C L L

14. ( ( ( ( ( ( ( ( ( ) ) ) ) ) ) ) ) ) ( ( ( ) ) ) ) ( g(EL) E+ L+ C+ E- L- C- E+ L+ C+ EL LL CL E ± L ± C± ER LR CR EL LL CL EL LL CL E- L- C- 0 0 0 -K +K -K JWQ JWQ JWQ 0 0 0 +A -A -A + external inputs g(LL) + + + d/dt = - -H -H -H -B +B -B g(CR) Linear approximation leads to decoupling ± = “+” mode “-” mode ) ) ( ( E- E+ + external inputs d/dt d/dt = - = - + + + + + + L- L+ E E C C C+ C- L L Swimming mode C- becomes excitatory. The connections scaled by the gain g’(.) in g(.), controlled by external inputs. Left and right sides are coupled Left Right E C C E L L Swimming mode always dominant

15. The swimming mode ( ) K 0 JWQ ( ( ( ) ) ) E- L- C- E- L- C- E- C- A 0 -H B E=L, J=W, K=A, simplification ( ) ( ) J Q -1 K E- = d/dt ( ) -H B -1 E- C- = - L- d/dt + Prediction 1: H>Q needed for oscillations! C- Oscillator equation: d2/dt2 E + (2-J-B) d/dt E + [(1-J)(1-B) +K(H-Q)] E =0 inhibit C- (E, L)- excite Damping Restoration force Experimental data show E &L synchronize, C phase leads

16. Segmt. i Fji Fij Segmt. j Coupling: Fij= (Jij + Bij) d/dt Ej +[B+J] ijEj -[BJ+K(H-Q)]ijEj Oscillator equation: d2/dt2 E + (2-J-B) d/dt E + [(1-J)(1-B) +K(H-Q)] E =0 Single segment: Jii + Bii < 2 Self excitation does not overcome damping An isolated segment does not oscillate (unlike previous models) Inter-segment interaction: When driving forces feed “energy” from one oscillator to another, global spontaneous oscillation emerges. d2/dt2 Ei + a d/dt Ei + wo2Ei = Σj Fij Driving force from other segments. ith damped oscillator segment of frequency wo

17. Controlling swimming directions Segmt. i Fji Fij Forward swimming (head phase leads tail) B+J > BJ+K(H-Q) Segmt. j Coupling: Fij= (Jij + Bij) d/dt Ej +[B+J] ijEj -[BJ+K(H-Q)]ijEj Backward swimming (head phase lags tail) B+J < BJ+K(H-Q) Feeds energy when Ei & Ej in phase Feeds energy when Ei lags Ej Feeds energy when Ei leads Ej Given Fji > Fij, (descending connections dominate) Prediction 2: swimming direction could be controlled by scaling connections H, (or Q ,K, B, J), e.g., through external inputs

18. ( ) E- C- ( ) ( ) J Q -1 K E- = d/dt -H B -1 C- Backward Swimming So, increasingH (e.g, via input toLneurons) More rigorously: Its dominant eigenvector E(x) ~ elt+ikx ~ e-i(wt-kx)determines the global phase gradient (wave number) k For small k, Re(λ) ≈ const + k · function of (4K(H-Q) –(B-J)2) +ve k forward swimming -ve k backward swimming Eg. Rostral-to-caudal B tends to increase the head-to-tail phase lag (k>0); while Rostral-to-caudal H tends to reduce or reverse it (k<0).

19. Simulation results: Forward swimming Backward swimming Increase H,Q

20. Forward swim Turn Simulation Left EL,R Right Time Turning Amplitude of oscillations is increased on one side of the body. Achieved by increasing the tonic input to one side only (see also Kozlov et al., Biol. Cybern. 2002)

21. Further work: Control of swimming speed (oscillation frequency) over a larger range Include synaptic temporal complexities in model Summary Analytical study of a CPG model of suitable complexity gives new insightsinto How coupling can enable global oscillation from damped oscillators How each connection type affects phase relationships How and which connections enable swimming direction control.

22. Renshaw Cell

23. Renshaw Cells : Spinal Inhibitory Interneurons Motor neurons send collaterals to small interneurons Excitability of motor neurons was reduced following electrical stimulation of its peripheral axon

24. Renshaw Cells : Spinal Inhibitory Interneurons Renshaw cells produce ‘Recurrent Inhibition’. Increase motor neuron firing causes increased inhibition via Renshaw Cell (also to inhn of synergist muscles) Acts to prevent large transient changes in motor neuron firing

25. Renshaw Cells : Spinal Inhibitory Interneurons Renshaw cells ‘Dysinhibit’ antagonistic muscles Recurrent inhibition of muscle & dysinhibition of antagonist acts to prevent large sudden movements Descending Cortical Pathways modulate Renshaw Cells

26. Recurrent inhibition: Renshaw cells control the degree of efferent signal sent to the muscle Reflexes Presynaptic inhibtion Interneurons mediate the afferent signal before it gets to motor neuron, influencing the signal sent back to the muscle

27. Evolutionary Training of a Biologically RealisticSpino-neuromuscular SystemStanley Gotshall

28. Results With Renshaw Interconnections Behavior with network with Renshaw interconnections Fitness = -10.08

29. Results No Renshaw Cells Behavior with no Renshaw cells Fitness = -114.51

30. ResultsNo Renshaw Interconnections Behavior with no Renshaw interconnections Fitness = -16.09

31. Results Trainability Average Best Fitness v. GA Iterations 75,000 GA Iterations

32. Observations • GA is successful in training controlled behavior. • Removing Renshaw cell connections makes training difficult. • GA takes advantage of more complex model.

33. Second PaperUsing Genetic Algorithms to Train Biologically Realistic Spino-neuromuscular Behaviors • Introduction • Experiments • Results • Conclusions

34. Introduction • Uses more complex model • Train model with six muscle units and proxy afferent neurons. • Allow GA to evolve the strengths of each muscle unit. • Hypotheses • The GA will find better solutions when training muscle unit strengths (force multipliers) • Adding a proxy afferent neuron will result in more realistic alpha-MN firing patterns.

35. Subnet Input Synfire Chain Alpha-MN New Network Model Renshaw Cell Gamma-MN Proxy Afferent Neuron

36. Experiments • Renshaw cells vs. no Renshaw Cells • Training with Muscle Force Multipliers With and Without Renshaw Cells • Training With Proxy Afferent Neurons • Vary Forearm Mass With and Without Afferents

37. Renshaw cells vs. no Renshaw cells • Fixed force force multiplier (0.8) • Upward frequency 1 in 6 • Downward frequency 1 in 140 • Purpose • (confirm that results are consistent with first paper)

38. Results – Renshaw cells vs. no Renshaw cells • Trends from first paper hold. • This is a good thing!

39. Trained Muscle Force Multipliers With and Without Renshaw Cells • 2 Configurations • Same parameters as first experiment • Purpose • In biology, strengths of muscle units vary. So we add this flexibility to the model. • No longer only training synaptic weights. Now we also train the strengths of the muscle units.

40. Results – Trained Force Multipliers With and Without Renshaw Cells • Both cases show improvement when force multipliers are trained.

41. Results – Trained Force Multipliers With and Without Renshaw Cells Renshaw cells become less relevant because GA uses force multipliers to compensate

42. Results – Trained Force Multipliers With and Without Renshaw Cells

43. Trained with Proxy Afferent Neuron • Add proxy afferent neuron and gamma motoneuron • 6 motor units up – 3 motor units down • Upward frequency 1 in 6 (same) • Downward frequency 1 in 12 • Potentially smoother downward motion • The GA can train a given motion with different frequencies. • Purpose (Observe effect of active feedback)

44. Results – Trained with Proxy Afferent Neurons and Force Multipliers

45. Results – Trained with Proxy Afferent Neurons and Force Multipliers

46. Results – Trained with Proxy Afferent Neurons and Force Multipliers

47. Results – Trained with Proxy Afferent Neurons and Force Multipliers 0.2 Second Delay

48. Vary Forearm Mass With and Without Afferents • Train 4 configurations • 0.55 kg / no afferents • 0.55 and 0.65 kg / no afferents • 0.55 kg / with afferents • 0.55 kg and 0.65 kg / with afferents • Purpose • To test networks on parameters they weren’t explicitly trained on

49. Results – Vary Forearm Mass With and Without Afferents • Training with multiple masses results in more robust solutions

50. Conclusions • Training with a GA is a reasonable method to model biologically realistic behaviors in neuromuscular systems. • GA is an effective training tool for the system • GA finds better solutions as the model gets more complex (Renshaw cells, variable muscle strenghts) • Active feedback from the muscle results in more biologically realistic behavior. • Model has promise for testing hypotheses regarding SNMS pathways and components.