University of Edinburgh University of Sheffield University of the West of England

1. Functions of Distributed Plasticity in a Biologically-Inspired Adaptive Control Algorithm: From Electrophysiology to Robotics University of Edinburgh University of Sheffield University of the West of England

2. Background to project AwayDay 2005

3. In some respects animal movements better than robot movement • Could in part be due to characteristics of biological control algorithms • Which region of the brain particularly concerned with skilled movement? AwayDay 2005

4. Cerebellum • located at base of brain (here a human brain) • looks like a small version of overlying cerebral cortex? • cerebellum = ‘little brain’ AwayDay 2005

5. Cerebellar Function • Clinical and experimental observations of cerebellar damage • Does not cause paralysis, but makes many movements inaccurate, slow and uncoordinated • Similar to effects of alcohol: tests for intoxication may resemble clinical test for cerebellar impairment AwayDay 2005

6. Conclusion: cerebellum is particularly associated with those features of movements that distinguish animals from robots • Framework of project: to investigate whether there are features of cerebellar control that are likely to be of interest to robotics AwayDay 2005

7. Framework: is cerebellar control of interest to robotics? • Problem AwayDay 2005

8. Cerebellar Cortex • Adjacent to and connected with the the brainstem • Has its own cortex (= rind) AwayDay 2005

9. Cerebellar Cortex • Small number of cell types in cerebellar cortex • Connected to form a distinctive microcircuit AwayDay 2005

10. Cerebellar Microcircuitry • Classic work published in 1967 • Investigated anatomy and electrophysiology of microcircuit • Same basic circuit repeated many times (hence “neuronal machine”) • Important: half the cells in the entire brain are in the cerebellum AwayDay 2005

11. Idea of Cerebellar ‘Chip’ • Structure of cerebellar cortex is very uniform over its entire surface • Different regions have different inputs and outputs, (microzones) but same basic organisation • Gives rise to idea of cerebellar chip: ~5000, each with its own particular connections. Mossy Fibres AwayDay 2005

12. Choose Your Task • Consequence of this arrangement: all motor tasks using the cerebellum employ the same basic cerebellar algorithm • The investigator can therefore choose the most ‘appropriate’ motor task • In our case, control of the vestibulo-ocular reflex (VOR) AwayDay 2005

13. Vestibulo-Ocular Reflex (VOR) • Vision is degraded if the image moves (‘slips’) too much across the retina • Retinal slip would be produced by movements of the head, such as occur in locomotion • The VOR acts to counter-rotate the eyes to prevent retinal slip, i.e. to maintain stable gaze • Usually not aware when we use it AwayDay 2005

14. Secondary Vestibular Neurons Primary Vestibular Neurons Ocular Motor Neurons Semicircular canals Extraocular Muscles VOR Control: Basic Circuit • Input from vestibular position, senses head movement • Passed to interneurons in vestibular nuclei (secondary vestibular neurons) • Thence to motor neurons that control the eye muscles • This circuit in brainstem (just below cerebellum) AwayDay 2005

15. VOR Control: Cerebellum Flocculus Retinal slip • Cerebellar flocculus receives information about • Head velocity • Eye movement commands • Retinal slip • Projects back to brainstem Eye Muscles Orbital Tissue head eye motoneuron Brainstem velocity firing velocity AwayDay 2005

16. Flocculus and Brainstem Eye Muscles Orbital Tissue head eye motoneuron velocity firing velocity reference command output Controller Plant r(t) u(t) y(t) VOR Control: Generalised Version AwayDay 2005

17. Not Feedback Control reference command • Retinal slip signal is delayed by 100 ms (visual processing) • Feedback control would become unstable at ~ 2.5 Hz, yet VOR operates up to ~25 Hz • Feedback control not suitable output Controller Plant r(t) u(t) y(t) X Sensor AwayDay 2005

18. Inverse Plant Model P-1 Plant P reference command output r(t) u(t) y(t) Control Method: Open-Loop • If feedback not available, then open-loop control must be used • If reference signal is desired output, then the controller becomes an inverse model of the plant (‘plant compensation’) AwayDay 2005

19. Adaptive Control desired output Inverse Plant Model P-1 Plant P command • How can we be sure the inverse plant model is accurate? • Requires constant calibration – ‘adaptive control’ • Use information about system output for learning, rather than on-line control output r(t) u(t) y(t) Sensor training signal AwayDay 2005

20. Eye Muscles Orbital Tissue Brainstem eye motoneuron head firing velocity velocity Flocculus retinal slip VOR Equivalent • Available training signal is retinal slip, known to be sent to the flocculus • Consistent with flocculus being the adaptive part of the controller • Consistent with e.g. lesion evidence that VOR adaptation is lost after floccular inactivation AwayDay 2005

21. Why VOR Calibration? • Well-defined adaptive control problem • Eye movements are relatively simple • single joint instead of up to ~6 joints in finger movements • constant load • Great deal known about underlying circuitry • Well established cerebellar involvement AwayDay 2005

22. Framework: is cerebellar control of interest to robotics? • Problem: adaptive calibration of VOR • Approach: multidisciplinary AwayDay 2005

23. Multidisciplinary Approach • Modelling • (theoretical neuroscience, Sheffield) • Electrophysiology • (experimental neuroscience, Edinburgh) • Robotics • (University of the West of England, Bristol) AwayDay 2005

24. General Modelling Task • Devise a working algorithm that connects the microcircuit to the behavioural competence • Obeys known anatomical and physiological constraints AwayDay 2005

25. Cerebellar Modelling • Cerebellar microcircuit has been extensively modelled, starting with classic work of Marr (1969) and Albus (1971) • Here in more modern form of the adaptive filter AwayDay 2005

26. Specific Modelling Problem • Extensive experimental work shows that in VOR calibration there are TWO sites of plasticity • In cerebellar cortex, as predicted by adaptive filter models • In the brainstem • What are the computational advantages of this distributed plasticity? AwayDay 2005

27. Mayank B Dutia Centre for Integrative Physiology University of Edinburgh Electrophysiology: Problem • What are the learning rules underlying brainstem plasticity? • Existence known for ~20 years, rules yet to be identified • Critical for understanding computation significance AwayDay 2005

28. Rostral Midline Medial Vestibular Nucleus Caudal Rat Brainstem Slice Electrophysiology: Technique • Record from neurons in slices through brainstem • Look for neurons that receive input for the flocculus (flocculus target neurons, FTNs) AwayDay 2005

29. RoboticsDoes Algorithm Work in Real World? • Tony Pipe, Chris Melhuish, UWE Bristol • Camera stabilisation • How does algorithm compare with control engineering alternatives? AwayDay 2005

30. Multidisciplinary Approach • Framework: is cerebellar control of interest to robotics? • Problem: adaptive calibration of VOR • Approach: multidisciplinary Modelling: plausible candidate algorithm Electrophysiology: biological underpin Robotics: real world application AwayDay 2005

31. AwayDay 2005