M Systems Presented at CASYS ‘09

1. M SystemsPresented at CASYS ‘09 Harry R. Erwin, PhD Hybrid Intelligent Systems Group Faculty of Applied Sciences University of Sunderland

2. The MiCRAM Project • “Midbrain Computational and Robotic Auditory Model for focussed hearing” • This is a collaborative study of the Inferior Colliculus by the Universities of Sunderland and Newcastle. • Sunderland • Dr. Harry Erwin (PI) • Professor Stefan Wermter (co-PI) • Dr. Mark Elshaw • Dr. Jindong Liu • Newcastle • Dr. Adrian Rees (PI) • Dr. David Perez-Gonzalez • Funded by the UK Engineering and Physical Sciences Research Council.

3. M Systems • The concept of an M-system has its origin in Don Griffin’s ideas about animal consciousness. • An M-system is a system that maintains a model of its environment and uses that model to assess the current value of actions leading to future rewards and penalties. • The model of the environment need only have sufficient detail to support action assessment. • There are a number of possible ways that behaviour can be produced by an M-system.

4. The People Involved • Adrian Rees (Newcastle PI) • Jindong Liu (Sunderland Researcher) • David Perez-Gonzalez (Newcastle Researcher) • Stefan Wermter (Sunderland co-PI)

5. Realistic Modelling of Neural Systems • Robots can be designed to emulate natural intelligence abstractly or in detail. • In the MiCRAM project, we are simulating the detailed processing of the inferior colliculus (IC), a large neural module at the top of the auditory brainstem. • This module seems to play a major role in localizing and classifying sound sources.

6. Categories of Behaviour • Stimulus-response or reflexive • May be learned or innate • Habitual • Action values are cached based on experience. • Modelled well by actor-critic systems • Goal-directed or goal-oriented • The animal plans ahead to rewards and back propagates predicted rewards—as they change—to current action values. Much faster than real-time. • If the reward value changes, behaviour may change. • Seen in bats and rats.

7. Well, How Do Bats Capture Targets? • Figure from Webster and Brazier, Experimental Studies on Target Detection, Evaluation and Interception by Echo-locating Bats, 1965. • A bat (Myotis lucifugus) capturing a moth in foliage. • 100 millisecond intervals. • The bat had first detected the tree about 500 milliseconds before the first image. • Data available to the bat—a few biosonar snapshots in the dark.

8. A Simple Task Performed by Bats • Handle non-stationary target acceleration, velocity, and position accurately enough to be able to approach a moving target within 5-10 centimetres. • Address asynchronous echo return timing (with inter-cry intervals ranging over 2-3 orders of magnitude). • Predict forward over a variable time interval ranging up to a second. • Observed to abandon target capture as late as 30 msec prior to contact when the target is inedible. • Strong evidence for goal-oriented behaviour in a small lissencephalic mammal (ca. 10 gr).

9. Modeling Results • The performance seen in bats cannot be matched by optimized predictor-corrector algorithms. • Algorithms that approach bat performance use target location collected over time to fit target motion models.

10. Where is this Time-Space Representation in the Brain? • New result: Lubenov EV, Siapas AG (2009) Hippocampal theta oscillations are travelling waves. Nature 459 (7246):534-539, 28 May 2009. • “Our results demonstrate that theta oscillations pattern hippocampal activity not only in time, but also across anatomical space. The presence of travelling waves indicates that the instantaneous output of the hippocampus is topographically organized and represents a segment, rather than a point, of physical space.” • The hippocampus is already contains place cells. • So at least one area of the brain contains a time-space representation of the animal’s environment.

11. What is the Mechanism? • Spike Frequency Adaption (SFA) is believed to underlie theta wave generation (R. D Traub, et al., 1991; R. D Traub, et al., 1994; X-J Wang, 2002). • The CA3 neurones of the hippocampus seem to play an important role in this. These contain a number of specialised Ca channel types in their dendrites and soma that interact with Ca-activated K channels to produce SFA. • SFA then produces neurone activation at specific phases of the theta wave.

12. Where else are these channels found? • Inter alia… • Thalamic relay cells • Inferior colliculus (IC) rebound cells

13. And What is the Inferior Colliculus (IC)? • Largest auditory structure of the brainstem on the roof of the midbrain. A tectal structure behind the superior colliculus (SC). • Primary point of convergence in the auditory brainstem. Sounds arrive here 2-5 msec after the inner hair cells are activated. • Bidirectional connectivity with the auditory cortex (AC). This is fast enough to support cortically-controlled analysis of current sound afference.

14. Spiking Patterns seen in the IC • Sustained-regular cells—seem to detect continuous sounds. • Onset cells—seem to detect the beginning of sounds. Appear important in eliminating echoes. • Pause-build cells—seem to play a role in delay sensitivity. • Rebound cells—seem to play a role in echo-delay measurement, possibly in match-mismatch processing, and now possibly in generating a time-space representation of the auditory environment. (Classification by Sivaramakrishnan S, Oliver DL (2001) Distinct K Currents Result in Physiologically Distinct Cell Types in the Inferior Colliculus of Rat. Journal of Neuroscience 21:2861-2877.)

15. Sustained-Regular Pattern S&O MiCRAM modelling

16. Onset S&O MiCRAM modelling

17. Pause-Build S&O MiCRAM modelling

18. Rebound S&O MiCRAM modelling

19. Speculation • The Inferior Colliculus contains a time-space map. This allows the cortex to do pattern-matching over: • Time • Space • Frequency

20. Implications • Wherever the hippocampus does, the IC seems to do as well. • In particular, it supports the monitoring of moving sound sources. • This may be the reason the bat can react effectively to object target motion during the last few tens of msec prior to contact.

21. Open Questions • Given how time-space is represented in the hippocampus (and potentially the inferior colliculus), how are plans represented? • How are plans controlled over time? • How are plans generated?

22. Acknowledgements • Cynthia F. Moss, Ph.D., Department of Psychology, Program in Neuroscience and Cognitive Science, University of Maryland. • John Murray, Stefan Wermter, Jindong Liu and the other members of the Hybrid Intelligent Systems Group, School of Computing and Technology, University of Sunderland. • Adrian Rees and David Perez-Gonzalez, Institute of Neuroscience, The Medical School, Newcastle University • The MiCRAM project is a collaboration between the Universities of Sunderland and Newcastle, supported by the EPSRC (ref EP/D055466/1)