Bioinspired Computing Lecture 15

1. Bioinspired ComputingLecture 15 Evolutionary Simulation Models & Bio-inspired robotics Netta Cohen

2. Evolutionary & co-evolutionary algorithms Neural nets (RNNs, DNNs, GasNets) Evolution as training for artificial neural nets Today Previous lectures • Hardware & software models • Evolutionary simulations • Some applications to robotics • … and to biological modelling

3. What is a Model? Model building is a tool to discover, to probe and to test the implications of a particular way of thinking, of a theory, or of a hypothesis about the world. Scientific models are notimitations of reality. If they were, accurate models would be as hard to understand as the systems they are modeling. Instead of fully capturing a real system, a good model will focus on one or a few key features, whose behaviour it is designed to replicate.

4. model How do we model? Models come in many flavours: • Mathematical: E=Mc2 • Philosophical: I think therefore I am • Physical: Galileo’s helicopter • Political: Utopia • Platonic: forms theory observation Scientific process: prediction

5. Robotics Robots are embodied and situated; they interact with the environment; they sometimes interact with each other. Their primary functions rely on sensory and motor behaviours. They are usually designed to perform a very specific task. Physical robot requirements often include: cost effectiveness and robustness. One current trend and effort is to move forward to more complex robots that perform multiple, complex and adaptive tasks. The design involved in such problems is increasingly hard the theory is usually lacking.

6. Male crickets sing rapid repeated bursts of pure tone. Females use these to approach a single singer despite rough terrain/obstacles/other singers. Female take a zig-zag path towards male of choice. Example: Cricket Behaviour • Existing theory incorporates different tasks sequentially & modularly: • Recognition: songs are recognised & extracted from general noise • Selection: songs are compared • Approach: the chosen singer is approached

7. How do crickets do it?Robots as models of bio-computation • “A mobile robot architecture must include sensing, planning and locomotionwhich are tied together by a model or map of the world …” • (Kreigman et al, 1987) • But evidence suggests this may not be how crickets work… • Very few neurons appear to be involved – how can 10 neurons build a “model or map of the world”? • Calculating a singer’s location appears dauntingly difficult.

8. Webb’s robot mimics the physiology of the real cricket: Its ears are on its elbows and are linked by a tracheal tube Sounds arrive twice at each ear, causing a phase difference For sounds of a particular wavelength, these phase differences indicate the direction of the source. external sound auditory response latency comparison turn? A single, simple, embodied & situated mechanism performs recognition, choice & approach behaviours simultaneously. Left Left firing threshold? Delay Adapted from Webb (1995) Right Right firing threshold? Adapted from Webb & Hallam (1996) Webb’s Cricket Barbara Webb built a simple mobile robot to explore the true mechanism.

9. Simulation Models A simulation model is “executable”. Simulations unfold over time according to a set of instructions (a protocol/algorithm) that capture selected aspects of the dynamics of a particular model system. By building a simulation model according to our current theories, and observing how it unfolds under different circumstances we can improve our understanding of our theories and develop better ones. Hopefully, we can make specific predictions. With those in hand, we can return to the real world and test them by experimenting on the real-world systems.

10. Evolutionary Simulation Models Evolutionary simulation modelling is a cross-disciplinary tool for complex (often multi-agent & adaptive) systems. • Game theory: evolution of sex, collective behaviour, etc. • Linguistics: learning, evolution. • Geography: urban sprawl, traffic networks, etc. • By their very nature, simulations extend beyond equilibrium or steady state solutions and allow us to characterise the dynamics of the system at hand. • Examples:

11. Evolutionary Simulation Models The strength (and weakness) of evolutionary simulation stems from a functional definition of fitness. Auke Ijspeert used a GA to evolve recurrent neural network models of salamander locomotion. The neural net was interfaced with a simulated salamander anatomy (muscles, joints). The fitness function was behavioural (rewarding forward motion). Simulations of the salamander were assessed for its ability to walk/swim. Two types of simulation: 1) Evolution (of the RNN) 2) Locomotion (of the body controlled by the RNN)

12. The salamander model The strength: We need not have knowledge of the biology to define a fitness function that gives rise to efficient and robust locomotion. A fitness function that rewards fast forward motion might suffice. The weakness: If we wanted to model a real salamander, we are in for a disappointment. The neural network that evolved bares little resemblance to the biological one.

13. What does it do? http://birg2.epfl.ch/movies/SIMS/anim_small_salam.gif The salamander can: • Walk • Swim • Switch between walking & swimming across a border • Switch to swimming if it falls into the water • Follow targets, turn, modulate speed, and more...

14. How simple is it? In fact, designing or evolving such simulation robots based only on high-level descriptions is a daunting challenge. A high level description of the solution hides the crucial role of low-level components and solutions. In fact, the salamander motor behaviour was designed according to very similar principles as biological neural nets for motor behaviour which we neglected to mention until now.

15. Bio-inspired robotics By deriving first principles from biological robots (humans or often simpler animal behaviour) we may be able to overcome many of the hurdles. Applications span a wide range of disciplines, including: Industry (e.g. car production), leisure (e.g. games industry), medicine, and research.

16. Biological motor behaviour Biological motor behaviour has evolved to offer refined and flexible solutions to a variety of challenges. Motor behaviour involves any muscle activity - heart contractions, talking, chewing, digestion, walking, swimming, flying, even scratching, etc.

17. Biological motor behaviour (cont.) Brain control Central Pattern Generating Neural Networks (CPGs): Small, relatively simple neural systems with well-defined units, well-defined circuitry, and well-defined function modulation feedback Central Pattern Generators reflexes control Muscles Such central pattern generators are believed to be responsible for practically all known muscle behaviour.

18. Where do we start? In “simple” motor systems (insects, molluscs, crustacea), central pattern generators have identical architectures in all animals of the same species. They are typically distributed throughout the body and form a distributed coordinated network of activity. They also receive high level instructions from the brain and feedback from the low-level muscles. Ijspeert’s salamander model, while ‘high level’ in its fitness function, was based on a simulation of CPGs and muscles. http://birg2.epfl.ch/oldbirg/SIMS/sal_wsn.htm http://birg2.epfl.ch/movies/SIMS/anim_swim_trot_opt.gif

19. Rhythm generation in CPG circuits Understanding CPG circuits: Models of biological neural circuits generating self-sustained out-of-phase (or anti-phase) oscillations. Figure with permission: A. Ayali

20. A Hexapod Robot As a different example of switching behaviour consider Randall Beer’s hexapod robot controller… The hexapod’s six legs are free to swing laterally and to be raised or lowered. It is stable when the polygon formed by the lowered feet (dashed line) contains its centre of mass (cross). • The hexapod’s task is to fast forward motion. • Six-legged insects achieve this task is a number of ways: • Metachronal Wave: Pairs of legs swing while others stand • Tripod gait: 3 legs achieve stability, the other 3 swing This and all subsequent figures from Beer (1995).

21. angle sensor Single Leg Controller swing backward raise foot forward swing Full Layout The Neural Architecture Each of the six individual legs is controlled by a fully-recurrent network of five continuous-time neurons. Each neuron receives the leg’s current angle as an input. Motion is governed by the output of three “motor” neurons. Each neuron inhibits its counterpart in the adjacent leg controllers.

22. Finding An Effective Controller • The problem: • To discover weights that provide the controller with dynamics that, when coupled with the dynamics of the hexapod body, cause the hexapod to move forward. Details of the GA: During evolution, the controller generally had access to the leg angle input, but sometimes this input was missing. This prevented the controller from relying on the input alone. (Real insects can tolerate the removal of limbs, readjusting their gait appropriately.)

23. A B R1 R2 R3 L1 L2 L3 R1 R2 R3 L1 L2 L3 B: without V V Finding An Effective Controller The successful evolved controllers all generated tripod gaits. A: with sensors Notice B’s slower, more erratic gait. Adapted from Beer (1995)

24. How Does It Use Its Sensors? The same controller improves its performance by exploiting sensory information when it is available: The leg-angle input (continually oscillating between forward and backward values) is used to entrain the controller. As the leg swings forward sensory feedback promotes the stance phase. As the leg swings back, feedback promotes the swing phase – feedback fine tunes the cyclic behaviour. This solution allows the hexapod to respond adaptively to changes in the length of its legs. Although longer legs are slower, the net will automatically slow down to compensate.

25. Additional examples? The same principles can be used to design complex adaptive walking behaviour for more human-like robot activity. Rybak et al. Have used CPG principles together to design and build physical walking robots. Again, the adaptive mechanisms of the robots allow them to save the balance problem without an explicit specification by the designer.

26. Salamander simulations revisited: The salamander simulation was performed in small steps • Small simplified CPG-like networks were assumed for each vertebrate of the spinal cord • Each CPG was evolved to generate anti-phase oscillations • A series of connections were evolved down the spinal cord • Swimming behaviour was evolved • Sensory inputs were added on • Walking behaviour was then superimposed • Additional features (switching, turning, vision, etch.) http://lslwww.epfl.ch/birg/salamander.shtml

27. Some take home messages Evolutionary simulation models are important scientific as well as engineering tools. Bio-inspired concepts and design can help develop robust robots and simulation robots may also help us understand the biological systems. The choice of fitness function, the choice of assessments, and the amount of detail in a model all depend on the specific motivation underlying the research.

28. Suggested reading • Ijspeert AJ (2001) “A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander” Biological Cybernetics, 84, 331-348. • Randy Beer (1995), A dynamical systems perspective on agent-environment interaction, Artificial Intelligence, 72,173-215. • Visually guided walking: http://bach.ece.jhu.edu/~etienne/labweb/projects/index.html and • http://www.rybak-et-al.net/legloc.html and • tutorial: http://www.iguana-robotics.com/presentations/cpgchip/ppframe.htm • http://www.med.unifi.it/didonline/anno-I/informatica/analogcomputers.html • http://www.comp.leeds.ac.uk/johnb/celegans/