Bioinspired Computing Lecture 2

1. Bioinspired ComputingLecture 2 Collective Behaviour and Swarm Intelligence

2. Nearly all slides … • By Seth Bullock (2005 lectures)

3. This Time: command & control vs. self-organisation the sophisticated abilities of insect colonies stigmergy algorithms inspired by insect intelligence Foraging Clustering & Sorting Building Swarming & Flocking Overview… • Next Time: • insect-like social robots • sorting & clustering • cooperative transport • Applications • sport • education • entertainment

4. Imagine you are faced with the task of organising a huge number of Amazonian species. There are hundreds of ways in which these species might be divided up: colour, size, etc. Amazonian Categorisation • Your boss tells you that your categorisation will have to match that of his customers over a very long time scale, so it must be flexible, because not only will customers change, but species may also change (colour, size, prevalence, etc.). • One approach to this knowledge management problem is to interview customers, devise a general-purpose, explicit categorisation scheme, pay someone to keep it up to date, and hope customers and species change little and slowly. • amazon.com solve an analogous problem like this: • “Customers who bought this book, also bought…”

5. Collective problem−solving • "Problem−solving can occur at a level above a collection of idealized agents, without "intentional solving" on the part of the individual.” • N.L. Johnson, Collective Problem Solving, LANL tech−report, 1998. • In other words, the individual agents do not know they are solving a problem, but their collective interaction so solves the problem. • Emergent functionality

6. Why is this automatic categorisation approach successful? Amazonian Self-Organisation • no need to interview customers • no need to discover explicit categories before using them • adapts to changing trends automatically as they happen In contrast, “command and control” approaches suffer from the problem that explicit, hand-designed categorisations… • may be hard to discover through customer interrogation • will require constant updating and may still be out of date • may sometimes require a radical overhaul In the next few lectures, we will learn that simple, self-organising systems such as amazon.com’s often enjoy advantages over their command and control cousins…

7. These systems tend to involve many partially independent entities working together to solve a problem without a central executive. Each entity may be unaware of many or all of its colleagues, attending to only its local environment. Advantages • Such systems are: • parallel systems of simple agents • robust to noise & damage • dynamic • flexible • self-organising • adaptive • possibly complex • hard to build? • hard to understand? • fast, cheap, out of control? • intelligent?

8. Some of the most impressive natural self-organising systems are to be found in the world of colonial insects… Ants, Termites, Bees, etc. • Such species… • forage for food, dividing colonial resources effectively • construct complex hives, nests, etc. • efficiently & dynamically divide labour amongst the colony • sort and cluster different objects (eggs, corpses, etc.) • cooperate in moving objects, defeating enemies, etc. that would be impossible for a single individual to deal with …with no central planning and very little communication – complicated, coordinated, goal-directed behaviour often seems to arise spontaneously from the interactions of many simple insects.

9. E.g.

14. Recently software engineers and roboticists have begun to exploit our understanding of social insect behaviour to design new kinds of algorithm and new kinds of robot. Ant Algorithms and Bee Bots • These systems idealise insect behaviour in much the same way that ANNs idealise the behaviour of neurons (topic 3) • Researchers pick and choose aspects of natural systems in the hope that the artificial systems they inspire will share some of their desirable properties. • Of course, ants and bees are not designed to solve the problems of today’s software engineers or roboticists. • A piece of software or a robot will not perform well just because it behaves like an ant colony – the trick is to find aspects of insect behaviour which can be profitably exploited.

15. A particularly striking result from ant experimentation concerns the ability of a colony to discover the shortest routes to the resources it requires: nest food Foraging for the Shortest Route As ants forage they deposit a trail of slowly evaporating pheromone. Those that reach the food first return before the others. • One pheromone trail is now stronger than the other, directing the ants to the food via the shorter route. • It is not just ants that need to find optimal pathways. • Traffic on telecommunications systems, the internet, roads, rail, and sea would all benefit from the reduction in congestion that efficient routing algorithms could provide.

22. Consider an in-car system that suggests the best route to take for any journey from A to B across a road network. Ant-Inspired Routing B An “ant” traverses the network following the strongest pheromone trails from A to B. At the end of the journey the ant lays down pheromone along the path taken, leaving less pheromone at nodes that were congested. A In this way, routes via congested nodes are gradually weakened, prompting ants to take alternative paths. Since many ants traverse the network constantly and their pheromone evaporates gradually, the system automatically adapts to the current load on the road network.

23. In their book Swarm Intelligence, Eric Bonabeau, Marco Dorigo & Guy Théraulaz claim that “work on ant-based routing is only beginning … but in all tested situations it appears that … ant-based routing with agents patrolling the network outperforms all other routing algorithms”. How Good Is It? • France Télécom and BT are developing algorithms for their systems, but the application of ant-based routing is potentially much wider – e.g., routing internet traffic. • Like amazon.com, these algorithms rely on constant user traffic to build an up-to-date picture of what is going on (whether it be trends in book shopping, jams on the Otley Road, or congestion at telecom hubs). The power of these algorithms is their simplicity and their ability to direct traffic and build this picture simultaneously.

24. Many species of ant cluster corpses into cemeteries, gradually piling them up together. Brood sorting is also observed, with larger larvae lying further from the brood centre. In addition, some species are able to construct walls, arches and other architectural structures. Sorting, Clustering & Building • These behaviours are yet to be fully understood, but have all been modelled as the result of simple probabilistic rules: • Clustering relies on two rules concerning the ant’s local environment: • items are more likely to be picked up when they differ from those around them, and • items are more likely to be put down amongst similar items. • Wall building, etc., is slightly more complex, relying on chemical templates to direct what are essentially the same basic processes.

25. Clusterring and sorting • Deneubourg (1998): • Memory • I N items, time T: f = N/T

26. Agent based system Starting form a homogeneous distribution

27. Imagine you’re want to seat many guests. It’s best if you group guests that know each other together. But how? Ants for Catering • First draw a graph that represents which of your guests know each other. Then apply an algorithm inspired by ant clustering: scatter the nodes of the graph and a load of ants on a page; let an ant pick up a node, i, if it is surrounded by nodes to which i is not connected; let an ant dropi if it is surrounded by graph neighbours of i; let the ants wander about at random picking up and dropping nodes. Slowly, clusters of acquainted guests will form on the page. This graph-partitioning technique has applications in chip design (where connected components must be placed close together on a chip) and load balancing on parallel processor machines.

28. Here we see a random graph being partitioned by ants… Partitioning a Graph After the ant algorithm of Kuntz, Layzell & Snyers (1997) has been at work, a few clear clusters have emerged. Cluster members are more connected to each other than to members of other clusters. This technique can be used to efficiently load the processors of a parallel machine – minimising the amount of communication required.

29. How can insects in a colony coordinate their behaviour in order to build highly complex architectures? Ants and termites don’t appear to have blueprints in their heads: they seem to follow simple rules in an almost random manner. Ants for Architecture If the blueprint isn’t in the insects’ heads, it may be in their environment: ants appear to use their own previous work to stimulate their behaviour – the building of arches, towers, etc., appears to be governed by the structures themselves. “The worker does not direct his work, but is guided by it”

30. Stigmergy is a slippery concept. At its root is the ability of agents to influence each other and their future selves by altering their environment – often seemingly unknowingly. Stigmergy • Some examples: • pedestrians crossing a park make paths in the grass – the most popular will guide future walkers and be reinforced • amazon.com customers buy books – their purchases change the descriptions of the books, guiding future customers • cells divide & differentiate during morphogenesis according to chemical gradients that they themselves influence In these examples, the behaviour of individual walkers, shoppers, cells, etc., is shaped by their environment, which in turn was shaped by their own prior activity.

31. Bonabeau and Théraulaz have demonstrated these ideas in action through simple artificial paper wasp architects. ? Artificial Architects Wasps wander at random over a 3-d grid of cells and follow a simple set of microrules that govern building behaviour. Depending on the contents of the 26 cells that surround the wasp, it can deposit one of two types of brick, or leave the cell alone – thus wasps are reactive agents with no memory. B&T show that starting from a single brick, swarms of these wasps following simple sets of microrules can construct complex structures that resemble natural paper wasp nests – layered, cellular combs with internal cavities and a surrounding envelope. Could sets of these microrules be evolved to build human habitation or useful artefacts? - cf work on urban planning by Georg Vrachliotis, ETH Zurich

32. Termite Construction Simulation (D Ladley, Leeds) Building the queen chamber • 3 kinds of termite: • trail-laying • Follow/lay pheromone • Nursing • Move towards/away queen up to fixed distance • Building • To place block: • Pheromone level must lie within a certain bound and one of: • Material above or below • (allows vertical stacks) • Site shares face with horizontal loc containing material and satisfying (1) • (allows stack horiz extension) • 1 face of the site must neighbour 3 horizontal locations that each contain material • (allows gradual construction of elevated horiz surfaces) • Queen: green, ground red, yellow: deposited building material. • 1. formation of "pillar-like" structures: • simulates reality • 2. construction of walls between the pillars www.comp.leeds.ac.uk/danl/cconstruct.html

33. We have already seen how swarms of insects are able to cluster similar objects together. Software engineers have exploited similar ideas to cluster database records, discovering trends in, say, financial information (which customers are likely to default on their loan), or health data (which patients are most likely to develop heart disease). Swarms for Data Mining James MacGill has developed an approach to spatial data mining inspired by insect swarming. Consider a map of the UK with every case of CJD marked. Eyeballing the map will reveal the existence of clusters. But most of these are probably just over the major UK cities, that’s where most of the people live, after all… We need to spot anomalous clusters – where there are more cases than you would expect

34. Imagine a flock of agents flying across the CJD data set… • When it finds an anomalous “hotspot” it gets excited • When it finds no interesting data it gets bored • When it finds no data at all it dies Swarms for Data Mining Each agent is aware of nearby agents and of the data that passes under it, constantly comparing the number of CJD cases in its vicinity to the local population size. Agents that are excited slow down, agents that are bored speed up and agents that are dead don’t move at all. In addition, agents are attracted to nearby excited agents and repelled by any nearby deceased agents. How does such a system behave?

35. The swarm adaptively scans the data set focusing on the interesting parts and ignoring the boring areas. Self-Organising Data Foragers • The swarm quickly isolates parts of the map with no cases of CJD, littering them with dead agents which dissuade further exploration of those areas. • Areas where CJD cases are in line with local population density are boring and are quickly passed over. • Areas where the number of CJD cases in abnormally high or low attract more and more agents of different sizes, scanning the area at different scales and resolutions. In comparison with exhaustively scanning the map at many different spatial scales, the flocking approach is faster, but perhaps slightly less reliable – MacGill suggests a hybrid approach combing both methods.

36. The development of swarm intelligence is only just beginning. Many open questions exist… Issues in Insect Algorithms • how can we design individual agents such that en masse, they are able to achieve a desired swarm-level behaviour? • should they be complex or simple? • should they differ from one another? • should they be reactive or non-reactive? • should they learn? How? • should they communicate? How? • should they utilise stigmergy? How? • should we worry that swarms are not predictable/reliable? As yet we have few answers – what would a theory of swarm intelligent systems look like?

37. Last Time: command & control vs. self-organisation the sophisticated abilities of insect colonies stigmergy algorithms inspired by insect intelligence Foraging Clustering & Sorting Building Swarming & Flocking This Time: insect-like social robots sorting & clustering cooperative transport Applications sport education entertainment Recap…

38. So far, we have considered software algorithms inspired by the various behaviours of social insects. Insect-like Social Robots • Researchers are also implementing insect-like robot control systems for a range of tasks. These include single robots inspired by crickets & cockroaches, and multi-robot systems. • Before reviewing some of the collective behaviours that have been achieved with multiple robot systems, we will consider the question: • Why take a multi-robot approach? single complex robot working alone versus many simple robots working together

39. There are many reasons to consider building multiple simple robots rather than a single complex solution: Multi-Robot Advantages • robustness – a simple machine may be less likely to fail than a complex one • redundancy – many robots carrying out the same task ensures that the system can tolerate the failure of a few • cost-effectiveness – many simple robots may be built for the price of a single complex machine • simplicity – we often may not know how to build a single robot solution • cooperation – the task may simply be impossible to carry out by an individual

40. We have already seen the application of insect-inspired clustering and sorting algorithms in software. Hardware implementations of similar ideas have shown how little intelligence each agent really needs… Sorting and Clustering in Reality At UWE Owen Holland, Chris Melhuish and colleagues, have been developing collective robot systems for some time. In experiments with robots the size of footballs, they have shown that individuals with very crude sensors, no memory or sense of spatial location, and no communication, can… • segregate two kinds of object • sort them into a central cluster of object A surrounded by a ring of object B – annular sorting resembling the brood sorting of ant species Leptothorax unifasciatus.

41. Several mobile robots equipped with special grippers patrol an arena littered with pucks of two kinds – red and blue. arena pucks robot with gripper The Basic Sorting Task The robots can detect the gripper pushing against anything heavier than a puck, a gripped puck’s colour, and whether an obstacle is directly ahead… Three rules achieve simple clustering: • if (gripper pushed & obstacle) then turn at random • else if (gripper pushed) then reverse (dropping puck) and turn at random • else go forward

42. Notice that the robots know nothing about the location of a cluster or even the local density of pucks. How Does It Work? • The robots succeed at the task because… • they pick up loose pucks without noticing • when they have a puck, they tend to deposit it next to another puck (pushing a puck into another triggers rule 2) • when they don’t, they tend to only pick up pucks which are on their own (pucks in groups trigger rule 2) Interestingly, as in some experiments with ants, the robots often clustered the pucks at the edge of the arena. Through artificially varying the probability that rule 1 would fail to fire, the researchers found that they could increase or decrease this edge-seeking tendency.

43. In an attempt to speed up the clustering, the robots’ rule set was altered slightly to treat blue and red pucks differently: Annular Sorting • if (gripper pushed & obstacle) then turn at random • else if (gripper pushed) then • if (blue puck) then pull puck backwards endif • reverse (dropping puck) and turn at random • else go forward The new robots treat the red pucks as normal, but drag blue pucks back a small distance before letting go of them. These robots did not cluster pucks any faster, but succeeded in roughly arranging the blue pucks in a doughnut-shaped ring around a cluster of red pucks – in a similar fashion to the brood sorting Leptothorax species…

44. These robot experiments show that what appears to be organised collective behaviour can be achieved by very simple robots following very simple rules: Collective Stigmergic Behaviour • the robots made use of properties of their physical environment to help them coordinate their behaviour • they exploited the stigmergic effects of each other’s behaviour to achieve their joint task • the global-level effects of changes to the rules of individual robots was very hard to predict Through this kind of simple robot research it may be possible to gain an understanding of… • natural collective behaviour in insect colonies, etc. • the nature of stigmergy itself

45. Social ants often transport large items to their nests in teams, achieving feats that would be beyond a single ant. Natural Collaborative Transport In contrast, getting robots to work together as a team on a joint project, as we have just seen, is a tricky task. Bonabeau, Dorigo and Theraulaz list some of the interesting questions that this insect behaviour suggests: • Do individuals behave differently when they are in a team? • How are enough ants recruited when help is needed? • How do ants co-operate and co-ordinate their behaviour? • How do they handle deadlocks, obstacles, conflicts, etc.? Studying the behaviour of collective behaviour in groups of robots is helping to answer some of these questions...

46. In addition to helping us understand natural insect behaviour, achieving effective robot collaborative transport techniques would represent a significant feat of engineering. Building Robot Collaboration • Some engineering considerations: • How can interference between team-mates be reduced? • What is the optimal team size for a particular task? • How should team-mates accommodate each other? • How complicated must conflict resolution mechanisms be? • How should team-mates communicate with each other? Observation of co-operative transport in ants tends to suggest that an efficient solution could be engineered using a group of robots lacking explicit means of communication, or any centralised control.

47. Co-operative box pushing is considered a bench-mark test for teams of multiple robots. Many researchers have taken an approach involving planning and explicit centralized mechanisms for conflict resolution. In contrast, Kube and Zhang have developed an ant-inspired system… Ant-Like Robot Box Pushing • The box-pushing task is as follows: • A lit box, which is too heavy for a single robot to push, must be moved from some random location in the middle of an arena to the arena’s edge. The sensors on Kube & Zhang’s circular, wheeled robots can detect i) the box, ii) other robots, and iii) nearby obstacles. Each robot is identical and can carry out five different behaviours: GOAL, AVOID, FOLLOW, SLOW, & FIND. www.cs.ualberta.ca/~kube/research.html

48. Kube & Zhang use a subsumptionarchitecture: obstacle? avoid goal? goal slow robot? follow find wheels A Subsumption Architecture Each behaviour is triggered by a certain stimulus and suppresses those below it in the architecture. The robot wanders until it detects either another robot, which it follows, or the goal, which it approaches. Encountering an obstacle at any time will trigger a simple avoidance behaviour – turning away from the obstacle. This controller enables robots to converge on the box. How do they co-operate in pushing it to the wall?

49. The robots described so far will succeed in locating the box and pushing it to the wall, but they are far from efficient. Co-operative Box Pushing Like ants, the robots suffer from periods of stagnation in which the box makes no significant progress. Kube and Zhang’s approach to solving this problem is inspired by ant behaviours: realignment and repositioning. If the box has not moved for a while a robot will realign itself by rotating slightly. Realignment alters the forces acting on the box while ensuring that it is still pushed in the same general direction. If realignment proves fruitless, a robot will reposition itself by backing away and re-approaching the box. This allows other robots to act on the box in its absence and changes its influence on the box.

50. In this movie, six of Kube and Zhang’s robots attempt to push a lit box to the side of the arena. As described, periods of stagnation cause robots to realign, or back off & reposition themselves. Box Pushers in Action…