Agent-Based Models: Understanding Complex Systems through Individual Level Modelling

Agent-Based Models Dr Andy Evans With additions from Dr Nick Malleson

Agent-Based Modelling CAs ABM Structural issues with modelling Artefacts Timing Data Geography Model Construction Frameworks Intelligent agents

Individual level modelling Aggregate representation was good when it all had to be done in our heads. Systems of small scale elements that combine to make large scale systems in unclear ways are called complex. We struggle to understand emergence– how simple rules of individuals end up in complicated system behaviour. Now we have power for individual representation, i.e. as we understand the world.

Verisimilitude Abstract models: Simplified objects Subset of processes Thought experiments Verifythat core processes are appropriate Alternative is to try and model everything accurately. Is this possible in systems that don’t have closure?

Modelling Identify interesting patterns Build a model of elements you think interact and the processes / decide on variables Verify model Optimise/Calibrate the model Validate the model/Visualisation Sensitivity testing Model exploration and prediction Prediction validation We’ll come back to the modelling process, but for now let’s look at how we build individual level models.

Cellular Automata (CA) Maybe simplest to look first at CAs as a foundation. Can be used for simulation but much much more. Essentially a grid of cells which may be different states dependant on the conditions around them. Each cell has a rule base determining how it reacts.

One dimensional CA Simplest CA: a line of cells. Each cell has a number of states it can be in, depending on the state of its neighbours and itself at the last time step. Usually drawn as a line of cells, with each line below each other being another time step. Simple rules can generate complex emergent behaviour.

Two + dimensional CA Grid of cells that interact. Complex behaviour emerges that includes movement like a multi-cellular organism. John von Neumann (1966) came up with the idea. John Conway (late 60’s) “Game of Life” simplified it and gave it purpose.

The Game of Life Game of Life If dead, and alive neighbours = 3 → Alive If alive, and alive neighbours < 2 → Die of loneliness If alive and alive neighbours > 3 → Die of overcrowding Otherwise, stays as it is.

Classic two dimensional CA uses Modelling forest fires. Modelling animal and plant systems. Finding routes through mazes / landscapes etc.

Other uses SLEUTH Model of Keith Clarke (Santa Barbara). Uses CA to investigate urban growth. Uses current urban areas, slope, transport, landuse, exclusions. Grows cities. Used to study “what if?” scenarios. Essentially a whole new way of looking at urban form. Gigalopolis: http://www.ncgia.ucsb.edu/projects/gig/

Implementing CA 2D array. Run through array applying rules. Need to decide how to handle boundary issues. We need to avoidartefacts.

Artefacts Say we had a time step in which cells became x when a neighbour up and left was x. If we calculate from the top left corner and run across each row, we get this: We see that in one time step, the third cell shouldn’t be x. We’d get a different result if we ran from the lower right. x x x x x x

Solution 2D array with data Empty temp array While model running: loop through array: calculate result for cell put result into temp array end loop copy temp array -> 2D array end while

The lessons of CA Perfect fidelity (representation) isn’t necessary to model the world. Simple rules can lead to realistic and complex behaviour. These can be quite robust – missing information can turn out to be irrelevant to generating patterns, and they can often survive changes. However, representing the world as fixed and movement by state-changes seems wrong. Why not move the objects?

Intelligent Agents In part evolved from CAs. Essentially free moving cells or programs that intelligently interact with each other and their environment.

The Schelling Model Thomas Schelling (Harvard) looked at racial segregation. Found that even if people were willing to live in areas made up of almost 50% other skin colours, segregation would develop. 1D CA. Cells either + or 0 (“black” or “white”). If neighbourhood > 50% of another colour, move to nearest area this isn’t true.

Coding an Agent Agent Class Has an update method called each iteration, eg. move(), trade(). Has a position. Has a list of all other agents and can get their position. Can communicate with other agents if necessary. Environment Class Has environmental conditions. Calls the agents to update. Agents might, for example, trade with their nearest neighbours.

Examples Petrol Price Modelling: can we determine the rules used by petrol stations to set their prices by guessing them from the patterns? Landscape fragmentation and aphid infection. Itzhak Benenson’s model of Tel-Aviv migration under ethnic influxes. Kai Nagel’s team at ETH in Switzerland. Model ~1million people in Zurich.

Artefacts With CAs this seems simple. However, with large agent models we don’t want to be copying the whole system each time. So, we might just wait until the end of a step and update everything. This is certainly one solution.

Synchronous scheduling In general models will have a global time sequence of time-steps (sometimes called ticks) maintained by a clock. The agents are told to enact their behaviour at given time-steps by a scheduler. When there is one global time and all agents act at this speed, none getting ahead in the sequence, this is known as synchronisation. All agents broadly see it as the same time. Synchronisation doesn’t, however, need every agent to do something each time-step: this would unnecessarily increase processing requirements. Different agents may run on different timescales (every five ticks, for example).

Event-based scheduling Different agents may run based on event triggers. In general, these still work within a synchronised time-series. Either way, as with our array problem, we need to make sure updates of contacted agents are either complete or stored for later depending on our model. However, this is often a hard choice. Does a housebuyer always arrive too late to put in an offer? Sometimes they must arrive in time. In general, therefore, we randomise the order of agent steps.

Monte Carlo sampling It is also the case that if we have input data repeatedly entering the model, this can lead to harmonic artefacts. Data running through the system can lead to internal dynamics that cycle at the same periodicity producing artefacts; this is particularly problematic with self-adjusting systems. We need some randomisation. In general we sample input data randomly, even if we act to include certain periodicities (e.g. seasonal). We sample weighted by distribution – so-called Monte Carlo sampling.

Stochastic models Deterministic models always end up with the same answer for the same inputs. Stochastic models include some randomisation, either of data or behaviour. As we have seen, these are usually quantified by repeated runs. However, sometimes we want to run under the same “random” conditions (e.g. to understand some behaviour). Standard computers without external inputs never generate truly random numbers – they are random-like sequences, adjusted by some random seed. If we record the random seed, we can re-run the “random” model exactly.

Space Boundary types: Infinite. Bound. Torus. Other topologies. Organisation: Continuous. Grid (square; triangular; hexagonal; other) Irregular areas, or quad-tree. Network Neighbourhoods: Moore Von Neumann Diagonal Euclidian Network

Issues Ideally, then, we need: A clock/ scheduler, if we’re not calling each agent every time-step (and/or some kind of event-watching system). Some way of randomising sampling of agents/agent locations. Some way of running Monte Carlo sampling of both inputs and parameters. A variety of projections/space types/boundaries.

Helpful Easy I/O. Saving model sequences as video. Connectivity to R, Excel, GISs etc. Options for distributing and describing models. Easy GUI production and visualisation of data. Given all this, the thought that someone might build an agent framework that does all this for you sounds increasingly good of them.

ABM Frameworks What are they? Pieces of software to help people build ABMs Wide range of tools Pre-written functions Entire graphical environment Somewhere in the middle

Why use them? For non-programmers: Graphical “point-and-click” model development. Easier than having to learn a programming language. For programmers: No need to write ‘external’ functionality (e.g. drawing graphs, scheduling events, creating displays). Can concentrate on model logic. Save time (?)

Other Advantages Create an webpage (Netlogo). Talk to other programs, e.g. R, Weka, MATLAB. Output information (to databases, flat files, reports etc). many more…

Basic tools But many use simpler toolkits, like Netlogo: http://ccl.northwestern.edu/netlogo/ A cute ABM environment is Scratch: http://scratch.mit.edu/ Also, Microsoft’s game builder, Kodu: http://www.kodugamelab.com/

Commonly Used Platforms Repast: http://repast.sourceforge.net/ MASON: http://cs.gmu.edu/~eclab/projects/mason/ Ascape: http://ascape.sourceforge.net/ ABLE: http://www.research.ibm.com/able/ Modelling4All: http://www.modelling4all.org/ SecondLife: http://secondlife.com/ Agent Analyst: http://www.spatial.redlands.edu/agentanalyst/

Introduction to Netlogo Based on “StarLogo” Popular teaching tool Aimed at school children, very easy to learn and use Implemented in Java Cross-platform (models will run on any computer which runs Java). Uses its own scripting language –Java is hidden Pretty good documentation http://ccl.northwestern.edu/netlogo/docs/

What’s Netlogo good for? Rapid model design. Abstract thought experiments. Verification testing on abstract landscapes.

Recursive Porous Agent Simulation Toolkit (RePast) Argonne National Laboratory. Based on Swarm. Includes a Logo-based language, ReLogo and graphical programming. Imports NetLogo. Largely Java programmed. Includes 3D GIS using GeoTools. Two main versions: Simphony (Java etc.) RePast for HPC (C++) (MPI based) Based on Eclipse.

Functionality Flexible scheduling including synchronised and event-based scheduling. Randomisation toolkit. Monte Carlo simulation framework. Different spaces and boundaries, including multiple spaces at once.

Functionality Links with R, Weka, GRASS, Pajek. Libraries for genetic algorithms, neural networks, regression, random number generation, and specialized mathematics: http://repast.sourceforge.net/docs/RepastSimphonyFAQ.pdf Exports model shapefiles. Exports applications.

Example Predictive Model: Burglary Simulation GIS/ABM Crime Model Burglar agents situated in an environment that mimics Leeds Agents use roads, railways and public transport to travel around the city burgling real buildings

Why not use a framework? Overheads: sometimes there are better ways of doing the same job if that’s all you have to do. For example, RePast’s watch timing is quite heavy but makes sense as a general framework. Constraints: sometimes it is hard to squeeze a model into a Framework’s way of doing things, let alone then move it to a different framework.

Intelligent agents Usually agent will have some kind of cognitive framework. Within this there will be rulesets to enact depending on a variety of factors.

Belief-Desire-Intention We need some kind of reasoning architecture that allows the agents to decide or be driven to decisions. Most famous is the Belief-Desire-Intention model. Beliefs – facts about the world (which can be rules). Desires – things the agent wants to do / happen. Intentions – actions the agent has chosen, usually from a set of plans. Driven by Events, which cascade a series of changes.

Rulesets Most rules are condition-state-action like: “if hunger is high go to café” Normally there’d be a hunger state variable, given some value, and a series of thresholds. A simple agent would look at the state variable and implement or not-implement the associated rule.

How do we decide actions? Ok to have condition-state-action rules like: “if hunger is high go to café” And “if tiredness is high go to bed” But how do we decide which rule should be enacted if we have both? How do real people choose?

Picking rules One simple decision making process is to randomly choose. Another is to weight the rules and pick the rule with the highest weight. Roulette Wheel pick probabilistically based on the weights using Monte Carlo sampling. How do we pick the weights? We may try and model specific cognitive biases: http://en.wikipedia.org/wiki/List_of_cognitive_biases Anchoring and adjustment: pick an educated or constrained guess at likelihoods or behaviour and adjust from that based on evidence.

Decision making BDI decisions are usually made by assuming a utility function. This might include: whichever desire is most important wins; whichever plan achieves most desires; whichever plan is most likely to succeed; whichever plan does the above, after testing in multiple situations; whichever a community of agents decide on (eg by voting) Desires are goals, rather than more dynamic drivers.

Learning Could we weight based on experience? In a Bayesian Network the states are linked by probabilities, so: If A then B; if B then C; if C then D Not only this, but this can be updated when an event A happens, propagating the new probabilities by using the new final probability of B to recalculate the probability of C, etc. Or we can evolve a picking mechanism using a genetic algorithm.

Thinking for agents Ultimately we have to trade off complexity of reasoning against speed of processing. On the one hand, behaviour developed by a GA/GP would be dumb, but fast (which is why it is used to control agents in games). On the other, full cognitive architecture systems like Soar, CLARION, and Adaptive Control of Thought—Rational (ACT-R) are still not perfect, and take a great deal of time to set up.

Agent-Based Models: Understanding Complex Systems through Individual Level Modelling