AGENT-BASED COMPUTATIONAL ECONOMICS: APPLICATIONS TO ECONOMIC MODELLING, MARKET AND POLICY DESIGN

1. AGENT-BASED COMPUTATIONAL ECONOMICS:APPLICATIONS TO ECONOMIC MODELLING, MARKET AND POLICY DESIGN Lecture 1 Slides Sheri M. Markose Economics Department and Centre For Computational Finance and Economic Agents (CCFEA) University of Essex, UK. scher@essex.ac.uk Talk Prepared for Prime Minister Strategy Unit: 28 Sept. 2005

2. Road Map of Talk • Where has ACE come from ? Essex Centre CCFEA’s pioneering role • What are its foundations ? • What sort of applications has ACE resulted in • ACE models at CCFEA: The Artificial Stock Market model and Herding; Interbank Large Value Payments : Project with Bank of England; Endogenous Risk : Collapse of Currency Peg – Black Wednesday; Cap and Trade -Design of Smart Market for Congestion : Foresight Project • Demos • Calibration and Real Time Analysis : Future Challenges • Concluding remarks

3. AGENT BASED COMPUTATIONAL ECONOMICS (ACE) : A NEW,EXCITING SUB-FIELD OF ECONOMICS • THIS IS BASED ON THE NEW AGE WORLD OF ARTIFICIAL ENVIRONMENTS WITHIN COMPUTERS • ENTIRE SYSTEMS CAN BE RECREATED AND CAN THEN DYNAMICALLY EVOLVE AND GROW : SOMETIMES CALLED ARTIFICIAL LIFE • THESE ENVIRONMENTS MAY HAVE PURELY ARITIFICIAL AGENTS AND/OR CAN HAVE INTERFACES WITH HUMANS • EXAMPLES OF THIS AT A HIGH LEVEL OF SOPHISTICATION ARE COMPUTER GAMES • E-bay and use of ‘BOTs’ in Search Engines for lowest price • THIS NEW TECHNOLOGY IS INCREASINGLY BEING ADOPTED FOR ECONOMIC MODELLING, MARKET AND POLICY DESIGN

4. CCFEA: CENTRE AT ESSEX FIRST IN UK TO DEVELOP POSTGRADUATE CURRICULA IN ACE • THE CORE MODULE IN CCFEA MSc in Computational Agent Based Networks and E-Markets For Which we are gaining recognition is CF902 :Computational Models of Agent Networks, Markets and Self-Organization • This June 13-15 2005 , CCFEA hosted the 10 Anniversary of WEHIA, the pre-eminent Interacting Economic Agents Conference http://www.essex.ac.uk/wehia05/ • Economists in the UK have been somewhat under the cosh with bureaucracy dictating what is ‘good’ science . The RAE has effectively prevented many from taking risks, innovating and going beyond the traditional and mainstream. Hence, it is historic that WEHIA2005 was the first Economic Agents conference that was held in the UK.

5. Thus, this new subfield of Economics uses the artificial environment of agent modelling to understand phenomena that are anomalous in deductive models of traditional Economics. • Increasingly it is felt, the so called complexity sciences which is interdisciplinary in scope is the way forward not only to understand socio-economic systems but also to pragmatically intervene and design institutional change.

6. The ACE Revolution:Foundations Interdisciplinary Edifice of Complexity Studies: Physics,Biology and Evolution,Population,Society and Markets Advances in Maths of Computation: Godel-Turing-Post Advances in Computer Hardware and Software leading to artificial worlds

7. Application of ACE I.Markets As Complex Adaptive Systems (CAS) Absence of Command and Control vs. Self Organization Dynamics from large Numbers of interacting agents Examples:Innovation;Cities and transport Social and Economic Networks,Stock market phenomena II:Market Design Examples Trading Platform, Cap and Trade Systems Pollution Markets Congestion Markets Computational Testbedding and wind tunnel tests Combine with human experiments **Experiments in real time III.Policy Design Avoids Lucas Critique of Econometric models Check out unintended consequences of bad policy design **Experiments In real time

8. ACE in Box I applications overcome the limitations of deductive methodology of traditional Economic Analysis. That is problems that are NP-hard or non-computable can only self- organize as a result of agent interaction. Shyam Sunder is famous for having discovered the irrelevance of intelligence in achieving efficiency at a collective level in some widely used auction markets. The outcomes cannot be brought about by command and control. Studies on how the entire structure of socio-economic networks arise are also best understood using ACE. Eg. Jing Yang (BOE)/CCFEA analyse interbank network structure to understand their implications for the endogenous generation of systemic risk. • ACE in Boxes II and III Covers the fundamental pragmatic aspects of economic agent models in the new field of Computational Mechanism Design • In pre internet economy – markets were a given. In the post internet era the design and implementation of markets/ trading platforms is in the purview of all. John Ledyard who is famous for designing the markets for pollution rights is a pioneer in this. New future of micro-economics

9. Who or what are Agents in ACE? • Agents are computer programs with varying degrees of autonomy and/or computational intelligence • Agents can have fixed simple decision rules or have full powers of adaptive algorithms with capacity of self-referential calculation • In traditional economics agents are assumed to have full rationality and that efficiency of the system as whole is meant to reflect this. This could be a mistaken view • Example is Greek World Cup football team vs. English Football team. Greek case:Each player is not star but team wins championship; English case: Each player is a star but team plays like an idiot !! Ants and bees– each ant is stupid but ant/bee colony is amazing.

10. What are agent-based simulations? • Using a model to replicate alternative realities • Agent-based modelling has three characteristics: • Heterogeneity • Strategies: Fixed or Adaptive • Adaptive learning

11. Agent based vs. Analytical models • Analytical models make simplifying assumptions: The representative agent ; equal size banks with equal size payments • Agent based models can process and run data in real time and can simulate a system in “model vérité” to replicate its structural features and perform “wind tunnel” tests. Eg. Of Interbank large value payments Simulator : ¼ of a country’s GNP goes through the system daily. • Nirvana of Agent based Computational Economics (ACE) • Have agents respond autonomously and strategically to policy changes

12. Example of An Agent Based Economic Model: Canonical Example of Self-Reflexive Systems and Contrarian Structures • First example developed by Santa Fe Institute is the Artificial Stock Market (ASM) • Brian Arthur gave a powerful rebuttal of why traditional economic analysis will fail to understand stock markets and why ACE modelling is needed • In a stock market an investor makes money if he/she can sell when everybody else is buying and buy when everybody else is selling. In other words, one needs to be in the minority or contrarian • Arthur called this the El Farol Bar problem. You want to go to the pub when it is not crowded. Assume everybody else wants to do the same. How can you rationally decide/strategize to succeed in this objective of being in the minority ? • If all of us have the same forecasting model to work out how many people will turn up – say our model says it will be 80% full – then as all of us do not want to be there when it is crowded – none of us will go. • This contradicts the prediction of our model and in fact we should go. If all reasoned this way – once again we will fail etc. So there is no Homogenous Rational Expectations and no rational way in which we can decide to go. Traditional economics cannot deal with this • Hence, Brian Arthur said we must use ACE models and see how the system dynamically self-organizes

13. Simple Stock Market Model WithAgents Relying on Investment ‘Tips’ From Others • http://privatewww.essex.ac.uk/~aalent/herding/herding.htm • Agents have to buy or sell one unit of an asset; they take advise from their neighbours; they act on the basis of the majority view amongst their neighbours; • Neighbours who give bad advise are eventually cut off and new advisers are found • All agents are identical except for how far back they can remember; Some have zero memory and they give random advise; others with memory give the average trend of the market Who will give best advise in a minority winning structure ? Eventually what does the communication network look like? • Hence, paper is called Dynamic Learning, Herding and Guru Effects

14. Features of Herding Simulator • The aim is to model a network that has the properties of a real world network The main feature of real world networks is: - High clustering coefficient (Internet example) • Star formations • The paper contrasts clustering which represents the network topology of the underlying communication network with herding which represents aggregate behaviour with regard to a binary decision problem.

15. Starting from a random graph we study how star formations can take place by dynamically updating the links. This type of study would be very difficult to carry out with traditional economic models. Kirman (1997), Kirman and Vignes (1991) suggest dynamic link formation: reinforced by good experience and broken by bad ones. • Peyton Young, one of the pioneers of network economics has now introduced the notion of radical decoupling . Unlike, traditional games where agents know the rules of the game, here and in most real world situations, one can learn to win only by having the ‘right’ connections or advisors.

16. Properties of NetworksDiagonal Elements Characterize Small World NetworksWatts and Strogatz (1998), Watts (2002)

17. Dynamic Updating of Links • The weights wij to the neighbours who give correct advise are reinforced by a rate of increment Ri+, up to a maximum threshold Γmax • And weights to neighbours who give incorrect advise are reduced by a rate of reduction Rr- • There is a Minimum thresholdΓmin, after which the agent breaks the link to the neighbour, and randomly selects another agent in the network to take advice from.

18. Clustering coefficient • Clustering coefficient:average probability that two neighbours of a given node (agent) are also neighbours of one another. The clustering coefficient Ci for agent i is given by: • The clustering coefficient of the network as a whole is the average of all Ci’s and is given by ; Crand = p

19. Herding coefficient

20. Results:Highly connected agents • We find that agents with zero-memory become highly connected. • Why? Because playing the Minority game in isolation, zero-memory agents perform best, while other agents become trend-followers. • These highly connected nodes can be seen as “gurus”: • Many agents take advice from them

21. Degree distributions Degree distribution of the initial random network Degree distribution of the network after the dynamic updating of links

22. A graphical representation

23. We find that a necessary condition for the agents to find the “gurus” is that: Rr > Ri But too much inertia (Rr >>) cause instability Rates of adjustment

24. Influence of gurus on herding Dynamic Learning in Minority Game : Herding With Clustering C= 0.57 ( p= 0.2; R- =-0.4, R+ =0.2 ;T= 1000) Dynamic Learning in Minority Game : Herding With Clustering C= 0.84 (p= 0.1; R- =-0.4, R+ =0.2 ;T= 1000)

25. 4. Conclusions of Herding Simulator • Agents discover the gurus in the system, by simple adaptive threshold behaviour and random sampling. • The dynamic process of link formation produces the star/hub formations in the network topology often found in real world networks. • When updating the links, the rate of reduction has to be greater than the rate of increment. • We succeed in producing small world network properties of C>Crand and shorter average path length than random graphs.

26. Other ACE Model at CCFEA: Policy Design • Real time interbank payment game • Modelled for the Bank of England to understand how a new set of policy rules may fare • http://privatewww.essex.ac.uk/~aalent/IPSS/

27. Expert Forum: Payment System Architecture and Oversight - 1st Feb 2005 Designing Large Value Payment Systems: An Agent-based approach Amadeo Alentorn CCFEA, University of Essex Sheri Markose Economics/CCFEA, University of Essex Stephen Millard Bank of England Jing Yang Bank of England

28. What are the design issues in a LVPS? Three objectives : • Reduction of settlement risk • Improving efficiency of liquidity usage : ¼ of a country’s GNP goes through the interbank system on a daily basis • Improving settlement speed (operational risk)

29. Design issues Two polar extremes: • Deferred Net Settlement (DNS) • Real Time Gross Settlement (RTGS) + Hybrids

30. Example: DNS vs. RTGS Bank D

31. Logistics of liquidity posting • Intraday liquidity can be obtained in two ways: waiting for incoming payments; or posting liquidity. • Two ways of posting liquidity in RTGS: • Just in Time (JIT): raise liquidity whenever needed paying a fee to a central bank, like in FedWire US • Open Liquidity (OL): obtain liquidity at the beginning of the day by posting collateral, like in CHAPS UK • A good payment system should encourage participants to efficiently recycle the liquidity in the system. • Folk theorem: “A dollar posted earlier in the day improves the liquidity recycling capabilities of RTGS”

32. Risk-efficiency trade off (I) • RTGS avoids the situation where the failure of one bank may cause the failure of others due to the exposures accumulated throughout a day; • However, this reduction of settlement risk comes at a cost of increased intraday liquidity needed to smooth the non-synchronized payment flows.

33. Risk-efficiency trade off (II) • Free Riding Problem: • Nash equilibrium à la Prisoner's Dilemma, where non-cooperation is the dominant strategy • If liquidity is costly, but there are no delay costs, it is optimal at the individual bank level to delay until the end of the day. • Free riding implies that no bank voluntarily post liquidity and one waits for incoming payments. All banks may only make payments with high priority costs. • So hidden queues and gridlock occur, which can compromise the integrity of RTGS settlement capabilities.

34. What can IPSS do?http://privatewww.essex.ac.uk/~aalent/IPSS/IPSS_2_10.exeInterbank structure • Heterogeneous banks in terms of their size of payments and market share -tiering N+1; -impact of participation structure on risks.

35. Herfindahl Index • measures the concentration of payment activity: • In general, the Herfindahl Index will lie between 0.5 and 1/n, where n is the number of banks. • It will equal 1/n when payment activity is equally divided between the n banks.

36. Herfindahl Index Asymmetry And Liquidity Needs Note that total value of payments is the same in all scenarios

37. Bank Failure analysis • IPSS allows to simulate the failure of a bank, and to observe the effects. For example, under JIT: • Note that, because of the asymmetry of the UK banking system, a failure of a bank would have a very different effect, depending on the size of the failed bank.

38. Endogenous Risk : Lucas CritiqueAnd Policy Ineffectiveness • CCFEA Modelling of a classic example of poor policy design leading to collapse of a system: Black Wednesday and Collapse of the ERM Currency peg on 19 Sept. 2002 • George Soros made £2bn taking a short position against the Sterling and the Bank of England. He is alleged to have used the Liar or Contrarian Strategy. Why did Soros win : Or why did all Currency pegs collapse (from Mexico to the Asian ones) • Soros cut above ordinary speculator: student of Karl Popper and knows the self-reflexive problem of the Cretan Liar. Liar can subvert only from a a point of certainty or computable fixed point. Hence, if the policy position is perfectly known – hostile agents can destroy it. Indeterminism or ambiguity is a essential design element for success of market systems and zero sum games • 30 tests using a ABM wind tunnel test of the currency peg with a central bank intervening to raise the exchange and speculator taking a short position shows that the bank cannot win even once viz. ran out of reserves

40. ERM Currency peg requires Central Banks to precommit and support the currency when the Forex rate falls below peg: Central Bank Rule • Figure shows that the state of the fundamentals relating to the long term viability of the parity was neither necessary nor sufficient for speculative attacks. U.K with a 20% overvalued currency sustained attacks as did the other ERM currencies whose parities appear to be virtually unchanged within the pegged regime and when it effectively floated. • The only material difference in the case is with the widening of the bands from  2.5 % to  15% was that it rendered the rule dead letter and when the conditions of a defence were made ambiguous, the speculative attacks ceased dramatically. • Flawed Macro economic literature on precommitment to transparent strategy caused IMF to support currency pegs and led to the worst policy induced failures of our time • What provokes the attacks is the transparent defence : Speculator Sells forward after the central bank raises the exchange rate to above the lower bound : Speculator Rule • After the collapse at least Charles Goodhart said : If at the first whiff of trouble the best response is to float : why peg ?

45. CAP and TRADE: Smart Market for Congestion – IIS Foresight Project • Joint with Transport and Operations Research Group (TORG, Newcastle) and Cranfield Centre for Complexity Studies (Peter Allen) • SMPRT : Smart Market For Passenger Road Transport Based on a uniform sealed bid Dutch Auction Design where the K Highest bid that clears the market for K travel slots applies to all bidders who bid above this. SMPRT algorithm also covers externality costs • Rationale (See over)

46. The traditional view is that economic development with its ever increasing demand for road transport in urban settings and the consumption of non-renewable energy sources with their respective consequences of congestion and pollution are but necessary evils that must be collectively borne. A program of economic development that fully prices and internalizes the externality costs that the private cost- benefit calculus cannot incorporate is seen as essential to prevent the overuse and degradation of resources. The latter is powerfully brought out in Garett Hardin’s classic paper on the Tragedy of the Commons where a decline in social welfare and total output occurs as there is no institution to signal and correct for the negative impact of private behaviour on society as a whole.

47. It is increasingly being understood, with the earliest and successful implementation of a program for pollution control with the US EPA (Environmental Protection Act), that the way to internalize and account for the negative externalities of some economic activities is to use market solutions rather than command and control type regulation. The ‘cap’ and trade solution to the problem of externalities is what underpins the SMPRT. Further, only from bids submitted by motorists can we obtain information on private value

48. II. How to ‘cap’ road use constitutes the first part of the project The cap is based on the actual physical network characteristics of an area of road network which is identified as a congestion hot spot and is determined on the two respective sets of factors relating to the deterioration in traffic efficiency and the growth of environmental degradation from chemical and noise pollutants. These are gauged by a state of the art traffic micro simulator : embellished with all real time features of the cityscape: traffic lights, round abouts etc. The cap determines the maximum number of road users permitted to use the road during a time slice in terms of the above criteria. The cap is given as K passenger car units (PCUs).

50. The GCV simulations done by TORG highlights The sensitivity of traffic efficiency, measured in total travel times and total distance covered, to congestion is clearly shown.  Likewise, the rate of pollution given in terms of the different is sensitive to congestion.  The deterioration of traffic conditions and the pollution emission shows marked non-linearities at critical points. The assumption in traditional analysis that there is constant cost of environmental pollution in terms of linear distance travelled by PCUs is wrong.  The determination of the ‘cap’ is clearly self-evident from the TORG simulations of the GCV model. This is shown in Figure 1 CAP  23,000 PCUs Demand  26, 000 PCUs