Deborah Duong Agent Based Learning Systems MORS 82 nd Symposium

1. The Nexus Cognitive Agent Simulation: Using Computational Game Theory in Social Networks subject to Market Processes Deborah Duong Agent Based Learning Systems MORS 82nd Symposium

2. New Economic Analysis Tools are Needed Social Impact Model • Military analysts need tools to foresee the effects of irregular warfare actions on developing economies. • The standard analysis tools of economics are not predictive of the developed countries they are tuned for, much less developing countries. • For example, the European Union considers the neoclassical macro-economic models used by banks to have partial blame for the 2008 economic crisis. • The problem is, neoclassical models assume unbounded knowledge and rational expectations, and that humans communicate solely through a price signal. • Neoclassical models are mathematical and not empirical, making unfounded assumptions in order to be able to get any answer at all. Fortunately we now have better techniques…

3. Behavioral Game Theory is Better… • Neoclassical economics models order, but not how we become ordered or maintain order. • Empirically, people do know things and have rationality, just not to the extent that neoclassical economics must assume to work. • For example, the Rational Expectations Hypothesis (REH), an assumption of neoclassical economics, states that people have errors on their expectations, that average out correctly. • But, how did they have any knowledge of what to expect at all? • Behavioral game theory addresses these questions. • In Nobel prize winning game theorist Thomas Schelling ‘s theory of focal points agents develop corresponding trade plans that approach equilibria through symbols. This was the first empirical economics study. • Herbert Simon posited the idea that a “rule of thumb” is a more plausible explanation for coordination than the super-rationality of REH.

4. Agent Based Model Technology can Implement Behavioral Game Theory • The European Union is looking at agent based models to make better predictions than the standard neoclassical economic models. • Several agent based models used by the military implement behavioral game theory. • Senturion (proprietary model by Sentia) • IESE (Argonne National Labs) • Nexus (Office of the Secretary of Defense/US Army Training and Doctrine Analysis Center) • Infrastructure and Essential Services Economics (IESE) Simulation Framework is designed to add modules of behavioral economics, including Prospect Theory.

5. Senturion • Senturion is a model of political bargaining, that blends rational elements of game theory with the bounded reason of risk aversion. • Opposing the Rational Expectations Hypothesis, Senturion posits consistent bias in the way that agents view the world. However, it does not explain coordination either. • Based on Expected Utility theory, a well accepted theory. Senturion’s literature says that it outperforms Subject Matter Experts in predicting the outcome of political conflict most of the time. • However, it is bottoms up, without learning. Agents already have the “answer” – the hyper rational “iterative deletion of strategies” to the problem finding an optimal political position to take, when the problem is well defined.

6. Computational Game Theory applies to Optimal Strategies that are not already known • Game theory often can not be applied analytically to many complex phenomena. • Computational Game Theory helps in these cases. • Multiple strategies are tried out, and selected from for how much the contribute to the agent’s goals. • In Game trees, this is done with a look ahead and without memory or growth. • In Genetic Algorithms, this is done by changing the agent so that it remembers the strategy, and so it may be built upon. • Example: Iterated Prisoner’s Dilemma tournaments. • Axelrod invented agents that play a game over and over, until it learns the best strategy for playing with other agents. • Riolo introduced tags to the game, arbitrary symbols that communicate intention, serving as Schelling’s focal points. • These first attempts at modeling Behavioral Game Theory used a single genetic algorithm of homogeneous agents.

7. Using Schelling Points to Coordinate Behaviors Schelling’s focal points, in the form of “tags”, have the ability to coordinate agent behaviors in general. The “tag” or sign that agents display and read comes to mean whatever it is that makes an agent increase the utility of another agent when interacted with. Schelling points thus explain social order, the expectations that help agents to learn social interactions that increase their utility. Schelling points are modeled with tags in heterogeneous agents in SISTER and then for more realistic agents in Nexus.

8. How Schelling Points Implemented as Tags Coordinate Expectations For Trade Behaviors SISTER Agents Every agent that wishes to make a trade displays a trade offer and an arbitrary sign. Agents don’t remember each other as individuals but make offers to each other depending on the signs that the other agents display. Once two agents trade and both have increased utility, they tend to repeat the trade. Because they are looking for a sign and not an individual, other agents can get in on it. The sign holds recipes for learned behaviors. A succotash inventor may display a sign and attract trades for the ingredients for succotash as well as customers that want succotash. Other agents learn to make and sell succotash if they display the sign.

9. Nexus is a more realistic version of SISTER Both Nexus and SISTER use genetic algorithms like Axelrod’s Iterative Prisoners Dilemma model. Genetic algorithm models iterate scenarios, changing their strategies to increase their utility until they can not increase it anymore. That is, if they converge, they stop when they reach a Nash Equilibrium. The important difference between them is that Nexus is designed to model real world data. Nexus agents coordinate behaviors with tags, but those behaviors are more complex and demographically in proportion to more realistic scenarios, such as a scenario of corruption in Africa. In these scenarios, agents behave according to real world counterparts, but then react to international interventions based on what increases their utility given their present trade relationships. Since Nexus models market processes, changes such as Civil Military Operations development programs may be tested for its effect on the host nation’s emergent economy.

10. Example: Nexus models a Natural Market Process A small economy of 65 roles, including Trade Network, Kin Network and Bureaucratic Network roles is created in Nexus, with trade behaviors in the same statistical proportions as an African country. Agents participate in trades in the same proportion as in their country. They choose their role partners according to learned knowledge of what to look for, and have corrupt strategies in accord with what their genetic algorithm tell them to have, based on whether their kin are well cared for. A food shortage is exogenously created, cutting food supplies in half. Immediately, Nexus agents increase bribing behaviors to food distributers, a price correction. Corruption can then be seen as price corrections in regulated economies.

11. Summary Neoclassical economic models depend on unrealistic assumptions, are not empirical, and do not predict. The Rational Expectations Hypothesis assumes people have knowledge but doesn’t explain how. Thomas Shelling’s empirical economics showed that coordination of behaviors occurs through symbols. Senturion starts out with the strategies that solve the problems that it models, while Axelrod’s Iterated Prisoner’s Dilemma discovers the strategies that increase agent utility. SISTER models how Nash Equilibria in trade is achieved through Schelling focal points. Nexus achieves Nash Equilibria and price with data-realistic Schelling focal points.

12. Questions and Comments POC: Deborah Duong dduong@agentBasedLearningSystems.com Social Impact Model