What’s right about social simulation?

1. What’s right about social simulation? Edmund Chattoe-Brown (ecb18@le.ac.uk) Department of Sociology, University of Leicester http://www.simian.ac.uk

2. Thanks This research funded by the Economic and Social Research Council as part of the National Centre for Research Methods (http://www.ncrm.ac.uk). The usual disclaimer applies particularly regarding Nigel Gilbert (co-PI SIMIAN, Sociology, Surrey). http://www.simian.ac.uk 2

3. Some context Research methods are spread unevenly across the social sciences in different combinations. They are a “small N” phenomenon and even smaller in specific social sciences. While pursuing “novelty” in research methods (as in other areas of social science) we seem to be hazy about its potential implications. This is particularly true of major innovations (as opposed to “more of the same” developments like focus groups from qualitative interviewing). This is no disparagement. Most innovation is necessarily MOTS. http://www.simian.ac.uk 3

4. Some novel questions I think these questions have been “off the radar” because N is too small to generalise in each specific social science. What is a research method? On this basis, “how many” are there? Have we found them all, taking the social sciences as a whole? If not, how would we go about looking for new ones? What kind of thing(s) might be missing? What are the ramifications of a “new” research method? How important are they to find? http://www.simian.ac.uk 4

5. Two cases The simpler case is “import” of methods across disciplines. We tend to forget that the “main” methods of sociology (ethnography and statistical models) were both novel (and one an import) in the twenties and thirties. What would happen if qualitative methods were imported into economics tomorrow? Whatever it was, it would be dramatic! The more interesting case is the discovery of a really new research method: What implications would that have for the social sciences? We seem to be in a state of “doublethink” as academics, claiming as producers that most things are novel while actually acting as consumers as if nothing were really new. http://www.simian.ac.uk 5

6. First claim Social simulation/agent based modelling (SS/ABM) is a radical innovation in research methods. (More humility here than might appear.) It focuses on the theory building stage of the research process (as opposed, for example, to the data collection or analysis phases). Thought: Could we define the space of research methods in terms of their distinctive contributions to the research process? Corollary: If a novel method makes a distinctive contribution to the research process, it should be “compatible” with existing methods rather than claiming to replace them. SS/ABM requires both qualitative and quantitative data for validation and calibration. This is part of its “formal” methodology. http://www.simian.ac.uk 6

7. First important implication All research methods (including SS/ABM) shape the questions we ask and the answers we offer to social understanding. However, a novel method has the opportunity (despite its own biases which will have to be revealed later) to cast light on the biases, lacunae and theoretical preconceptions of research until that point. This deep re-evaluation (and the process based nature of SS/ABM) often helps with synthesis of competing views. This is a hugely important opportunity and one would expect it to lead to an active search for novel methods. http://www.simian.ac.uk 7

8. Quick case study Social capital is a hugely important idea which even its advocates worry may be being “blurred into uselessness”. Key “dimensions” of SC contend (networks, resource access, trust, repeated interaction …) to produce disjointed research agendas which are all “plausible”. (This is a very general phenomenon in social science reflecting our inability to deal with large complex systems.) It is possible to show, using really quite a simple simulation, how all these processes are linked aspects of the same phenomenon. This leads to important insights. For example, the “trust” dimension of SC sits awkwardly with the repeated interaction dimension because most research represents RI in game theoretic terms as simultaneous rather than sequential games. http://www.simian.ac.uk 8

9. Second important implication Because of its methodology, SS/ABM can cast light on the “how complex is it” problem that besets existing methods. For a real social domain, methods presume (but can’t actually demonstrate) that they are of appropriate complexity. This leads to a futile debate between quals and quants. Quals says quants is “too simple” but quants says it “fits”. Quals cannot show that the “descriptive complexity” it finds actually bears on model fit. Quants can’t be sure that associations between variables really reflect causal process. SS/ABM can “show” (for example) that a simplified simulation explains “no less” than a more complex one in terms of tracking a variety of real data with simulated data. http://www.simian.ac.uk 9

10. Second claim Because of its relationship to qualitative and quantitative data, SS/ABM has a distinctive methodology. Rather than “fitting” data to a model (which tells us how well we are doing but only on the presumption that the method was applicable in the first place - an idea to be revisited), SS/ABM directly compares real and simulated output based on “separately established” micro foundations. The former is the quantitative component and the latter the qualitative but both are essential and allow, at least in principle, for falsification. http://www.simian.ac.uk 10

11. Third claim SS/ABM does not mix up “technical” assumptions with empirical ones. YHTTMOT but I have never seen a social process specified that can’t be turned into a programme. (Compare non-computable decision processes in economics.) A lot of older simulation methods “bundled” assumptions to make the method “go” with claims about behaviour (i. e “pools” of identical actors in system dynamics). This problem is very real in methods we take for granted i. e. various kinds of normality and independence assumptions in statistics. These are often invisible to non-specialists. http://www.simian.ac.uk 11

12. Fourth claim With the appropriate infrastructure, SS/ABM is now no harder to learn than basic statistics. As another example of “novelty doublethink” we don’t compare like with like. The comparison is not between learning simulation and statistics “now” but learning simulation a few years back and statistics in 1920 when there was no SPSS and all the sums had to be done by hand. http://www.simian.ac.uk 12

13. Fifth claim SS/ABM may form a genuine basis for interdisciplinarity. Discussion can focus on process and does not necessarily have to be couched in terms of the theoretical categories of a particular discipline (i. e. rational choice, “class”, attitudes). By direct observation, simulators seem to have a lot less trouble talking to each other across disciplines than some people within disciplines have talking to each other! Again, with some amendments (that should not be too unpalatable) different disciplines can continue to focus on what they are “good at”. This is not an approach that intends to turn simulation into the “queen of social sciences”. http://www.simian.ac.uk 13

14. Sixth claim A novel approach always looks “messier” than an established one but that doesn’t mean it won’t get sorted out. Applied statistics looked pretty clunky in 1920 too but we tend to forget that now. YHTTMOT but simulation knows what its problems are and they are being addressed. I haven’t yet spotted anything that scares me enough to want to go and do something else before the “bubble bursts”. There is some element of the dialogue between methods that doesn’t deal with technicalities but with implicit beliefs, the establishment of fair bases of comparison and so on. Discussions about how we establish the novelty of a method are not “mere” propaganda. http://www.simian.ac.uk 14

15. Tentative conclusions Some of the most distinctive opportunities of SS/ABM arise not from details of the method but from its novelty. A novel method also has “meta implications” for thinking creatively about the whole scope of research methods. It is important for people to recognise just how novel it is (avoiding various kinds of “novelty doublethink”) and not assume that existing intuitions will serve them in understanding it. In particular, it is not now (though it may have been in the past) just a “flavour” of statistics. http://www.simian.ac.uk 15

16. Further resources Simulation Innovation, A Node (Part of NCRM: research, training and advice): <http://www.simian.ac.uk>. NetLogo (software used here, free, works on Mac/PC/Unix, with a nice library of examples): <http://ccl.northwestern.edu/netlogo/>. Simulation for the Social Scientist, 2nd edition, 2005, Gilbert/Troitzsch. [Don’t get first edition, not in NL!] Agent-Based Models, 2007, Gilbert. Journal of Artificial Societies and Social Simulation (JASSS): <http://jasss.soc.surrey.ac.uk/JASSS.html>. [Free online and peer reviewed.] simsoc (email discussion group for the social simulation community): <https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=SIMSOC>. http://www.simian.ac.uk 16