1 / 23

Bootstrapping Knowledge about Social Phenomena using Simulation Models

Bootstrapping Knowledge about Social Phenomena using Simulation Models. Bruce Edmonds http:// bruce.edmonds.name Centre for Policy Modelling http:// cfpm.org. The purposes of this talk are…. To examine the role of simulation models in the understanding of social phenomena

tekla
Download Presentation

Bootstrapping Knowledge about Social Phenomena using Simulation Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bootstrapping KnowledgeaboutSocial PhenomenausingSimulation Models Bruce Edmonds http://bruce.edmonds.name Centre for Policy Modelling http://cfpm.org

  2. The purposes of this talk are… • To examine the role of simulation models in the understanding of social phenomena • (i.e. not entertainment, illustration, art etc.) • Remind people how difficult the task is! • To criticise both naïve positivist and relativistic post-modernist positions • To argue against models that fail to meet any set of reasonable criteria • ‘floating models’ • To sketch a way forward for social simulation as a messy but evolving process Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 2 slide-2

  3. Difficulties of social simulation • The complication and complexity of social phenomena (more than biology) • The difficulty of building, checking and understanding simulations • The sheer inadequacy of formal models for representing rich phenomena • The lack of rich and multifaceted data about social phenomena • The many assumptions not supported by evidence in any social simulation... ...e.g. how the parts that are being represented behave (compared to, say, chemistry)! Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 3 slide-3

  4. The Astounding Assumption That we can make useful computational models of social phenomena... ...even though we know that many of the details in our models are wrong! Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 4 slide-4

  5. Despite these difficulties... • There are those that hope to find models that are both simple and useful (the optimistic, Nobel-prize-winning scenario) • By: • Looking for a very clever‘fundamental’ behind broad classes of social phenomena • Simply hoping that the details will not happen to matter: • On average when many agents are interacting • Or over the long-run Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 5 slide-5

  6. Justifications for such optimism Such hopes are (at times) justified by appeals to: • Simplicity – that simpler models are more likely to be truth (or truth-like etc.) • The Law of Large Numbers – that the ‘noise’ will cancel out en masse (i.e. is random) • Abstraction – that abstracting from detail will result in greater generality • Plausibility – that an academic’s intuitions are sufficient to ensure relevance • Data Fitting – that the model outcomes vaguely match that of some data Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 6 slide-6

  7. Rather it looks like that... • There will be no ‘short-cut’ to useful models of social ‘systems’ that reliably refer to observations of these systems • Social phenomena is at least as complex as biological phenomena but with not evidence that there is a single structuring process (like that of the Darwinian-synthesis) • What appears to be promising simple models will turn out to be nothing more than computational analogies (at least until a lot more empirical work and modelling is done) • When it does appear, theory will be mundane and specific and not a ‘grand theory’ Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 7 slide-7

  8. A Picture of a Modelling Relation Simulation run decoding or prediction encoding or measurement Target Process Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 8 slide-8

  9. A Picture of an Attenuated Modelling Relation Simulation run encoding or measurement decoding or prediction Target Process Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 9 slide-9

  10. A Messy ‘Evolution’ of knowledge • Useful knowledge might be ‘evolved’ • But it will be a ‘mess’ of bits • Each useful in specific ‘niches’ • Not determined by the evidence of the niches but sufficient and competitive there • Gradually increasing the ‘fit’ to the ‘niches’ based on previous knowledge • Compatible with a rich but context-specific range of evidence (including personal) • (Co-evolving with new data collection) Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 10 slide-10

  11. But this depends on... • There being a formal replicator or ‘back-bone’ for the evolutionary process to act on (i.e. the formal models) • A ‘Cambrian Explosion’ of specific representations are allowed to adapt in complex and specific ways (different purposes, groups, etc.) (simulation) • There is sufficient ‘selective pressure’ from rich evidence to drive the process so that ‘toy models’ are eliminated/refined Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 11 slide-11

  12. Can this process be progressive (in any sense) or only relativistic? • Neither completely! • Social processes, institutions and purposes do change fairly rapidly so this will continually require new kinds of knowledge and models • But if models can be compared as to which possibilities are more likely (or which are ruled out) to some extent then localised progress is possible • If some restriction or focus of the suggested possibilities can be established even if ‘only’ about specific context, purpose, assumptions • Allowing a bootstrapping process of increasing reliability, starting from very fallible knowledge Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 12 slide-12

  13. Analogy with the development of measurement • From crude, intuitive measures (length of King’s foot) to sophisticated measuring technologies (SI in advanced countries) • More reliability when accompanying theory is developed but considerable progress with no or only ‘mundane’ theory • Initial uncertainty slowly used to produce more accurate measurement methods • Measurement accuracy is boot-strapped over time based on previous measurement Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 13 slide-13

  14. Kinds of modelling that are conducive to such a process • Probing: Modelling that tests, reveals and documents model assumptions and weaknesses • Evidential: Modelling that finds and strengthens relationships with evidence (including stakeholder opinion) • e.g. Explanatory, predictive, instrumental • Pseudo-maths: revealing such general and important model patterns that this will be systematically applicable to other models Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 14 slide-14

  15. Evidence-Relevant Modelling • Includes models with an explanatory purpose or a predictive purpose • But not those that fail to be any! • Not explanatory because they use overly strong assumptions (encoding is weak) • Not predictive because they are not tested on unknown data (decoding is weak) • Not shown to be effective because they are not used by stakeholders • Includes evidence in the form of stakeholder’s opinions and narratives (as in ComMod approach) Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 15 slide-15

  16. The problem is that there are models that do none of these • A plethora of ‘floating’ models that are: • Motivated according to the intuitions of the modeller but use strong assumptions • Fitted known evidence of outcomes • Not general enough to be pseudo-maths • Fail against any of the mentioned sets of criteria • Are more like computational analogies Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 16 slide-16

  17. A Picture of a Computational Analogy Simulation run decoding or prediction encoding or measurement A way of thinking about the Target Phenomena Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 17 slide-17

  18. Personal vs. Scientific Knowledge • Playing with a model canusefully inform the intuitions of the modeller and users • Including playing with floating models • But what is learnt is personal knowledge • Which is not necessarily something transferable to others in another context • (The model can be transferred but there is almost no selection against the evidence) • (The ideas can be useful in helping to understand phenomena but requires embedding in rich expert knowledge base) Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 18 slide-18

  19. Sufficient and appropriate selective pressure on models is very important • Otherwise there is no adaptive evolution (we already have lots of model variety) • To be maximally relevant to observed systems selective pressure has to be from these niches(rather than, say, fun to play with) • Analogies and ideas are important but they are not the evolutionary back-bone • Surviving models will probably not be simple or have wide scope • But will be those that say something about the evidence (however specific and conditional) Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 19 slide-19

  20. Some Possible Directions for Social Simulation Computational analogies that come and go A retreat into purely formal explorations Simple floating models that fail to predict or explain An evolving mess of specific and complex models that relate to observations Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 20 slide-20

  21. Conclusion • The bootstrapping of useful and reliable knowledge of social phenomena is feasible • Using the evolution of computational simulations across a body of researchers • But only if these simulations are adapted to social phenomena via rich evidence • Most importantly that selective pressure (i.e. criticism) is applied to ‘floating models’ • But it will be a ‘mess’ of quite specific and context-dependent knowledge Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 21 slide-21

  22. Postcript – after the philosophy • If we think that there is a role for simulation in the understanding of social processes... • more than can be obtained through natural language discourse and analogies alone... • and we think that the epistemological problems of simulation are intertwined with the social processes of science • Then surely we have to simulate the social processes of science and knowledge development and not only discuss it Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 22 slide-22

  23. Gratuitous Advert – ESSA 2009 Special stream for papers about “Simulating the Social Processes of Science” Bootstrapping Knowledge about Social Phenomena using Simulation Models, Bruce Edmonds, EOPS III, Lisbon 2008, slide - 23 slide-23

More Related