Staging Abstraction using Chains of Models

1. Staging Abstraction using Chains of Models …and…the Problem of Context-Dependency Bruce EdmondsCentre for Policy ModellingManchester Metropolitan University

2. Talk Outline Introduction: a fundamental difficulty! Problem 1: needing bothrelevance andrigour Possible strategy: using chains of models to stage abstraction Problem 2: context-dependency :Possible strategies: reducing scope and including context Concluding discussion: connecting the worlds of policy and science Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-2

3. The Anti-AnthropocentricAssumption • That the universe is not arranged for our benefit (as academics) • e.g. that assumptions like the following are likely to be wrong: • Our planet is the centre of the universe • Planetary orbits are circles • Risky events follow a normal distribution • Humans act as if they followed a simple utility optimisation algorithm • The one that I am particularly arguing against here is that our brains happen to have evolved so as to be able to understand models adequate to the phenomena we observe Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-3

4. Versions of this assumption Whilst other animals have severe limitations and biases in their cognition, we don’t That our tools (writing, computers etc.) allow us to escape our limitations and biases to achieve general intelligence That simplicity (that which is easier for us to analyse) isany guide to truth If your model is not simple enough to analyse and understand, you are: (1) not clever enough, (2) lazy (have not worked hard enough), (3) premature (don’t yet have the tools to crack it) or (4) mistaken Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-4

5. Living with the AAA Accepting that that much of the world around us is fundamentally beyond capturing in a model that is both adequate and sufficiently simple and general for us to cope with Acknowledging our (brain+tools) biases and limitations and so considering how we might extend our scientific understanding as much as possible Phenomena that are simple enough for us to scientifically understand are the exception – the exception to be sought and struggled for Simplicity is the exception – a science of non-simple systems makes no more sense than a science of non-red things Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-5

6. Problem 1: Needing both rigour and relevance Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-6

7. A Dilemma KISS: Models that are simple enough to understand and check (rigour) are difficult to directly relate to both macro data and micro evidence (lack of relevance) KIDS: Models that capture the critical aspects of social interaction (relevance) will be too complex and slow to understand and thoroughly check (lack of rigour) Butwe need bothrigour and relevance Mature science connects empirical fit and explanation from micro-level (explanatory and phenomenological models) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-7

8. KISS vs. KIDS as a search strategy Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-8

9. The Proposed Approach Not to use a single model but rather a closely related “chain” of models Starting with narrative and statistical evidence for the micro-level behaviour of individuals etc. To build models that are more adequate to the processes that are thought to occur Which are checked and assessed against as many kinds of evidence as possible (including macro statistical evidence) And only later abstract to simpler simulation, analytic and social network models Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-9

10. Social Complexity of Immigration and Diversity • A 5 year EPSRC-funded project between: • University of Manchester • Institute for Social Change • Ed Fieldhouse, Nick Shryane, Nick Crossely, Yaojun Li, Laurence Lessard-Phillips, HuwVasey • Theoretical Physics Group • Alan McKane, Tim Rogers • Manchester Metropolitan University • Centre for Policy Modelling • Bruce Edmonds, Ruth Meyer, Stefano Picassa • Aim is to apply complexity methods to social issues with policy relevance Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-10

11. The Modelling Approach SNA Model Analytic Model Abstract Simulation Model 1 Abstract Simulation Model 2 Data-Integration Simulation Model Micro-Evidence Macro-Data Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-11

12. An example of the layering of related models in chemistry from 1990 Adapted from:Gunsteren, W. F; Berendsen, H. J. C. (1990) Computational Simulation of Molecular Dynamics: Methodology, Applications and Perspectives in Chemistry. Angewandte Chemie - International Edition in English, 29:992-1023. Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-12

13. Roles of each kind of model Each is constrained by those “beneath” them, i.e. are consistent with them What each component should clearly represent something Models “above” analyse, check and explain what is happening in those below Models immediately “below” can be used to explore the safety of assumptions It might well happen that simpler, more abstract models have validity (w.r.t. a lower model) only under some settings Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-13

14. But why not just jump straight to simple models? There are many possible models and you don’t know whyto choose one rather than another, this method provides the underlying reasons Much social behaviour is context-specific, and this approach allows one to check whether a particular simple model holds when background features/assumptions change The chain of reference to the evidence is explicit, allowing one to trace their effect and possibly better criticise/improve the model This approach facilitates the mapping onto qualitative stories/evidence Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-14

15. Data Integration Models Intended more as a computational description of a particular case than a theory (at least a general theory) Its aim is to represent as much of the relevant evidence as possible in one coherent and dynamic simulation Provides a precise target for abstraction (which are then checkable against it) Stages abstraction from data to theory Separates representation and abstraction Preserves chains of reference Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-15

16. Aims and Objectives of DIM To develop a simulation that integrates as much as possible of the relevant available evidence, both qualitative and statistical (a Data-Integration Model – a DIM) Regardless of how complex this makes it A description of a specified kind of situation (not a general theory) that represents the evidence in a single, consistent and dynamicsimulation This simulation is then a fixed and formal target for later analysis and abstraction Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-16

17. DIM Development Method A relatively tight interactive “loop” between the social scientists who are experts in the subject matter and their data and the simulation developers... ...trying to give as much ownership and control to social scientists as possible. First target: What makes people vote (within the context of a diverse community)? Started with developing a fairly complete list of “causal stories” concerning the various processes that might contribute from Then initial model iteratively developed in NetLogo to enable maximum responsiveness and transparency To be reimplemented in Java/Repast when the target becomes more “settled” for more extensive simulation exploration and analysis Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-17

18. An overview of model structure Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-18

19. Demonstration Run Pictureof World ParametersandControls Indicative GraphsandHistograms SimpleStatistics concerningOutcomes Pseudo-narrative log of eventshappening to a single agent Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-19

20. Example Output – Turnout Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-20

21. Example Output – one agent 1945: (person 712) did not vote 1946: (person 712) started at (workplace 31) 1947: (person 712)(aged 29) moved from (patch 4 2) to (patch 5 3) due to moving to an empty home 1947: (person 712) partners with (person 698) at (patch 5 3) 1950: (person 712) did not vote 1951: (person 712) separates from (person 698) at (patch 5 3) 1951: (person 712)(aged 33) moved from (patch 5 3) to (patch 4 2) due to moving back to last household after separation 1951: (person 712) did not vote 1952: (person 712) partners with (person 189) at (patch 4 2) 1954: (person 712)(aged 36) moved from (patch 4 2) to (patch 23 15) due to moving to an empty home 1955: (person 712) did not vote 1964: (person 712) started at (activity2-place 71) 1964: (person 712) voted for the red party 1966: (person 712) voted for the red party 1970: (person 712) voted for the red party 1971: (person 712) started at (workplace 9) 1974: (person 712) voted for the red party 1979: (person 712) voted for the red party 1983: (person 712) died at (patch 23 15) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-21

22. Social Network at 1950 Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-22

23. Social Network at 1980 Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-23

24. Social Network at 2010 Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-24

25. On-going research and issues Suggests that this approach might be able to include and integrate qualitative evidence alongside quantitative evidence, but the method to do this is not well developed Creating and maintaining chains of models takes a LOT of time, resources etc. Does allow a more principled abstraction to physics type models, since at least some of the assumptions can be tested New trade-offs will no doubt be revealed! Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-25

26. Problem 2: Modelling and Context-Dependency Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-26

27. Another modelling trade-off Some desiderata for models: validity, formality, simplicity and generality these are difficult to obtain simultaneously (for complex systems) there is some sort of complicated trade-off between them (for each modelling exercise) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-27

28. Another Dilemma! In order to meaningfully model, communicate or apply knowledge it has to be valid in more than one specific situation Yet since the AAA rules out accessible models that have general applicability… …we are stuck with models that seem valid and are comprehensible only in specific contexts One response is to make fairly simple models that give the perception that they have considerable generality, but in fact are only useful as elaborate analogies Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-28

29. known unknown Object System encoding(measurement) decoding(interpretation) input(parameters, initial conditions etc.) output(results) The (direct) modelling relation Model Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-29

30. Analogies Analogies are usually verbal, but can also be formal (equations, simulations, etc.) Their mapping to what is being considered is built “on the fly” for each situation Analogies seem to be very basic to the way humans think and communicate Their mapping to the situation is different for each context and each person (in contrast to a model where the mapping is defined) This is done automatically and largely unconsciously This gives the illusionof generality Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-30

31. Object System conceptual model Model Modelling a concept of something Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-31

32. What is Essential to (empirical) Science? This part of the talk argues for the following strategy:weakening the generality of our formal models to achieve more validity in the face of the AAA … or, to put it another way, against the following strategy:weakening validity (e.g. to analogy) to preserve(the illusion of) generality Validity: agreement of models to what we observe (the evidence), not science otherwise Formality: formal models (maths, simulation) are precise and replicable – essential to being able to build knowledge within a community of researchers Simplicity: ability to analyse/understand our models, good to have but unattainable in general (AAA) Generality: the extent of the applicability/scope of a single model, there needs to be some small generality to apply models in places other than where developed, but wide generality not necessary Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-32

33. Context “Context” is used in many different senses across different fields The senses and concepts herein come from discussions and papers presented at the international series of conferences on “Modelling and Using Context” Somewhat of a “dustbin” concept resorted to when more immediate explanations fail Problematic to talk about, as it is not obvious that “contexts” are identifiably distinct Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-33

34. Situational Context The situation in which an event takes place This is indefinitely extensive, it could include anything relevant or coincident The time and place specify it, but relevant details might well not be retrievable It is almost universal to abstract to what is relevant about these to a recognised type when communicating about this Thus the question “What was the context?” often effectively means “What about the situation do I need to know to understand? Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-34

35. Linguistic Context This is the set of all language that precede or surround a focus utterance or phrase (the linguistic subset of the situational context) E.g. what pronouns might refer to Historically the last resort of the linguist when trying to pin down meaning Now thought central to natural language production and understanding Can be extensive, relying on distant texts or linguistic norms learnt previously Sometimes includes common knowledge needed to distinguish meaning Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-35

36. Cognitive Context (CC) Many aspects of human cognition are context-dependent, including: memory, visual perception, choice making, reasoning, emotion, and language The brain somehow deals with situational context effectively, abstracting kinds of situations so relevant information can be easily and preferentially accessed The relevant correlate of the situational context will be called the cognitive context It is not known how the brain does this, and probably does this in a rich and complex way that might prevent easy labeling of contexts Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-36

37. The Context Heuristic Divide the world into different sorts of situation (hereafter simply called a context) Learn/recognise these in a rich and “fuzzy” manner “Crisp” knowledge is “packaged”“within” such for reasoning, update etc. Makes within-context reasoning, models, update etc. more feasible Whilst each model has limited scope, together they might cover more ground, albeit in a more “patchy” manner Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-37

38. About Context-Dependency Context-dependency is not relativitysince contexts can be reliably recognised (and/or corrected if wrongly recognised) But since it might be recognised in a “fuzzy” and unconscious manner the bounds of the context may not be reifiable in crisp terms This is a heuristic – a strategy that may help push forward the boundaries of formal empirical science There is some evidence that our cognition is context-dependent in many ways which means that to a considerable extent it may be unavoidable Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-38

39. Why might the world we study be usefully split into such “contexts” In some domains, e.g. ecology or social science contexts might be co-developed over time between the entities (e.g. a niche, or social context like a lecture) In some others it may be the only practical way to proceed, as argued above In yet others our cognitive, unconscious tendency to deal with the world in terms of contexts might lead us to try and divide the world along less useful lines Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-39

40. Context and Causality In almost all situations (and all social situations) there are an unlimited number of things that could be attributed as a cause Related to “Causal Spread” (Wheeler); “Wild Disjunction” (Fodor); and “Embeddedness” (Granovetter) Without a limitation as to the scope causation makes no sense However given a context there are many factors that can be assumed to be insignificantly relevant and/or constant Thus causality makes sense givena context, since it excludes most possibilities Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-40

41. Attending to Context Given that attending to context is not unscientific and is inevitable (I argue) Then rather than pretending to generality by using models as analogies (only)… …I suggest attending to and incorporating context-dependency in our investigations (as far as this is possible)… …and hence pushing the “envelope” of science a further in the face of complexity Fortunately, computers allow us to keep track of a complication of multiple contexts and avoid premature generalisation (we no longer have to weaken validity to get formality) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-41

42. Globally coupled Variance(scaled by size) Model with random noise Size Kaneko (1990) • Exhibited a system of parallel chaotic but weakly coupled processes • Each process seems chaotic and independent • But as system size increases, variance as a proportion of size does not disappear • Law of large numbers does not apply Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-42

43. An Illustration of Masked Context-Dependency Global models are simply uninformative when the phenomena is context-dependent Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-43

44. Cleveland Heart Disease Data Set – the processed sub-set used In processed sub-set: • 281 entries • 14 numeric or numerically coded attributes • Attribute 14 is the outcome (0, 1, 2, 3, 4) • Some attributes: age, sex, resting blood pressure (trestpbs), cholesterol (chol), fasting blood sugar (fbs), maximum heart rate (thalach), number of major vessels (0-3) colored by flourosopy(ca) • From the Machine Learning Repository Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-44

45. General Correlations (1% Sig) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-45

46. Fitting a Global Model (R=56%) Num = -0.01*age + 0.17*sex + 0.20*cp + 0.00*trestbps + 0.10*restecg + -0.01*thalach + 0.23*exang + 0.18*oldpeak + 0.16*slope + 0.43*ca + 0.14*thal + -0.60 (+/- 0.83) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-46

47. Looking for Clusters in HD Data Set (Start of Process) Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-47

48. After Solutions Locally Evolve Speciation of Solutions In some areas no solution dominates Some Solutions Spread over area of applicability Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-48

49. Final Set of Clustered Solutions Final solution set after some time. Still complex but some structure is revealed Note presence of “fbs” despite not being globally correlated and that “chol” helped define the context space Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-49

50. About this approach Difficult to quantify the extent to which one is “cherry-picking” (overfitting) models This only looks at one aspect of the data – one predicted variable A more meaningful correlate of our cognitive contexts would cluster situations as to many different aspects But could inform the specification of individual-based models, encoding different behaviours for each detected cluster Staging Abstraction using Chains of Models, Bruce Edmonds, CCSA Seminar, York, March 2012. slide-50