Revealing the weakness of SNA and possibly fixing it , using MAS

1. Revealing the weakness of SNA and possibly fixing it, using MAS Bruce EdmondsCentre for Policy ModellingManchester Metropolitan University

2. Revealing the weakness of SNA and possibly fixing it, using MAS Introduction Modelling and Social Network Analysis Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-2

3. known unknown Object System encoding(measurement) decoding(interpretation) input(parameters, initial conditions etc.) output(results) Modelling parts and relations Formal Model All the stages are necessary for the model to be useful Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-3

4. Object System conceptual model Model Modelling ideas rather than observed systems Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-4

5. Some ‘scientific’ uses of modelling • Prediction: Provide information about a current unknown by inference from known information • Explanation: Provide an explanation how an outcome resulted from some conditions • Analogy: Provide a framework for (or a way of) thinking about a complex system • But there are many other uses: illustration, personal exploration, persuasion, counter example, etc. Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-5

6. Social Network Analysis • Abstracts a target system to a system of (possibly rich and dynamic) nodes and arcs • It is necessary to decide what a node and an arc are(in terms of what nodes and arc represent in the target system) • Key idea: the structure of the abstracted network tells us something useful about the properties of the system Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-6

7. known unknown Object System representation in terms of arcs and nodes interpretation of output Modelling parts and relations with a Social Network Model SNA Model output in terms of visualisation, measures etc. It is not a model until there is an analysis of a network that can be interpreted in terms of the object system Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-7

8. About SNA Models • Representing anything as a network involves many decisions as how to do this • The resulting representation is only a model if one can infer anything from it • Often this inference is implicit or informal • If the inference is specified it can be called a Social Network Model (SNM) • Any model of an observed system is a contingent (mini-)theory • that is, it could be wrong however plausible it seems Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-8

9. Descriptive Network Analysis Choose and use measure etc. to illustratethat understanding A measure on the network, M(x) B A Based on an already existing good understanding of what is happening in the target system Measures upon networks – the very idea!, Bruce Edmonds, Social Netwrok Conference, London, 14th July 2007. slide-9

10. Network Analysis as a Generative Model Given an observed system model it with a SNM E.g. using a measure on the network, M(x) B A Use the model to infer something about the model that is meaningful in terms of the observed system Measures upon networks – the very idea!, Bruce Edmonds, Social Netwrok Conference, London, 14th July 2007. slide-10

11. Validating a Social Network Model • Since a SNA model is a (complex) contingent hypothesis about the target system • To be trusted it needs to be independently validated (strong validation) • This is very expensive to do with SNMs since not only does the data need to be collected and the model built but it also tested against what is measured in the target system • So instead it is usual to validate weakly using the intuitions of the researcher who did the analysis which is clearly insufficient if we are to rely on it for any purpose (e.g. understaning) Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-11

12. Using ABS to Probe SNA Assumptions • However we can explore the robustness of SNA against plausible social simulations • An artificial test-bed for SNA • This can indicate the conditions under which a particular SNM can be relied upon (or not) given which assumptions • If a SNM of an ABS cannot be made to work how could we rely on this when considering real social phenomena? Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-12

13. Revealing the weakness of SNA and possibly fixing it, using MAS Example 1 Analysing a Simulation of a P2P System Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-13

14. Collection of ‘servers’, each of which: Is controlled by a user to some extent ‘Knows’ a limited number of servers, with which it can communicate (the network) Makes some (or no) files available for download by other servers Search for files is by flood-fill: (i.e. send query to n others who send it to n others…) If query matches an available file it is sent back to originator E.g. Bittorrent Example I:A Peer-to-Peer (P2P) File-sharing system Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-14

15. A Simulation of a P2P System • 50 servers, each can decide to share files (coop) or not (def) at any time • Try collect ‘sets’ of related files stored (initially) randomly by sending queries • Satisfaction is measured by success at collecting files – (small) cost of dealing with others’ queries (but decays over time) • May look at and copy what a more satisfied server does, or may drop out and be replaced (especially if satisfaction is low) Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-15

16. Number of co-operators in a run of the simulation (out of 50) • Key issue is number (and manner) of cooperation • Why does anyone cooperate? • How does network structure impact upon this? Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-16

17. Typical Emergent Network Structure Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-17

18. Suggests four types of node • In-coop – those who share their files in core partition • In-def– those who don’t share their files in core partition • Out-coop– those who share their files but are outside the core partition • Out-def– those who don’t share their files but are outside the core partition Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-18

19. Some General Statistics Over all the runs for all nodes and later times Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-19

20. Stop!! Time for a Thought Experiment If this were observations of a real P2P system and not a simulation, what would youconclude from this analysis: • That the kind of node (using the above categorisation) was a significant factor in the utility of nodes? • That either a node’s number of links or centrality was a significant factor in achieving its utility? Wouldn’t a paper that came to positive conclusions on these questions be publishable? Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-20

21. Over all kinds of nodes and later times and runs Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-21

22. Regression coefficients with satisfaction levels of nodes Other measures and lags had lower correlations, including those that just did these in aggregate Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-22

23. Size of partitions during a run Blue – size of largest partition Green – 2nd largest (if there is one) Red, orange, etc. – even smaller ones Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-23

24. Conclusion of P2P Case-study • The global measures were not very useful in providing understanding leverage • It can be unsafe to assume that such measures derived from empirical studies give a helpful picture of the role of networks • The structural analysis based on the detailed understanding of the dynamics created a more useful categorisation of node types (but this is precisely the kind of understanding difficult to obtain when the system is real rather than simulated) • Given this understanding it might be possible to choose better measures etc. • It is important to distinguish demonstrating an existing understanding of a network from fishing for understanding using SNA measures Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-24

25. Revealing the weakness of SNA and possibly fixing it, using MAS Example 2 A “simple” abstract system Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-25

26. The Target System • So here I will consider this system looking at the question of whether any measure can be relied upon to indicate eventual node importance. • It is: • Relatively simple • Deterministic • About which we have almost complete information about behaviour, links, etc. to help us chose our measure Measures upon networks – the very idea!, Bruce Edmonds, Social Network Conference, London, 14th July 2007. slide-26

27. The Abstract System E ? 2 1 2 1 9 1 2 0 2 Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-27

28. Basic System Outline • Giving Agent System with Plans • Fixed number of agents: A1, A2, …An • Each agent, Ax, has • a store, Sx • a fixed number of plans: Px1, Px2, … • Each Plan, Pxy, composed of instructions: • A fixed number of “give one to” • And one test: If Si is zero then do plan j next, otherwise plan k next • Each time click, all do: get 1 unit; use current plan to: [do giving (while they have); test others; note next plan]. Using the Experimental Method to Produce Reliable Self-Organised Systems, B. Edmonds, ESOA 2004, New York, July 2004, http://cfpm.org/~bruce slide-28

29. Thus all that happens in GASP systems is: • That agents have a fixed set of very simpleplans/programs • Their state is the amount in their store and the index of the current plan • All they do is give fixed amounts to other agents accorind to their current plan • All they can perceive is whether an other agent’s store is zero or not… • …which determines the index of the next plan in a fixed way Using the Experimental Method to Produce Reliable Self-Organised Systems, B. Edmonds, ESOA 2004, New York, July 2004, http://cfpm.org/~bruce slide-29

30. An illustration of a GASP system Agent 1 Plan 1: G3 G2 JZ2,1,3 Plan 3: G2 G2 G2 JZ2,3,3 Plan 2: JZ1,2,3 Agent 2 Agent 3 1 2 3 4 1 2 3 Etc. Store: 27 Check if zero Using the Experimental Method to Produce Reliable Self-Organised Systems, B. Edmonds, ESOA 2004, New York, July 2004, http://cfpm.org/~bruce slide-30

31. Thus the reformulated question is... Given almost complete knowledge of a particular GASP system (except for the initial store of Agent-1), can you effectively find anymeasure, M, such that: • If and only if M(A) ≥ M(B) then... • Eventually S(t,A) ≥ S(t,B) [ where S(t,x) is the value of the store in agent x at time t ] • That is given this system is there an M: M(A) ≥ M(B) ↔  T; for t>T S(t,A) ≥ S(t,B) Measures upon networks – the very idea!, Bruce Edmonds, Social Network Conference, London, 14th July 2007. slide-31

32. And the answer is... No! • In other words, there are GASP systems, where even though we know: their complete behaviour (comparable to detailed interviews of all participants); everything possible about their social network (who they can make transfers to); and almost all of the initial conditions (except one value)... • ...there is no measure that will tell us from the structure which nodes will be more influential than others once running. Measures upon networks – the very idea!, Bruce Edmonds, Social Network Conference, London, 14th July 2007. slide-32

33. Proof Sketch • The class of GASP systems are Turing Complete, in other words they can compute anything a Turing Machine (TM) can (shown by a mapping into an Unlimited Register Machine a know TM equivalent). • If there were a such a measure, then we could use it to check (without computation) that the results of two GASP systems (the end value in the store of Agent-1) were equal by joining the two systems into one; finding the measure, M and then using it to see if the two output nodes would be equal. This is a known uncomputable problem. Measures upon networks – the very idea!, Bruce Edmonds, Social Network Conference, London, 14th July 2007. slide-33

34. (the pessimistic) Moral! • Even with very simple, deterministic systems, where we know everything about the behaviour and structure of the system, there are no measures that a priori inform us about node importance. • This backs up simulation studies where a set of apparently sensible measures fail to do the same. • Therefore the burden of proof is on those that claim, with a largely unknown complex system, that a measure will tell us such information! • Effective measurement follows understanding Measures upon networks – the very idea!, Bruce Edmonds, Social Network Conference, London, 14th July 2007. slide-34

35. Revealing the weakness of SNA and possibly fixing it, using MAS Part 4 The Cure? Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-35

36. Why SNM might be inadequate • SNM are simply too abstract to adequately represent social complexity • The jump from rich social phenomena to simple network model is too great • This is usually masked by • The prima face plausibility of SNM • That SNA work is traditionally divided between: • Theorists that study what can be inferred from SN • Social Scientists who represent using SN and then trust that the theoretical techniques work Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-36

37. Staging Abstraction using ABS Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-37

38. Example II: A Simulation of Ga-Selala (in the Limpopo Valley of South Africa) • A Complex Evidence-led Simulation of a particular village • Represents many aspects of life there, including: sexual network and HIV/AIDS spread, friendship network, kinship network, employment, savings clubs, household structure, birth and death, government grants and health • Purpose was to assess impacts of factors, in particular how fragile the social structure might be to these factors given the complex interplay of the various social structures and behaviours Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-38

39. Basic Methodology • Repeated iterations of model development in response to stakeholder criticism, expert opinion, statistics, interviews etc. • So that most aspects of the model had some (but varying) levels of justification from available evidence • Result is a context-specific but dynamic “description” using a computer simulation • Simulation is difficult to understand and slow to run, but open to experiment and inspection • Changes in network structure can be studied in the simulation even though it is highly dynamic Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-39

40. Observations from running simulation experiments • That (given the introduction of a new mining enterprise near the village) the social structure(s) collapsed • To try and show this, snapshots of the social network taken and their degree distribution compared using non-parametric statistics (Kolmogorov-Sinai) to see if there is evidence of significant change Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-40

41. Comparing the social network over time with that at time 0 Initialised with Watts-Strogatz Small-world network Initialised with Erdös random network P-scores of K-S test on the degree distributions of the social networks Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-41

42. Comparing the social network over time with the previous time Initialised with Watts-Strogatz Small-world network Initialised with Erdös random network P-scores of K-S test on the degree distributions of the social networks Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-42

43. Conclusion of talk • SNM are weak in the sense that they are contingent and yet almost always without any independent validation • Their apparent power comes from their simplicity and plausibility • ABS can be used to test the assumptions behind SNA analyses in vitro • ABS can be used to stage the abstraction from evidence to SNA, allowing chains of reference to be maintained and understanding gained to inform SNA Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-43

44. The End Bruce Edmonds http://bruce.edmonds.name Centre for Policy Modelling http://cfpm.org Manchester Metropolitan University Business School http://www.business.mmu.ac.uk Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-44