Michael Lanham, Geoffrey P. Morgan, Kathleen M. Carley

1. Multi-level Modeling and Learned ResilienceModeling the flow of information to maintain/retain normal performance in the face of adversity Michael Lanham, Geoffrey P. Morgan, Kathleen M. Carley gmorgan | mlanham | kathleen.carley@cs.cmu.edu

2. Overview • Overall Model Setup and Assumptions • Model Detail • Scenarios • Results • Visualization

3. Approach • Model information flow through dynamic network analysis using both • Network analytics • Agent-based Dynamic-Network modeling • Network Analytic Framework is ORA • Modeling framework is Multi-Level Construct

4. What is Multi-Level Construct? • Extension of Construct to support Meade’s idea of the “Generalized Other”. • As formulated by Meade (1922): An ego knows how it (and alters) should behave because the ego has some perception of what is typical behavior. • How we use this idea: The ego attributes X knowledge to alters it does not know because it perceives that: a) the alter is part of some group, and b) a majority of the alters I do know from that group know X. • Allows dynamic link change over time, as agents interact with agents they expect to have valuable knowledge based on their expectations.

5. The Organization • The organization has several parts of interest, these are: • Stylized C-Suite: The CEO and heads of various functional groups within that organization linked in a mesh (everyone to everyone) • The members of the C-Suite are: • CEO  HR • Ops  Security/IT • Logistics  Finance • C-Suite members communicate with peer C-Suite at other locations • Planning Team: • Comprised of Ops, Security, and Logistics • Per Location, Planning Team Size= 10% * OrgSize, [3..15] • Planning team members are distributed across locations and remain in contact with each other • Staff: all members of the organization

6. Uncontested vs. Contested Cyber Environments • Uncontested Cyber Environment - “Normal” • Multiple Communication networks: • Unsecure IT  Secure IT • Phone Direct/face-to-face • No Confidentiality, Integrity, or Availability concerns • Contested Cyber Environments - “Under Attack” • Attacks may vary across number of locations and/or systems affected. • Confidentiality: Humans may leak classified information to humans or IT systems not cleared for access • Integrity: A malicious actor may introduce harmful information to the planning process through backdoor entry to the key IT systems • Availability: Links to IT Systems are degraded, and users may not be able to communicate with these systems at all times

7. Organizational Structures • Organizations • Organizations are composed of both human and IT actors • We will examine six stylized configurations: • Erdos-Renyi Random  Mesh, with C-suite roles • Hierarchy by Functional Area  Scale Free • Matrix (Hierarchy with cross-hierarchy teams)  Team by Functional Area • We will will also examine stylized Air Operations Center (AOC) • Humans • Have both a formal “works with” structure, and also a stylized “small-world” social structure • Have group-based stereotypes of alters they do not actually know. • Because the setting is work-oriented, agents are more likely to interact across the formal work ties than interact across the informal social structure • IT Systems • Some IT systems are “Key”, with lots of relevant information and more connected to other IT systems than other IT Systems • Some IT systems hold classified knowledge, and some should not • We use a 5:1 ratio of people to IT systems in this particular example • We will examine two stylized IT-to-IT link configurations. These represent application-to-application exchange of data: • Scale Free Random

8. Hierarchy with C-Suite Roles • 6 agent mesh for formal C-Suite links • Formal Network • Hierarchy by functional area • Fanout= Uniform Random(7±2) • Informal Network: Small World • Prewire= 0.05 and Pdelete= 0.05 • Sizelocal =Uniform Random (7±2) • Agent x IT Level 1: Mesh • Cleared Agent x IT Level 2: Mesh • Visual Samples are to the left Blue Link = Social Network Red Link = Agent x Agent

9. Modeling IT • Each staff member is mesh connected to each Level 1 IT system • Cleared staff members are mesh connected to each Level 2 IT system • IT Systems are push/pull (they can initiate as well as receive messages to other actors) • IT-to-IT Connections examined in two conditions, • Scale Free, Level 1 (Non-Secure)  Erdos-Renyi Random, Level 1 and Level 2 (Secure) (Non-Secure) and Level 2 (Secure)

10. Interactions between Actors • Human to Human • Intra-Location • Face to Face • Phone • Email • Inter-Location • Phone • Email • Human to Local IT • Direct • IT to IT • Direct • Human Egos • are boundedlyrational • able to communicate knowledge facts & perceptions of other’s facts • can misperceive messages • can misperceive their alters’ knowledge • Give preference to plan-oriented knowledge • can update or get information from any IT system they have access to

11. Structuring Knowledge • Agents may have • Common organizational knowledge – “This is how we do things at Acme” • Locational-specific knowledge – “This is how we do things at Acme-Timbuktu” • Requirements knowledge • “Bad” Requirements knowledge – each bad knowledge fact negates the value of having one correct requirements knowledge fact • Knowledge has a classification level • Level 1 – Everyone can see it • Level 2 – Only cleared agents should see it

12. Modeling Multiple Locations • Organizations linked via • high density Erdos-Renyi of C-Suite agents • Key IT-systems per org • Special/Attack Agents linked to all key-IT systems • Inter-organizational Physical Proximity set Agent x Agent Agent x Agent (red) Agent x Knowledge (blue)

13. Outputs • Measures will always be context dependent (e.g., a BCT will not be combat ready during block leave periods) • Our measures are: • Relative Time Measurements • Time to get 70% of requirements knowledge to 90% of all human actors • Time to get 80% of requirements knowledge to 90% of C-Suite actors • Time to return to baseline Performance as Accuracy • Time to return to baseline Knowledge & Task Congruence • Assessments of knowledge prevalence • Amount of bad knowledge across the organization, the C-Suite, and the IT Systems • Amount of Lvl 2 “Classified Knowledge” possessed by non-cleared human and IT actors

14. Experimental Summary

15. Expectations • Contested Environments should slow down effective diffusion of the planning requirements • Effects of attack will be non-linear across locations and quantities of systems affected • Effects of attacks will also be non-linear across attack types • Confidentiality attacks may improve performance • Organizations with denser ties will tend to diffuse the information more rapidly, although there may be spillage of classified information.

16. Backup Slides

17. Modeling staff • Each staff member has a functional role • HR: 10% of Org : 10% has access to classified info • Sec/IT: 20% of Org : 90% has access to classified info • Ops: 30% of Org : 90% has access to classified info • Logistics: 30% of Org : 50% has access to classified info • Finance: 10% of Org : 10% has access to classified info • Staff members has both “works with” and “social” relationships, and “works with” relationships have priority for interaction • “Social” relationships are a stylized small-world network within the functional group • “Works With” ties are based on the experimental condition • Staff members do not have “shifts”, but that could be a later extension • Disabled: Beliefs, Beliefs Transactive Memory, Tasks, Task Learning, Knowledge Forgetting,

18. Erdos-Renyi Random with C-Suite Roles • 6 agent mesh for formal C-Suite links • Formal Network • Erdos-Renyi Random Plink= 0.15 • Informal Network: Small World • Prewire = 0.05 and Pdelete= 0.05 • Sizelocal=Uniform Random (7±2) • Agent x IT Level 1: Mesh • Cleared Agent x IT Level 2: Mesh • Visual Samples to the left Blue Link = Social Network Red Link = Agent x Agent

19. Matrix with C-Suite Roles • 6 agent mesh for formal C-Suite links • Formal Network • Hierarchy by functional area • Fanout = Uniform Random(7±2) • Cross-Hierarchy Teams • 3:1 Team:Functional Area • Curtailed Gaussian Random [1,∞), mean 8, std 2) • Functional Area Pop. per team <= 20% • Informal Network: Small World • Prewire = 0.05 and Pdelete= 0.05 • Sizelocal =Random Uniform (7±2) • Agent x IT Level 1: Mesh • Cleared Agent x IT Level 2: Mesh • Visual Samples are to the left Blue Link = Social Network Red Link = Agent x Agent

20. Complete Mesh with C-Suite Roles • 6 agent mesh for formal C-Suite links • Formal Network • Complete Mesh • Informal Network: Small World • Prewire = 0.05 and Pdelete= 0.05 • Sizelocal =Random Uniform (7±2) • Agent x IT Level 1: Mesh • Cleared Agent x IT Level 2: Mesh • Visual samples to the left Blue Link = Social Network Red Link = Agent x Agent

21. Scale Free with C-Suite Roles • 6 agent mesh for formal C-Suite links • Formal Network • Scale Free via Preferential Attachment • Build Edges after C-Suite and Social Network in place • Density = 0.15 • Informal Network: Small World • Prewire = 0.05 and Pdelete= 0.05 • Sizelocal =Random Uniform (7±2) • Agent x IT Level 1: Mesh • Cleared Agent x IT Level 2: Mesh • Visual samples to the left Blue Link = Social Network Red Link = Agent x Agent

22. Functional Mesh/Team with C-Suite Roles • 6 agent mesh for formal C-Suite links • Formal Network • Complete Mesh within each functional area • Informal Network: Small World • Prewire = 0.05 and Pdelete= 0.05 • Sizelocal =Random Uniform (7±2) • Agent x IT Level 1: Mesh • Cleared Agent x IT Level 2: Mesh • Visual samples are to the left Blue Link = Social Network Red Link = Agent x Agent

23. Scenarios of Interest • For all • A generic multi-location organization is engaged in planning an integrated COA • Each location has received a set of planning documents and tasked a Planning team (PT) to analyze the classified documents, distribute their analysis to the key leaders of the organization to gain their input, and develop a plan to meet the intent of the planning documents • This experiment will explore • Uncontested vs. 3 types of Contested Cyber Environments • Hierarchical vs. Teams vs. Matrix Organizations vs. Random vs. Scale Free • And will examines these outputs • Change in Time for plan knowledge to reach key decision makers • Change in Time to get general planning knowledge to all actors • Amount of bad knowledge present in the organization, in the minds of key decision makers, and in IT systems • Amount of “spillage” of classified Knowledge to actors and IT systems not cleared for that knowledge

24. Planning an integrated COA • A multi-location organization is attempting to develop an integrated (across all locations) Course of Action (COA) • The requirements of that Course of Action are represented as stylized knowledge facts – some of these facts are classified, and some are not. CMU FOCUS Location 1 Location 2 Location 4 Location 3

25. Communication Structure • Planning team members prefer to communicate with those physically near them (other planning team members) • Heads of divisions communicate with each other and members of their division • Communication network within division is set per experimental condition • Head of functional area communicates to the two most central people in their area • Functional heads, division heads and division sub-heads are in a small world network – 11 connections • Communication network within a functional area is random • Inter-AOC communication • through C-Suite & Planning Teams (e.g., HR communicates with HR) • through “Core” IT systems

26. Modeling Communications Media • Face-to-face: • maxMsgComplexity: 4 - max%Learnable: 100 • Time_to_live: 1 - Time_to_send: 1 • Phone: • maxMsgComplexity: 3 - max%Learnable: 50 • Time_to_live: 1 - Time_to_send: 1 • Email: • maxMsgComplexity: 6 - max%Learnable: 70 • Time_to_live: 3 - Time_to_send: 1

27. People to People Network Structure for ‘Regional AOC’ Sized by Authority Centrality and Colored by Betweeness Centrality (Blue is most central) Functional Groups’ Agents collapsed into ‘meta-nodes’

28. High Level Assumptions • Model Assumptions: • Humans must receive information to gain it • Humans tend to interact with people similar to themselves • Regular human interactions within organizations are typically constrained by role and function • Information is divisible into smaller portions, and those portions can be communicated by human actors in such a way as to be re-constructible by the receiving agent • Cyber Attacks: • Can modify or destroy information • Delay or inhibit communication

29. Ability to Operate • People need 25% of the knowledge of the OPORD to begin to plan • People need 60% to complete an effective plan • Under normal conditions they get this information eventually • There is a time to get minimum knowledge • There is at each time a number of people and so a number of divisions that have that minimum knowledge

30. How are Planning Requirements Communicated In bits and pieces • Requirements go to the planning team at each location, all core IT systems initially (level 1 & level 2), and small portion CEO • Planning team goes through a plan-brief cycle • Briefings: 3 • Total Planning time: 1/3:2/3 rule of total time • Total Briefing time: 1/5 of Planning Time • During Briefings • ‘Plan’ knowledge given probabilistic preference over ‘general’ or ‘location’ knowledge • ‘Attendees’ adjust preferences of interaction to planning team members • Human actors can pass 2 to 5 bits per time • IT systems can pass 5 to 15 bits per time • IT systems are ‘push’ (can initiate communications)

31. Where Can Cyber Attacks Manifest • Within Location • Between Locations AOC A AOC B

32. Point of Contact Prof. Kathleen M. Carley Wean 5130 Carnegie Mellon University 5000 Forbes Ave. Pittsburgh, PA 15213 USA Tel: 412-268-6016 Fax: 412-268-1744 kathleen.carley@CS.CMU.EDU