Pythagoras - an Agent-based Distillation

1. Pythagoras - an Agent-based Distillation A Presentation to ABSVal Workshop 25 March 2008 Edd Bitinas, Chief Modeler – 703-968-1196 Brittlea Sheldon, Analyst – 703-968-1137 Northrop Grumman Corporation NORTHROP GRUMMAN PRIVATE/PROPRIETARY LEVEL 1

2. Introduction To Pythagoras What Is Significance Of Pythagoras And Why We Developed It How Pythagoras Works What We Have Done To Improve Pythagoras What Our Clients And Other Users Have Done With It Overview

3. The Pythagoras Environment • Decision Making • Intelligence • Communication • Inspiration Leadership • Morale • Training • Confidence • Fatigue • Fear • Comradeship • Political Leanings • Anger • Influencers • Sensors • Platforms • Resources • Terrain • Weather Human Factors Physics The Three Regions Are Tightly Linked

4. Physics-Based Verifiable through Experimentation May Not Account for All Interactions Example: Weather Models Outcomes Could Be Calculated, but Math Is Too Hard Probability-Based Includes Some Physics Account for Some/Most Random Effects Range of Outcomes Could Be Calculated (See Above) Agent-Based Decision-making Entities Includes Probabilities Outcomes Not Predictable Types of Models/Simulations • Software Objects That: • Are Capable of Choice • Have Autonomous Behaviors and Are Distinct from the Background or Environment and Each Other • Perform Actions Like: • Sense • Order • Communicate • Move • Influence Pythagoras

5. July, 2001 – Northrop Grumman Wins USMC Contract To Support Project Albert via Archimedes Development and Modification September, 2001 – Northrop Grumman States ‘It Can’t Be Done.’ Project Albert Leadership States ‘We Need an Agent-Based Tool That: Uses Fuzzy Logic Can Run Both in a Batch Mode at Maui High Performance Computing Center (MHPCC) and from a GUI on a PC Is Easy Enough That a Non-Computer Science Major Can Use It.’ Pythagoras Development Timeline April 2002 – Pythagoras v1.0 Is Delivered: Reproduces Archimedes Functionality June 2003 – Pythagoras v1.6.3 Is Delivered: Added Complete Behavior Change Triggers September 2005 – Pythagoras v1.9.0 Is Delivered: Influencers Replace Weapons September 2006 – Pythagoras v1.10.5 Is Delivered: Adds Communications Devices April 2008 – Pythagoras v2.0.0 is Delivered – Upgrade to JAVA 1.5, JAXB 2.0, Add More Attributes, More... A Brief History of Pythagoras

6. Pythagoras Is an Agent-Based Simulation System That Puts the Complexity of Creating Behaviors in the Hands of the Analyst, Not in the Software It Offers the Analyst the Ability To: Mirror Actual Groups, Including Their Characteristics, Beliefs, and Standards of Behavior Examine Outcomes Based on Behavior Data Farm Over a Trade Space Using High Performance Computer Clusters Execute a Range of Runs Similar to Sensitivity Analysis Design of Experiments Can Reduce Runs and Preserve Interactions Assess Impact of: Uncertainty in Inputs Some Idea of the Minimum-To-Maximum Range Unclear Interactions Among Inputs User Interface Is Easy To Understand and Manipulate The Structure Allows the Functionality To Evolve as New Scenario or Operational Requirements Are Identified What is Pythagoras?

7. Pythagoras Software Components Engine MOE Output CLI Scenario Movement Output Legend XML File GUI JAVA Software Text File

8. Soft Decision Rules A Way To Capture Variability Agent’s Decisions Movement – Influencing Generic Attributes Characteristics To Measure Controls On Behavior Dynamic Affiliation Same Unit – Same Side Enemy – Neutral Rule-Based Influencers Lethal – Non-Lethal Triggers Change Behavior Rules When Triggered MOEs Sensors (Three Bands) Comms (Three Channels) Terrain Features Mobility – Protection – Height Concealment (Three Bands) How Does Pythagoras Work?

9. Affiliation/Uniform (Shades of Red, Green, Blue) Vulnerabilities (Multiplier on Lethality and Affiliation Changes) Up to 10 Optional Attributes Attribute Vulnerability (Multiplier on Influencers Affecting Attributes) Leadership (Charismatic or Hierarchal) Obedience – Repeating Orders Results in Nagging Knowledge Retention Communications Effectiveness Marksmanship – Strength of the Ability To Influence Height and Altitude Detectability (Multiplier on Sensor Detection) Vulnerability (Multiplier on Lethality) Speed, Speed Variability Movement Selection Mode (Mode 1, 2, 3 or 4) Has Weapon Type (Up to Ten) Has Sensor Type (Up to Ten) Has Comm Type (Up to Ten) Next Waypoint Waypoint Proximity (How Close Is Close Enough?) Target Value Hold-Fire Desire Target Persistence Fuel Tank Capacity, Fuel Initial Load, Usage per Step Resource X,Y,Z Capacity, Resource X,Y,ZInitial Load, Usage per Step Agent Characterization Italics Indicate Possible Soft Decision Rule Variable

10. Agent Time Cycle Die Self Evaluation Clear Knowledge Record Info Draw GUI Info Sense Environment Change Color/Attributes (if Necessary) Communicate Knowledge Choose Leader Move (if Can) Re-Supply (Resources) Pick Direction Load Balance (Resources) Set Speed Change Behavior (if Trigger Tripped) Influence Check if at Waypoint Reset Influencers & Pick Targets

11. MOEs Recorded Over Time Include: Agent Red, Green, Blue Agent Attributes Agent Behavior Agent Fuel, Resource X, Resource Y and Resource Z # Detected Units (Friend, Enemy, Neutral) # Known Units (Friend, Enemy, Neutral) Distance to Final Objective Killed By Kills Scored Shots Fired Survival (Alive, Dead, Injured) Recording Methods Include: Individual Agent Entire Class Average Frequency Options: Not Collected End of Run Beginning and End of Run Beginning, End and Every x Time Steps Time Series MOEs

12. Initial, Minimum, Maximum and Final Number of Agents Near the Final Objective This Can Measure How Agents Force the Enemy To “Run Away” Initial, Minimum, Maximum and Final Average Number of Unit, Friendly, Enemy, Neutral and Unknown Initial, Minimum, Maximum and Final Value of Red, Green, Blue, Attributes 1-10 Initial, Minimum, Maximum, Final and Used Amount of Resource X, Resource Y and Resource Z, Fuel Number of Survivors by Class Number of Casualties by Class Both Injured and Dead Number of Shots (by Influencer) and Effects End of Run MOEs

13. Modernization Java 1.5 JAXB 2.0 4000 X 4000 Pixel Playbox Multi-Dimension Visualization Map Location, Color, Health, Attributes and Resources to x, y, r, g, b, and Transparency Ten Generic Attributes Changed By: - Changer Types Weapons • Incremental Communications • Absolute Events • Relative Terrain • Multiplier Can Be Normalized New in Pythagoras 2.0.0

14. Agents Carry Up To Ten Each Of Sensors Weapons Communications Devices Agents May Now Shoot At Multiple Targets Per Time Step Pythagoras 2.0.0 (cont.) Status: Delivered To OAD & NPS, April 2008

15. Terrorist Development (Sandia Labs) Conceptual Model of Human Factors: Explored Terrorist Recruiting Population Dynamics (USMC) Disaster Relief and Counter-Insurgency: Examined Impact of MAGTF Commander’s Alternatives on Insurgency MAGTF Optical Requirements (Night Vision Lab/Ft. Belvoir) Peacekeeping at Night – Comparison of Alternative Optical Technologies/Suites Historical Analysis (USNA) Battle of Midway Battle of Ia Drang, Viet Nam Thermobaric Weapons Assessment (MCCDC) Urban Environment Shallow Water Obstacle Clearing (MCWL) Use of JDAM and Robotics Homeland Defense (Northrop Grumman) Pre-Proposal Analysis of Critical Node Defense from WMD Environmental Concerns (MITRE) Spread of Hemlock Woolly Adelgid User Community NPS – SOCOM – USNA OSD – USMC/MCCDC – TRAC/Monterey Northrop Grumman – USMA – Many Others How Has Pythagoras Been Used?

16. Government-Owned, Open Source Wide Variety of Potential Applications Able To Deal with Uncertainty Scenarios Are Expandable as New/More Data Become Available Not Limited to One Theory/Approach Compare Theories with the Same Software No ‘Scripts’ Group Behavior Emerges from Individual Behaviors Scenarios Can Be Built In Hours Complexity Can Be Added As Needed Summary

17. Backups

18. Similar To Sensitivity Analysis Multiple Simultaneous Variations In Inputs Stochastic Scenarios May Require Tens Of Thousands Of Runs Use Super Computer Clusters Design Of Experiments Can Reduce The Number Near Orthogonal Latin Hypercube, D-Optimal, Others Identify Correlations And Trends Outliers Also Provide Insight Why Was It Different? Can That Difference Be Exploited? What Is Data Farming?

19. Sample Animation Output Improvised Explosive Devices Convoy of Vehicles Detonator

20. Event-Driven Engine Option Current Version Is Time-Stepped Arc/Node Terrain Network-Oriented Terrain Option Would Provide New Capabilities Configurable GUI Start Simple, Add Capability as Needed Additional Functionality Imperfect Knowledge About Affiliation More Fire And Movement Desires “OR” Options for Attributes and Affiliations Moving and Shooting Detection Explicit Initial Locations Many Others... Application-Driven Upgrades What’s Next?

21. An Attempt To Turn English Language Orders Into Numbers Example: “Don’t Communicate Until The Enemy Gets Close Enough” How Close Is Close? People Interpret And Apply Orders Differently Example: A Team Agrees That ‘Close’ Is 100 Meters. Some Start Talking At 115 Meters; Others Wait Until 90 Meters People Interpret And Respond To Input Differently Example: MAGTF Optical Scenario Hungry Villagers Move Toward Marines, Terrorists Move Away From Marines Soft Rules Attempt To Capture This Variability Each Agent Selects His Own Thresholds From An Input Range Random Distribution May Cause Emergent Behavior Expect The Unexpected Soft Rules

22. Example of Individualized Numbers Fraction of Total Common Decision Threshold: Mean = 5 Firm/Low Individuality: Range = 1 Soft/Medium Individuality: Range = 2 Loose/High Individuality: Range = 4 Homogenous Populations React Alike Heterogeneous Populations May React Differently or at Different Times

23. INFLUENCER/WEAPON: Hold Influence Unless Being Influenced/Actively Influence Highest Effectiveness/Lowest Effectiveness MOVEMENT: Toward (if > d) or Away From (if < d) Closest Leader Toward (if > d) or Away From (if < d) Closest Unit Member/Friendly Toward Furthest Unit Member/Friendly (if > d) – Group Cohesion Toward Next Waypoint (if > d) Toward (if > d) or Away From Nearest Enemy (if < d) Toward Small Number of Enemy (if # Enemy or Ratio within d < e) – Advance Away From Large Number of Enemy (if # Enemy or Ratio within d > e) – Withdraw Toward Objective Toward Injured Friend – Restore or Heal Toward Friend (if > d) Needing Fuel, Resource X, Resource Y, Resource Z Toward Friend (if > d) Supplying Fuel, Resource X, Resource Y, Resource Z At Random – Crowds Continue Same Direction Stay in Place if Desire To Move Is Weak –Sloth TERRAIN: Avoid Bad Terrain if (Low Mobility or Poor Concealment or No Defense) Prefer Good Terrain if (Azimuth, High Mobility, Good Concealment, or Protection) Agent Decision Variables d – Distance, e – Enemy Count All Breakpoints Are Soft

24. Each Agent Calculates the Movement Direction for Each Non-Zero Desire They Are Then Combined Either by: Using Vector Algebra, with the Desires as a Weight, or Picking the ‘Strongest’ One (the Highest Desire), or Averaging the Top Two, or Using Monte Carlo, Weighted by Desire Thus, Movement Directions Are Calculated Based on the Current Situation, Not Scripted in Advance Movement Implementation Alternatives

25. Each Terrain Grid Point (1 x 1) Has a Concealment Value for Each of Three User-Defined Bands For Each Grid Box Traversed by the Line of Sight (Rather Than Proportional), the Concealment Value Is Multiplied V = V * (1 - C) Each Agent Pair Draws a Random Number for Each Band at the Beginning of the Run Determines the Range at which Detection Occurs from Input Pd Curves Detectability Is Held Constant during the Run Modified by Current Concealment Sensors Are Either Personal (Owner Only) or Broadcast (Owner and All Friends within Range) Target Location Error (TLE) Only Used for Communicating Sensed Agents’ Locations Sensor Modeling

26. Three Primary Alternatives (Represented by RGB) Agents Could Have Multiple Sympathies User Decides Which Colors To Use for Color-Distance Calculations (1, 2 or 3 Dimensions) Affiliation Changes Add or Subtract Color Example: Shot At = +.01 Green, -.01 Blue, -.01 Red Distances and Values Are All Soft Rules Behavior Change Triggers Occur When Sidedness Thresholds Are Exceeded Example: Green > .5 or Red < .5, Read New Behavior = Surrender Alpha (Transparency) of 0.0 Means Agent is Dead Sidedness

27. Example Two-Dimensional Affiliation Me My Unit Friendly Neutral Enemy 255 Blue-ness 0 0 255 Green-ness

28. Ten Alternatives: 1-10 Similar to Sidedness Except “Affiliation” Is Optional Attribute Change Rules Add To or Subtract From an Attribute Example: Shot At = +.01 Att[0], -.01 Att[1], -.03 Att[6]. Distances and Values Are All Soft Rules Behavior Change Triggers Occur When Attribute Thresholds Are Exceeded Example: Att[4] > .5 or Att[6] < .5 Read New Behavior = Follow Leader or Become Suicide Bomber Attributes

29. Know About Leader Leader from Same Unit Found Out About Don’t Know About Leader No Leader from Same Unit Found Out About See Leader Too Many Enemies Too Few Friends Outnumbering by a Ratio Outnumbered by a Ratio Being Influenced One Change for Each Occurrence Influencing At Waypoint At Objective All Is Well Used To Restore Original Conditions Example: No Leader for a While, Then Leader Is Reacquired Side or Attribute Change Events

30. Two General Types (Up To Ten InfluencersPer Agent) Directed Target Requires Line of Sight (LOS) P(hit) and P(influence) Are a Function of Range Examples: Leaflets, Food Only Affects Target Area Effect No LOS Required Accuracy (Hit Vs. Aim) a Function of Range P(influence) a Function of Distance from Hit Point Examples: Bull Horns Affects All Agents within Radius User Chooses Effect Carlton Damage Function Effect (Degrades with Distance), or Cookie Cutter Effect (Constant within Radius) InfluencerTypes Non-Lethal Effects • Suppression • Requires a Hit by Directed Target • Requires Possible Influenceby Area Effect • Causes Agent To Stop All Activity for Input Time • No Movement • No Sensing/Broadcast • No Influencing • Restore/Resurrect • Remove Damage (e.g., Medicine) • Paint Balls • Change the Color/Attribute of Targeted Agent • Absolute, Incremental, or Relative to Agent Firing the Weapon • May Cause Triggers To Execute

31. Fixed Box Size of Up To 4000 X 4000 Pixels User Decides What a Pixel Equates To Terrain Attributes Mobility (Affects Agent Speed: 0-100% = Impassable to No Effect) Concealment (Affects Agent Detection: 0-100% = No Effect to Hidden) Up To Three Detection Bands Protection (Affects Agent Vulnerability: 0-100% = No Effect to Invulnerable) Terrain Feature Height (Enables Looking Down From Above) Additional, User-Defined Polygons Can Be Overlaid Feature Floor & Ceiling (Enables Clouds) Terrain Can Change Agent’s Attributes (v2.0) Abstract Terrain

32. Events That Cause an Agent To Change Behavior: Loss of Leader (No Leader Within d) Red, Green, or Blue Becoming Greater or Less Than v Attributes Becoming Greater or Less Than v Resource X, Resource Y, or Resource Z Becoming Greater or Less Than v% Being Shot At Detecting an Enemy Detecting a Friend Arriving at a Waypoint or Objective Friendly Casualties (Fewer Than x% Known Remaining) Fuel Becoming Greater or Less Than v% Time Step (Absolute and Relative) Trigger Action Reads a New, Complete, Explicitly Named Behavior Template (Including Setting New Triggers) Delay Before Action (Future Capability) Simulates Communication/Spin-up Time Triggers x– Force Strength Threshold, d – Distance Threshold, v –Value

33. Triggers Are User-Prioritized If More Than One Fires, Only One Is Acted Upon Each Trigger Is Associated with a Named Behavior Example: Loss of Leader = ‘Surrender’ When Fired, a Trigger Reads in New Agent Behavior Movement Desires & Terrain Preferences Changes to Other Attributes (Speed, Altitude, Vulnerability, Leadership, etc.) New Waypoints New Current Color or Change Color by Actions New Current Attribute or Change Attribute by Actions New Weapon, Sensor, Comm Possessions New Engagement Desires (Hold-Fire, Weapon Selection Logic, Target Persistence) New Fuel, Resource X, Resource Y, or Resource Z, Settings New Behavior Change Triggers (and Priorities) No Limit To Number of New Behavior Changes (One Per Time Step) Records Time Step, Agent Number and Class Activities When A Trigger Fires

34. Example Behavior Tree Attack If Lose Leader If Take Casualties Initial: Patrol If See Enemy If Lose Leader If Take Casualties If Influenced Surrender Withdraw If Lose Leader Take Cover If Lose Leader If Take Casualties

35. BACKUP

36. Pythagoras USAF February 2008 Edmund J. Bitinas Donna Middleton John McGillvray Pythagoras: Helping the World Look at Problems from the Right Angle