Analytical Tradecraft

1. Analytical Tradecraft “ Solid intelligence on terrorism is not easy to develop… I would like to salute the unsung heroes of the struggle against terrorism. These heroes are the intelligence analysts. Often they have little to go on: a photograph, a fragment of an overheard conversation, the text of a communiqué, the summary of a meeting, a used airline ticket. Sometimes, it is like piecing together a gigantic jigsaw puzzle, but it is a puzzle that can save lives.” George Shultz Former Secretary of State

2. ANALYSTS The Challenge Analyst Work Environment

3. Velocity Volume IMINT HUMINT SIGINT IC Collection Sources MASINT Variety (@ Signal / Media Level) OSINT Volatility Expanding World of Data Expanding World of Data Variability (@ Content Level)

4. S E N S O R P H Y S I C A L D A T A O B J ECTS “Information Spectrum” B I T S S I GNALS • Types of • Intelligence • Current • Estimative • Warning • Scientific & • Technical • Research Intel Report IC Customers … INT Part of the “INTELLIGENCE CYCLE Processing Reporting Collection Analysis Raw Data Intelligence Turning Data into Intelligence IM … SIG … HUM … MAS … OS ... Single Discipline Analysis All Source Analysis Information

5. Events / Activities / Processes Relationships Analytic Knowledge Information Retrieval Content Data Mark-up Assessment & Interpretation Properties / Attributes Presentation & Visualization Time / Temporal Issues Location / Spatial Issues Reporting & Dissemination Data Filtering & Selection Instances Content Data Transformation Synthesis& Fusion IC Analysts Humans / Organizations Physical Objects Information Understanding Information Discovery Expanding World of Data Data  Information  IntelligenceSynthesis & Fusion of Observables within a Given Context “Observables” Who, What, When, Where, How Time / Temporal Issues Location / Spatial Issues

6. Intentions Motivation Attitudes / Perspectives Meaning Analytic Knowledge Information Retrieval Content Data Mark-up Assessment & Interpretation Presentation & Visualization Time / Temporal Issues Location / Spatial Issues Values / Beliefs Goals / Objectives Reporting & Dissemination Data Filtering & Selection Content Data Transformation Synthesis& Fusion IC Analysts Humans / Organizations Physical Objects Information Understanding Information Discovery Expanding World of Data Data  Information  Intelligence Assessments, Interpretations, Judgments & Predictions Location / Spatial Issues Time / Temporal Issues “Behavioral Factors” Why? Location / Spatial Issues

7. Active Cross- Fertilization Analytic Knowledge Information Retrieval Content Data Mark-up Assessment & Interpretation Presentation & Visualization Reporting & Dissemination Data Filtering & Selection Content Data Transformation Synthesis& Fusion IC Analysts Strong Interaction Information Understanding Information Discovery Location / Spatial Issues Time / Temporal Issues Humans / Organizations Physical Objects Breadth of Information ExploitationApplied across all of the “INTs” Data  Information  Intelligence “Observables” & “Behavioral Factors” are NOT Independent Activities “Observables” “Behavioral Factors”

8. Analytic Knowledge Information Retrieval Content Data Mark-up Assessment & Interpretation Presentation & Visualization Time / Temporal Issues Location / Spatial Issues Reporting & Dissemination Data Filtering & Selection Content Data Transformation Synthesis& Fusion IC Analysts Humans / Organizations Physical Objects Information Understanding Information Discovery Expanding World of Data Data  Information  Intelligence Combined “Observables” & “Behavioral Factors” Events / Activities / Processes Observables Relationships Time / Temporal Issues Properties / Attributes Goals / Objectives Values / Beliefs Instances Attitudes / Perspectives Meaning Intentions Motivation Behavioral Factors Location / Spatial Issues

9. IC Analysts Taking a Closer Look at IC Analysts

10. IC Analysts Universal Similarities do ExistIntelligence Community Analysts • They are far more than just casual users of information • They work in an information rich environment where they have access to large quantities of heterogeneous data • They are almost always subject matter experts within their assigned task areas • They track and follow a given event, scenario, problem, or situation for an extended period of time • They frequently have extensive collaboration with other analysts • They are focused on their assigned task or mission and will do whatever it takes to accomplish it • The end product that results from their analysis is often judged against the standards of: Timeliness Accuracy Usability Completeness Relevance

11. Major Differences Do Exist(between agencies and within agencies) • Between single source and all source analysts (data formats, degree of closeness to the raw information, accessibility to contextual information) • Between analytical domains (counter terrorism, WMD, regional analysis, weapons systems, etc) • Between types of intelligence produced (e.g. current intelligence, estimative intelligence, etc.) • Experience level in the art of analysis and ability to understand the analytic process • Understanding and use of analytic networks • Customer requirements (strategic, operational, tactical) • Unique individual differences

12. Intelligence Analyst’s Operational Context

13. Sample of an Analytic Work Flow Collaborate with Colleagues Library Research Tasking Thinking Completed Report Copy/Paste Annotate Internet Search Compose/Write … Compose/Write Revisions Time

14. Fusion of Multilayer Analysis Personal Relationships Organizational Relationships Technical Processes/Flows Transportation Networks Currency/Financial Transfers Electronic Connectivity/ Information Flows

15. Collector 5 picks up event planning material in a raid Collector 4 observe event-related info & reports Collector 1 observes event planning & reports Collectors 1,2,3 observe event Collector 1 reports Collector 2 reports Collector 3 reports H-n H+1 H-1 H-n Event Event Planning Aftermath of the Event H Hour The Challenge of Time in Analysis • Different sources do not report simultaneously on an event. • Data from different sources may be near real-time or take years to arrive. • The hypothesis of today may be thrown out by new data arriving next week. • Data must be visualized over time as patterns which change in time as updates occur

16. The Challenge of Credibility in Analysis What do we look for in a source? • Credibility • Reliability • Relevance • Can be confirmed What standards are we held to in reporting • Accurate • Timely • Actionable • Complete • Relevant

17. Where Do We Go From Here?

18. THE SOCIAL SCIENCES ARE, IN FACT, THE “HARD” SCIENCES. GROWING ARTIFICIAL SOCIETIES EPSTEIN AND AXTELL

19. “NOBODY KNOWS” QUESTIONS WHAT IS GOING ON INSIDE THE IRAQI/SERB/NORTH KOREAN REGIMES? WHAT WOULD COLLAPSE OF THESE REGIMES LOOK LIKE? HOW STIFF A RESISTANCE WILL THE FEDAYEEN AND OTHER SADDAAM SUPPORTERS PUT UP? WHAT IS GOING ON IN ANY COUNTRY, AND WHERE IS IT GOING? WHAT WILL THE MIDEAST LOOK LIKE IN 2010? WHAT ARE THE ROOTS OF TERRORISM & HOW CAN WE AFFECT THOSE ROOTS

20. CHARACTERISTICS OF “NOBODY KNOWS” QUESTIONS • COMPLEXITY • MANY INDEPENDENT ACTORS • MULTIPLE VARIABLES • DYNAMIC/ADAPTIVE BEHAVIOR • EMERGENT OUTCOMES • HUMAN BEHAVIOR • INDIVIDUAL AND GROUP PROCESSES

21. NEW APPROACHES INFORMATION INTELLIGENCE KNOWLEDGE UNDERSTANDING TO MOVE FROM PROVIDING INTELLIGENCE TO PROVIDING UNDERSTANDING NEW APPROACHES TRADITIONAL ANALYSIS FACTS DATA PROCESSES/ OUTCOMES CONNECTIONS/ RELATIONSHIPS

22. Potential New Approaches • NATURAL EXPERIMENTATION • COMPLEXITY SCIENCE • MODELING AND SIMULATION • WHAT ELSE?

23. SOME OTHER IDEAS • EXPERIMENT WITH AND INTEGRATE ONGOING • EFFORTS (IC Test Nets) • PUSH THE SCIENCE - PARTNER WITH NSF AND DARPA • INCREASE FUNDING FOR EXPLORATORY PROGRAMS • SMALL GROUP MODELS • SOCIETAL MODELS • SUBSIDIZE SELECT WARGAME DEVELOPERS (EXISTING GAME ENHANCEMENTS) • “LIBRARY” OR CROSS-REFERENCE FOR • INTELLIGENCE RELATED SIMULATIONS

24. ANALYSIS AND COMPLEXITY “THE RULES OF THE GAME: LEARN EVERYTHING, READ EVERYTHING, INQUIRE INTO EVERYTHING… WHEN TWO TEXTS, OR TWO ASSERTIONS, OR PERHAPS TWO IDEAS, ARE IN CONTRADICTION, BE READY TO RECONCILE THEM RATHER THAN CANCEL ONE BY THE OTHER; REGARD THEM AS TWO DIFFERENT FACETS, OR TWO SUCCESSIVE STAGES, OF THE SAME REALITY, A REALITY CONVINCINGLY HUMAN JUST BECAUSE IT IS COMPLEX.” Marguerite Yourcenar, Memoirs of Hadrian

25. Non-Linear Dynamics of Human Behavior (NDHB) Advanced R&D Program • Can we achieve a better understanding of Human Dynamics; Individual and Small Group Behavior; Leadership Decision Making; Large Group Dynamics? Can we model it? • Can we use modeling approaches to influence the present or to forecast potential future events/activities? What approaches are best for which intelligence problems? • Can we use models and simulations for knowledge discovery? • Can we model across missing data? • Can we use models and simulations as a method for training analysts in hypothesis generation and argumentation?

26. POSSIBLE OPERATIONAL PROBLEMS TO BE SOLVED IN MODELING HUMAN BEHAVIOR • Anticipating Surprise – Asymmetric threats & tactics • Avoiding technology surprise (novel ways of using existing technology and use of emerging technologies) • Anticipating political instability (including ethnic strife and state collapse) • Identifying plans & Intentions for threat operations against national US interests • Conventional military operations • Terrorism & other acts of violence • Information operations • WMD/CBNRE • Anticipating attacks designed to disrupt the economy

27. POSSIBLE OPERATIONAL PROBLEMS TO BE SOLVED IN MODELING HUMAN BEHAVIOR (cont.) • Leadership Analysis/Decision-making (both state and non-state) • Formation/Transformation (regime change/succession/etc) • Coalition Dynamics • Influences (external/internal) • Plans/Intentions/Policies/Strategies • Environmental Issues • Spread of diseases or C/B agents through human interactions and their associated social impact • Spread of diseases through livestock and plants and impact on society (e.g. associated economic upheavals, starvation) • Environmental disasters (either man-made or natural) and impact on society • Ethnic/Cultural/Religious/ Societal constraints on courses of action and decision-making (US, Allies, Neutrals, Enemies)

28. Possible Applications for Modeling Human Behavior • Development of Threat Models (using various modeling applications) and analysis of results from those models • Fusion of intelligence from different disciplines/domains/experts with support from modeling and simulation • Integration of modeling with traditional analytic methods. • Training analysts to use models as part of their cognitive toolset • Understanding what approaches work best with different intelligence problems • Comparative case studies • Definitions of modeling approaches • Development of new analytic strategies • Discovery of previously unknown data/patterns • “what if” • Modeling missing data and uncertainty • Comparative analysis • Anticipating surprise • Development of new patterns/trends • Comparative case studies • What else?

29. Possible Technical Needs for Modeling Human Behavior Some Modeling Approaches Individual Small Large Group Group Agent-Based Modeling System Dynamics Reaction-Diffusion Social Network Analysis Game Theory Multiscale Analysis Coupled Oscillators Neural Nets What else?

30. Possible Technical Needs for Modeling Human Behavior Cross-cutting technologies • Natural Language Processing • Semi-automated and automated data loading of the model(s) • Integration of modeling approaches • Intuitive visualization of modeling data • Tools to manage and analyze modeling data • Library of models (how each model works) • Architecture that permits sharing • Hypothesis Generation tools • Collaborative wargaming tools that can retain fidelity of data

31. CENTER FOR COMPLEX INTELLIGENCE ISSUES (CCII) ? ? ? COMPLEX ISSUES ANALYTIC CORE GROUP TOOLS GROUP ACADEMIC SUPPORT GROUP CONCEPTS APPROACHES TOOLS ANSWERS CONSUMERS

32. HUMAN ANALYSIS “...THE REPRESENTATION OF HUMAN CHARACTER AND PERSONALITY REMAINS ALWAYS THE SUPREME LITERARY VALUE, WHETHER IN DRAMA, LYRIC, OR NARRATIVE.” BLOOM: SHAKESPEARE: THE INVENTION OF THE HUMAN

33. BACK UP SLIDES

34. 1 Intelligence Requirements 2 Planning & Direction 7 Dissemination 6 Reporting 5 Analysis Intelligence Community & The Intelligence Cycle 3 Collection 4 Processing

35. Known Unknown You Know What You Know You Know What You Don’t Know You Know You Don’t Know What You Should Know You Don’t Know What Can Be Known You Don’t Know Sources of Novel Intelligence Information Sources Analytic Knowledge