Basic/Futures Research FODAVA and Homeland Security

1. Basic/Futures ResearchFODAVA and Homeland Security Joseph Kielman Director, Basic and Futures Research Command, Control and Interoperability Division and Manager, Center of Excellence for Command, Control, and Interoperability Office of University Programs Science and Technology Directorate joseph.kielman@dhs.gov 202-254-5787 December 3, 2009

2. Command, Control and Interoperability CNR Annual Program Status Review 8 February 2005 Mission Through a practitioner-driven approach, the Command, Control and Interoperability Division (CCI) creates and deploys information resources to enable seamless and secure interactions among homeland security stakeholders Vision Stakeholders have comprehensive, real-time, and relevant information to create and maintain a secure and safe Nation

3. CCI Mission Areas and Challenges Information analysis Knowledge management Threat assessment Situational awareness Decision support Information sharing Interoperable communications Surveillance and investigative operations Cyberinfrastructure protection

4. Digital Data – the WHY • In 2002, recorded media and electronic information flows generated about 22 exabytes (1018) of information • In 2006, the amount of digital information created, captured, and replicated was 161 EB • In 2010, the amount of information added annually to the digital universe will be about 988 EB (almost 1 ZB) • About 2 GB of digital information is being produced per person per year • 95% of the Digital Universe’s information is unstructured • 25% of the digital information produced by 2010 will be images • 30% will be produced by organizations

5. The Homeland Security Environment – the WHO Department of Homeland Security 22 components, 7 operational agencies 215,000 personnel Federal Government Partners 15 Intelligence Community agencies 11 Law Enforcement Community agencies 350,000 personnel Tribal, Local, and State Partners 80,000 public safety, public health, emergency response, law enforcement agencies 750,000 homeland security practitioners

6. New Dimensions of Homeland Security – the WHAT Time Scale – short to long-lived incident to development to crisis attack to epidemic to global change Consequence - limited to extensive local to regional to national to global human to group to society Action – insight to decision preparation to prevention to response to remediation prediction to anticipation to warning to detection

7. The Information Challenge – the HOW • CCI is developing a toolkit - the tools, technologies, knowledge, and resources to enable Homeland Security practitioners to acquire, manage, analyze, share, and secure information • Each of CCI’s thrust areas plays an important role in creating this toolkit

8. Basic/Futures Research • Cluster and Thematic views • Evidence Evaluation • Triage Networks • Visual analysis for multiple languages • Multi-Viewpoint Support • Affect and Emotion Measures • Correlation Analysis • Streaming Data • Collaborative Team Analytics • Conducts research on information and intelligent systems to address problems associated with synthesizing information and deriving insight from massive, dynamic, ambiguous, and diffuse data sets • The objective is support for an effective risk management approach to homeland security based on comprehensive, timely threat awareness and decision making informed by accurate consequence analyses

9. Research Interests (1 of 2) Dynamic, on-Demand Data Processing and Visualization: Capability for real-time management, analysis, and visualization of selected data in multiple forms and from multiple, diverse sources. These techniques would automatically select, rank, and correlate only those data relevant for purpose-driven decision-making. Hypothesis-driven Analysis: This capability would include three elements: automated retrospective analysis of collected or extant data using pre-selected hypotheses; automated generation of alternative hypotheses by constant updating of data; and prospective analysis of potential risks and threats using data-derived hypotheses. Visualization of Structured, Unstructured, and Streaming Data: Capability for integrated visual analysis of free text, database records, audio, video, imagery, transactional data, geographical data, and sensor information. The focus on this effort is twofold: development of a single, scalable framework for visual analytics and establishment and validation of reliable performance metrics for visual processing of data. Mathematics of Discrete and Visual Analytics: Development of the mathematical foundations for discrete processing and simulation and for visual analytics. This will provide a rigorous scientific basis for future algorithm development.

10. Research Interests (2 of 2) Scalable Filtering and Dissemination: Techniques for secure, privacy-aware identification and dissemination of information among international, federal, state, tribal, and local agencies. This includes advanced methods, processes, and procedures that ensure sharing of information for immediate decision-making by multiple partners under a range of technical, political, and organizational parameters Visualization and Simulation of Data: Application of visualization techniques, discrete mathematics methods, and game theory to diverse information, including development of new approaches to simulating multiple threats or disasters. Mobile and Light-Weight Information Analytics and Sharing: Information discovery, dissemination, and decision-making tools capable of being tailored for diverse homeland security applications and software architectures. These techniques need to focus on a range law enforcement, public safety, public health, and emergency response applications.

11. Basic/Futures Research Program Areas • Visual Analytics, Precision Information Environments Visually based mathematical methods and computational algorithms for discovering, comprehending, and manipulating diverse data and applying the resulting knowledge to anticipate terrorist incidents or catastrophic events and guide response and recovery activities • Data-intensive Computing, Privacy and Forensics Simpler, more efficient software algorithms and hardware architectures for extracting and managing data, assessing threats and consequences, ensuring information privacy, securing the cyber infrastructure, and ensuring telecommunications interoperability • HIGHLIGHTS • Canada-USA, UK-USA, and German-USA Collaborations - Collaborative Activity Agreements (CAA) under existing treaties between DHS S&T and Defense Research and Development Canada, UK Home Office, and German BMBF • Visualization and Analytics Complex – The National Visualization and Analytics Center (NVAC), the new CCI Center of Excellence, Joint NSF-DHS Foundations of Data and Visual Analytics (FODAVA) Program, and 10 industry partners • National Research and Development Agenda

12. Firm 5 Firm 6 Firm 4 Firm 7 Firm 3 Country A Firm 8 Firm 2 Firm 9 Firm 1 Firm 10 A Bank Technologies and Tools

13. Analysis - Retrospective Prediction Visual Analytics Synthesis Graph Matching Cognition Analysis Pattern Analysis Evidence Extraction Organization Content Management Link Discovery Integration Connect the Dots Extraction Aggregation Data Information Knowledge

14. Predictive, Prospective Analytics Context, Culture, Genes Policy Cognitive/Behavioral Analytics Visual Analytics Prediction Synthesis Graph Matching Cognition Analysis Evidence Extraction Pattern Analysis Content Management Organization Link Discovery Integration Connect the Dots Extraction Aggregation Discovery Data Information Knowledge Wisdom

15. The Top Ten – Framing the Discussion Conditions Untethered to Device/Network/Interaction Tethered to Data/Information Indefinite or Indeterminate Data Minimized Transaction Costs Trust Components Challenges Research Foundations Applications

16. The Top Ten Applications Synthetic Materials Disease Development Economic Stability Climate Change Environmental Stability “Slow” Catastrophes Genetics and Epigenetics Threat (All-Hazards) Awareness Human Learning Financial Policy

17. The Top Ten Challenges Dynamic, on-Demand Data Processing and Visualization Hypothesis-driven Analysis Structured, Unstructured, and Streaming Data Mathematics of Discrete and Visual Analysis Scalable Filtering and Dissemination Common Modes for Models, Simulations, and Real-World Data Mobile and Light-Weight Information Analytics and Sharing Provenance, Privacy, and Security Synthetic Environments and Data Purpose-driven Interaction

18. The Top Ten Research Foundations Cognitive and Perceptual Science Data Transformation and Representation Dimensionality Reduction Smart Query Machine Learning Computing and Digital Architectures Games and Synthetic Environments Risk Evolutionary Biology Cultural Anthropology

19. Principles • Information -There are real-world problems for which the availability of information is problematic • Utility -The capabilities being developed must address at least four information-related issues • Outcome -The improvements being provided must have real-world outcomes or implications

20. The Principle of Information • The problem (or need) is actually two-fold: increasing amounts of data along with increasing diversity (types, sources, modes) of data • This is compounded by two additional factors: multiple, diffuse users* and the need for real-time analysis and decision-making • In the future the indeterminate nature of the data, whose relevance is uncertain, transitory, or yet to be determined, will present more significant challenges • Homeland security involves a multiplicity of potential threats: for instance, terrorist attacks, natural disasters, pandemics, and border security *80,000 law enforcement, public safety, public health, or emergency response agencies forming the local, state, tribal, and federal homeland security environment

21. The Principle of Utility • Analyze single types of data in a more immediate or real-time fashion – that is, address scalability • Integrate multiple types of data – that is, address heterogeneity • Combine streaming data or sensor measurements, information and intelligence, models, and analyses into a comprehensive assessment – that is, deal with complexity • Create dynamic analyses of situations or incidents, which are simultaneously available in the field or in the laboratory, on an analysts’ desktops, in command centers or situation rooms, and to decision- or policy-makers – that is, enable decision-making on multiple levels

22. The Principle of Outcome • Understanding threats in sufficient time or to a sufficient degree to prepare or respond • Predict the scale or scope of individual homeland security problems, and the associated contributing or cascading secondary effects • More accurately gauge the immediate as well as lingering, that is, the short-term and longer term, consequences of threat agents and develop effective response and remediation technologies and strategies • Address complex issues or problems, such as climate change, economic stability, resource constraints, and population dynamics, which are interrelated and will increasingly affect our world in the future

23. Virtual USA (vUSA) • Support tactical, strategic and planning decisions for homeland security practitioners by enabling the exchange of information in whatever form is needed, delivered to whatever device is available • Seek to unify activities related to the gathering, analyzing, managing, sharing, and protecting of information • Create a way to organize efforts at multiple levels (local, state, tribal, national) • Advance application- and platform-agnostic technologies, standards, governance models, and other tools

24. Organizing Efforts at the National Level

25. Virtual USA Lead Efforts • Virtual City: A pilot demonstration as part of a national program that integrates multiple data sources and makes them more accessible for responders with and other city officials. • Virtual State: Builds on existing investments in Virtual Alabama with advanced visualization, video streaming and other technologies. • Regional Operations Platform Project: Expands the Virtual Alabama concept to a multi-state regional level and integrates existing platforms, enhanced visualization tools, and other data sets such as hurricane data to allow state systems to interoperate and exchange data with each other regardless of the particular platform or application in use.

26. New VAC

27. from Retrospective Analysis … … to Prospective Analytics

28. Data Deluge The man who acquires an encyclopedia does not thereby acquire every line, every paragraph, every page, and every Illustration; he acquires the possibility of becoming familiar with one andanother of those things. If that is the case with a concrete, and relatively simple, entity, then what must happen with a thing which is abstract and variable – ondoyant et divers? Jorge Luis Borges Shakespeare’s Memory

29. The Library of Babel

30. The Future World – Part 1 • Necessary and Sufficient Conditions for Interaction: • No personal computers; no WIMPS; that is, single-layer or flat interactions prevail • Access to tailored information is complete on public screens or on individual displays • Communications, information networks automatically, dynamically establish and maintain communities of interest • Interaction with information, data is mediated not by bandwidth but by cognitive properties • When I do search for information, that search is enabled by images, graphs, multimodal forms Knowledge acquired from the changeable and perishable objects of sense-perception is subject to doubt and error St. Bonaventure

31. The Future World – Part 2 • Necessary and Sufficient Conditions for Discovery: • data from all disciplines is available/accessible and indexed • Processes, controls are understood at a basic level (mature paradigm) • Modeling, simulation, gaming environments are more finely grained • Interactions become, replace experiments • Tailored collaborations are dynamically established, resourced Knowledge – the agent intellect provides the light Thomas Aquinas

32. The Future World – Part 3 • Necessary and Sufficient Conditions for Precision Information Environments: • Synthetic world and real world co-exist, are co-equal • Physical and information spaces are co-existent • Modeling and simulation capabilities apply to both • Streaming, dynamic data are available, accessible in real-world tiem • Environment is tailored, suited for multiple user groups on a dynamic basis • Information spaces are multimodal • Multiple users and manifold uses are managed simultaneously Knowledge involves the presence in the mind of a representation of its objects Duns Scotus

33. Research Futures: T-Square Model Threat Technology

34. VAC Map A Partnership with Academia,Industry, Government Laboratories Canada Alaska RVACUniversity of Washington PacificRim RVACPenn. State Drexel University NY/NJ Port AuthorityEmergency Op Center Hawaii IDS-UACUniv. of Illinois IVAC IDS-UACUniversity of Pittsburgh Europe Australia RVACStanfordUniversity Scholars IDS-UAC, Rutgers Univ. GVACs RVACPurdue University Indiana Univ.Schoolof Medicine Consortium DHS IDS-UACUniversity ofSouthern California New Zealand NSF RVACUniv. of North Carolina Charlotte, Georgia Tech Bank of America Detecting the Expected -- Discovering the UnexpectedTM