www.aptima.com Boston ▪ DC ▪ Dayton

1. www.aptima.com Boston ▪ DC ▪ Dayton © 2008, Aptima, Inc. Improving the Display of River and Flash Flood PredictionsPresentation to DOH Workshop, Silver Spring Lawrence Wolpert, Tom Aten, Bob McCormack July 17, 2008 Ó2008, Aptima, Inc.

2. Agenda • Introductions • Short Aptima Overview • Flood Prediction • Task 1: Knowledge Acquisition • Task 2: Uncertainty Modeling • Task 3: Visual Display Design • Future Work Ó2008, Aptima, Inc.

3. Aptima, Inc. Founded in 1995; consistent annual growth (43% CAGR) Serving government and commercial clients 300+ contracts with the DoD Offices Woburn, MA, (HQ) Washington, DC Dayton, OH Ft Walton Beach, FL • Interdisciplinary Small Business • 100+ staff (80% graduate degrees) Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 3

4. Human–Centered Engineering Social & Organizational Structures Technology Capabilities Human Agents Congruence or ... Disruption Mission, Tasks & Work Processes Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 4

5. Optimizing Performance in Complex Environments Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 5

6. Flood Prediction: The Challenge Weather and flood predictions are filled with uncertainty, which can be difficult to convey Misconceptions about probabilistic events “They say we had a 500 year flood last spring. At least that will never happen again in my lifetime.” Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 6

7. The Challenge - 2 Weather and flood predictions are filled with uncertainty, which can be difficult to convey • Uncertainty make decision-making more difficult • What would you do if: • You lived along the NC coast? • You had an important business meeting in New Orleans? • You were planning an outdoor party? Ó2008, Aptima, Inc.

8. Flood Prediction Grant - Summary Contract #: NA07NWS4620015 Customer: NWS Client Team: Pedro Restrepo Mary Mullusky Donna Page Contract Duration: 10/05/07 – 05/31/08 Contract Value: $60,748 Knowledge Elicitation Examine Current Capabilities Uncertainty Models Visualizations Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 8

9. Task 1: Knowledge Acquisition through Interviews and Group Discussions • Participants • 2 Hydrologists • 6 Emergency Managers • 2 Red Cross Disaster Coordinators • Flood Evacuation Shelter Operator • Dam Operator • 2 Fly Fishermen • 3 Kayakers • TV Weather Reporter • 6 NWS Employees Interviews conducted either in person or by phone Ó2008, Aptima, Inc.

10. Task 2:Uncertainty Model • Purpose • Identify the sources/types of uncertainty related to predicting, communicating, and acting upon flood events. • Identify ways to factor uncertainty estimates into the decision-making processes of each community of interest to improve overall quality and efficiency of outcomes. • Determine what each community of interest needs to know about the uncertainty that exists to assess potential risks and choose a course of action. • Method • Analytical investigation and integration of uncertainty research from multiple disciplines. • Probability theory • Categorization theory • Perception theory • Deliverables • Description of the model of uncertainty and how it can be used to enhance decision-making. Ó2008, Aptima, Inc.

11. Theoretical Models of Uncertainty • Probabilistic model of uncertainty • signal processing, information theory • “model predictive uncertainty” • Categorization model of uncertainty • set theory, fuzzy logic • “meaning, implication, knowledge organization uncertainty” • Perception model of uncertainty • human psychology, complexity theory, AI • “semantic uncertainty”, perceptual filters, information presentation vs. expectation mismatch Ó2008, Aptima, Inc.

12. Likelihood vs. Confidence Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 12

13. Task 3: Visual Display Design Support EMs’ decision making regarding courses of action. Visualizations of river forecasts with uncertainty Visualizations of “impact” of various flooding scenarios Visualization of uncertainty model inputs for clear understanding of “why” predictions are what they are. Allow for local knowledge to be incorporated into predictions Fusion / organization of information for clear understanding of relationship to one another and enhanced orientation. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 13 13

14. Landmark Map Ever present geographic depiction of current state of affairs Enhanced ability to select desired information for presentation on the map Increased saliency/access to relevant related information needed for accurate SA and decision making Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 14 14

15. River Height/Flow Prediction with Uncertainty Visual depiction of “river forecast” in terms of height / flow Clear depiction of uncertainty with cone of uncertainty Visualization of model parameters with ability of the EM to apply local knowledge. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 15 15

16. River Extent Prediction Visualizations Visualization of river extent for given point selected in the river prediction of flow/height Stage, Likelihood and Confidence of a selected point Impact of given scenario shown (items affected) Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 16 16

17. Interplay of Visualizations Enhanced Fusion / organization of information for enhanced decision making Clear understanding of relationship Easy access to needed information for decision making Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 17 17

18. Further Detail Regarding Flood’s Impact Provide further understanding of implications of flood at a: Global level Individual variable level Current and Future river impact predictions shown Additional understanding provided through references to similar previous floods Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 18 18

19. Summary • Elicited knowledge from multiple stakeholders • job roles, responsibilities, and information needs • Formulated a comprehensive theory of uncertainty • Environmental/weather • NWS models • Users • Designed visualization techniques • to maximize the comprehension and utility of river and flash flood data for Emergency Managers • Developed prototype displays • demonstrated these techniques in screen shots Ó2008, Aptima, Inc.

20. Contact information • Lawrence Wolpert (PM) lwolpert@aptima.com 617.913.5399 • Bob McCormack (PI) rmccormack@aptima.com 781.496.2476 • Tom Aten taten@aptima.com 202.552.6123 Ó2008, Aptima, Inc.

21. Ó2008, Aptima, Inc. Ó2008, Aptima, Inc. 21