AMS Commission on the Weather and Climate Enterprise Board on Enterprise Communication Ad Hoc Committee on Uncertainty

1. Town Hall AMS Commission on the Weather and Climate EnterpriseBoard on Enterprise CommunicationAd Hoc Committee on Uncertainty in Forecasts(ACUF) January 23, 2008 AMS Annual Meeting

2. Welcome and IntroductionGeorge Frederick, CWCE Commissioner • Providing forecast uncertainty information to Nation • Challenge Enterprise should take on together • CWCE and BEC • Provides forum to bring Enterprise partners together • ACUF not starting from scratch • Good work going on in community • 2006 NRC “Completing the Forecast” Report

3. Welcome and IntroductionMuch Interest in Committee

4. Welcome and Introduction Town Hall Purpose • For Committee to share their initial plans • For you to share your thoughts and comments • Committee will meet later today to digest what they’ve heard and kickoff their work • Those interested welcome to attend

5. Agenda • Remarks (J. Hayes, J. Myers, B. Ryan, P. Stephens) • Motivation (E. Abrams) • Committee overview (P. Hirschberg) • Workgroup 1: National needs, opportunities and benefits (B. Morrow) • Workgroup 2: Enterprise Goals (A. Stewart) • Workgroup 3: Needed Capabilities and Capacities (T. Hamill) • Workgroup 4: Roles and Responsibilities (B. Philips) • Workgroup 5: Roadmap (S. Tracton) • Open discussion • Contacts and additional information (A. Bleistein)

6. Remarks • Jack Hayes, NWS • Joel Myers, President and Founder, Accuweather • Bob Ryan, WNBC-4 in Washington, D.C • Pam Stephens, NSF

7. MotivationElliot Abrams (ACUF Co-Chair, AccuWx)

8. Characterizing and Communicating Uncertainty… To promote better forecast-based decisions To reduce losses caused by weather and climate

9. Are probability forecasts based on meteorological quicksand, rendering products made from them of dubious, uncertain value?

10. Current public forecast probabilities:Provide the percentage chance of measurable precipitation at a fixed location during a specified time period, but- Give no information about how long it will rain- Give no information about how much it will rain-Give nocontingency information (ex: the chance of rain is 50%, but if it does rain, the chance of it lasting more than 30 minutes is 5%)

11. Inspiration: BaseballBatting averages are taken to three decimal places (.334)Statistical verifications are conducted to measure past player performance in order predict future strategies

12. We are assigned a daunting, challenging taskBut we have a large group of interested, smart, engaged and determined people

13. Can we forge a way forward, get our questions addressed, and understand what comes next…and do all of this today?

14. Certainly!Let’s get started!

15. Committee OverviewPaul Hirschberg (ACUF Co-Chair, NWS) • Vision: Enterprise-wide partnership that generates and communicates forecast uncertainty information meeting Nation’s needs for informed decisions: • protecting life and property, • supporting national defense and homeland security, • enhancing the economy, and • meeting specific needs of partners, users, and customers. • Mission: Develop Enterprise-wide goals and roadmap for providing forecast uncertainty information, building off NRC recommendations • Deliverable: An Enterprise implementation (action) plan for forecast uncertainty that has been reviewed and coordinated with partners • Goal: Enterprise partners put into effect

16. Committee Overview Enterprise Plan Components • Needs, opportunities, and benefits of providing hydrometeorological forecast uncertainty products and services to the Nation • Why is uncertainty information important? • Enterprise Goals for forecast uncertainty products and services • Specifics – What will Nation get and how good will it be? • Description of what is needed to meet goals and reach vision • Solutions: What is needed to do this? • Suggested roles and responsibilities of enterprise partners • Who should do what and with who? • Enterprise-wide Roadmap • How will all the pieces fit together?

17. Committee Overview Committee Structure • Organized into Topic Workgroups along major elements of plan deliverable • Work must be done in parallel • Coordination, Communication is key • Looking to use new collaboration technology, e.g., Wikis

18. Topic Workgroup 1:National Needs and BenefitsWorkgroup Composition • Co-leads: Lee Anderson (NWS) and Betty Morrow (FIU) • Members: • Dan Bickford (WSPA – TV) • Julie Demuth (NCAR) • Jayant Deo • Jun Du (NCEP) • Bob Glahn (NWS) • Jenifer Martin (NCAR) • Dan Satterfield (WHNT – TV) • Paul Schultz (NOAA/GSD) • Dick Westergard (Shade Tree Meteorology, LLC)

19. Topic Workgroup 1:National Needs and BenefitsDeliverables • Needs/requirements/opportunities – categorized • Opportunities -- what types/forms of information can be provided? • Needs • End users and decision makers (weather and water) • Public • Emergency managers • Energy Sector • Service Providers • Private Sector • Academia – educational initiatives as proposed by NRC • National Benefits of providing forecast uncertainty information • Construction • DOD • DHS • Aviation • Government

20. Topic Workgroup 1:National Needs and BenefitsMethodology • Review, consolidate and complement existing sources of information: • Obtain an initial information set from various sources (e.g., surveys, research) • Identify major user-groups that use hydrometeorological information (e.g., forecasts) • Use various methods (e.g., surveys, conferences) to obtain users’ needs. • Inform users’ about available hydrometeorological information while obtaining their uncertainty needs. • Collaborate with ACUF sub-groups, and other related groups, to obtain additional requirements, and information resources. • Obtain and develop benefits of using uncertainty in products.

21. Topic Workgroup 2:Enterprise GoalsWorkgroup Composition • Co-leads: John Gaynor (OAR) and Alan Stewart (UGA) • Members • Barbara Brown (NCAR) • Chris Elfring (NAS) • Greg Fishel (WRAL – TV) • Pat Hayes (Northrup Grumman) • Carlie Lawson (Natural Hazards Consulting) • Chris Maier (NWS) • Bernard Meisner (NWS) • David Novak (NWS) • Scott Sandgathe (APL – UW) • Robyn Weeks (The Weather Channel)

22. Topic Workgroup 2:Enterprise GoalsDeliverables • Enterprise Goals for forecast uncertainty products and services (threshold and longer-term) • Outcome Goals • Output Goals • New products and services (collaboration with Subgroup 1) with performance goals • Awareness/Education/Credibility Goals

23. Topic Workgroup 2:Enterprise GoalsOutcome Goals • Lives saved and injuries prevented • Improve economic competitiveness and reduce weather-related costs • Enhance productivity and efficiencies in weather-sensitive industries • Some specific examples: • More efficient surface and air transportation • More effective decisions and timing for evacuations • Decreased impacts of severe weather • More effective decisions on fuel use for power generation • More effective decisions on water resources • Better educated citizenry in understanding and on the use of uncertainty in forecasts

24. Topic Workgroup 2:Enterprise GoalsOutput Goals (Collaboration with Subgroup 1) • By 2015, the routine dissemination and public communication of probabilistic weather and climate forecast products that are: • timely availability of data and information • highly reliable • sharp, to the extent possible • competitive in skill with the best offered worldwide; • available for both routine and high-impact events; • available in user-defined formats and in displays intuitive to all users • with forecast verification information readily accessible.

25. Topic Workgroup 2:Enterprise GoalsAwareness/Education/Credibility Goals • Probabilistic forecasting – standard in undergraduate and graduate education • Continuing education of meteorological community • Potential AMS and NWS certification • Public education, K-12, severe weather awareness talks, web pages • Continual innovation of product suite in response to user needs • User-defined verification routinely available

26. Topic Workgroup 2:Enterprise GoalsMethodology • Develop a vision and set goals as a group • Vet vision and goals • ACUF meetings • Appropriate AMS meetings • Extensive outreach to users and experts • Other coordination efforts

27. Topic Workgroup 3:SolutionsWorkgroup Composition • Co-chairs: Tom Hamill (OAR) and Ross Hoffman (AER) • Members: • Tom Dulong (UCAR) • Paul O. G. Heppner (3SI) • Eddie Holmes (CBM) • David Myrick (NWS) • Carla Roncoli (UGA) • John Schaake (retired NOAA) • Dan Stillman (Inst. for Global Env. Strategies) • Zoltan Toth (NCEP) • Bill Bua (NCEP)

28. Topic Workgroup 3:SolutionsDeliverables • Description of what is needed to meet the goals and reach vision

29. Topic Workgroup 3:Solutions Methodology • Workgroup co-leads interact with ACUF leads and other groups leads to formulate coordinated overall direction. • Subgroup 1, 2 will provide requirements; subgroup 3 will determine implementation strategies; subgroup 4 will suggest who should implement which parts of the solution. • Workgroup co-leads, with help from subgroup members, prepares section material. As needed, workgroup members interact with broader community to flesh out details. Regular telecons for coordination. • Existing examples analyzed–what works, what doesn’t • Workgroup’s material is synthesized by ACUF co-leads and workgroup leads. • Workgroup also vets ACUF reports. • Overall roadmap is critiqued by broader community. • Iterate based on feedback from enterprise members.

30. Topic Workgroup 4:Roles and ResponsibilitiesWorkgroup Composition • Co-chairs: John Sokich (NWS) and Brenda Philips (UMASS) • Members: • Matthew Biddle (OU) • David Bright (SPC) • Gordon Brooks (AFWA) • Gina Eosco (Cornell U) • John Ferree (NWS) • Jim Hansen (NRL) • Paul Nutter (Univ. No. CO) • Dan O’Hair (OU)

31. Topic Workgroup 4: Roles and ResponsibilitiesDeliverables • List of all Enterprise Partners with contact info and relevant areas of expertise for use by sub-groups • Compilation of ACUF Best Practices • Best practices for cross-enterprise collaboration • Impediments for collaboration • Implementation roles/responsibilities of enterprise members • Advocacy plan • After ACUF – how to move forward

32. Topic Workgroup 4:Roles and ResponsibilitiesMethodology • Information gathering • Town Hall meeting at AMS • Other Weather and Climate Enterprise meetings • Weather and Climate Enterprise database ? • Interviews with selected ACUF enterprise members for case studies/testbeds • Literature/policy review (eg: NRC report “Crossing the Valley of Death,” “Fair Weather”)

33. Topic Workgroup 5:RoadmapWorkgroup Composition • Co-chairs: Steve Tracton (retired NOAA) and Neil Stuart (NWS) • Members: • Jon Ahlquist (FSU) • Peter Browning (NWS) • Dan C. Collins (NCEP) • Chuck Doswell (OU) • Mary Erickson (NOS) • Rebecca Morss (UCAR) • Leonard Smith (London School of Economics) • Bernadette Woods (WJZ – TV)

34. Topic Workgroup 5:RoadmapDeliverables • Enterprise-Wide, “Strategic Roadmap” (including alternatives) • THE primary overall ACUF deliverable • “End-to-End” milestones and timelines • Relevant, effective and meaningful metrics of progress

35. Topic Workgroup 5:RoadmapMethodology • Integrate input from subgroups in context of appropriate strategic planning and roadmapping templates • Pro-active interaction/coordination between subgroups • and outreach to Enterprise members and constituents (push and pull perspectives; achieve consensus) • “Strategic Roadmapping” • Link strategies to realistically achievable action plans (“Threshold Objectives) • Re-evaluate and re-visit pathway as warranted (not static) • “Off ramps” to actions/implementations (“low-hanging fruit”) • “On ramps” to accommodate newly identified opportunities, strategies, etc.“

36. Topic Workgroup 5:RoadmapMethodology (cont’d) • “Critical Path Analysis” • Temporal ordering of sequential and parallel activities • represent stages of project co-dependencies • Identify critical activities (“predecessors”) required before other action can continue • Identify “float” activities; can be extended or delayed without jeopardizing project • Identify “what if” scenarios and linkages Start SIMPLISTIC SCHEMATIC End Transformation from traditional to ensemble based forecast system where uncertainty is assessed, propagated and conveyed throughout entire forecast process (“End to End”)

37. ACUFTimeline • Finalize Committee Charter; identify workgroup leads and memberships (Nov. 07) • Develop draft Committee work plan including workgroup deliverables, methodologies, tasks, timelines, and resource needs (Dec. 07) • Discuss and review work plan at AMS Annual Meeting – Town Hall and Full Committee Meeting (Jan. 08) • Finalize Committee work plan (Feb. 08) • Workgroups meet as necessary accomplish tasks (Jan.–Aug. 08) • Full Committee review of preliminary workgroup deliverables via briefings (Sept. 08) • First draft of Workgroup Sections (Oct. 08) • First draft of complete Enterprise Implementation Plan (Dec. 08) • Full Committee review of first draft Enterprise Implementation Plan - at AMS Annual Meeting (Jan. 09) • Stakeholder review and comment of draft Implementation Plan (Feb.-May 09) • Final draft of Enterprise Implementation Plan (Sept. 09)

38. Open Discussion ACUF Town Hall

39. ACUF Contact and Information • Please contact Andrea Bleistein andrea.bleistein@noaa.gov • ACUF Town Hall slides are posted at: http://www.ametsoc.org/dl • Full Committee Meeting Today – all welcome • 4:30 – 5:30 Hilton Riverside, Rosedown Room