Communicating Weather Forecast Uncertainty Information

1. Communicating Weather Forecast Uncertainty Information Andrea Bleistein and Julie Demuth Summer WAS*IS, July 18, 2007

2. The forecast high temperature for Boulder tomorrow is 95ºF. What do you think the actual high temp will be? 95ºF 94-96ºF 93-97ºF 90-100ºF 85-105ºF

3. All the choices listed below are the same as a probability of precipitation of 20%. Do you like this information given as… a) Chance of precipitation tomorrow is 20%. b) There is a 1 in 5 chance of precipitation tomorrow. c) The odds are 1 to 4 that it will rain tomorrow. d) There is a slight chance of rain tomorrow.

4. Definitions • Definition of uncertainty • NRC • Wikipedia • NOAA group definition

5. Outline • Motivation • NWS’s related efforts • NCAR’s related efforts • Discussion session • Other resources

6. Motivation • Atmosphere is nonlinear, chaotic, and complex  Forecast uncertainty is inevitable! • First public forecasts in the modern weather forecasting era were called “probabilities” • But forecasting generally evolved into more deterministic products * * Single future state of a system or single-value forecast

7. Motivation (cont) • Yet, most users understand that forecasts are imperfect • By communicating uncertainty information, we can • avoid misrepresenting the capabilities of weather prediction science • better convey what meteorologists know • help users make more informed decisions and avoid problems

8. An Example…

9. Community Interest and Action • American Meteorological Society (2002) statement • endorsing probability forecasts and recommending an increase in use • National Research Council (2006) report commissioned by the National Weather Service • National Research Council (2003) report on communicating uncertainties in weather and climate forecasts

10. NWS’s Related Efforts

11. NWS Forecast Uncertainty Steering Team • How to corporately address need/opportunity to improve generation and communication of forecast uncertainty products and services • How to address and respond to 2006 NRC Report, “Completing the Forecast. Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts” • In Jan 07, informal group began looking at forecast uncertainty from corporate perspective • Purpose not to hinder or duplicate ongoing efforts, but rather to corporately plan for success by ensuring necessary components of an “end-to-end” forecast uncertainty information system are accounted for

12. NWS Forecast Uncertainty Steering Team • Vision • NWS provides forecast uncertainty products, services, and information meeting customer, partner, and forecaster needs • Mission • Advise and coordinate NWS activities related to development, implementation, and evolution of forecast uncertainty products, services and information

13. Internally • Documenting ongoing NOAA forecast uncertainty activities • Collecting NWS forecaster needs and requirements (gaps) for forecast uncertainty tools • Looking at any “low-hanging fruit” changes/additions to products/services • Working with NOAA Programs and Goals to define forecast uncertainty needs for FY10-14 planning, programming, and budget cycle.

14. Externally • Working with Chair of 2006 NRC Study and other members of the NRC Panel to identify opportunities and feasibility for engaging the broader Weather-Water-Climate Enterprise in a Forecast Uncertainty partnership(s) • Led to the AMS Commission on Weather and Climate Enterprise hosting: ACUF (Ad Hoc Committee on Uncertainty in Forecasts) • Possible conjunction with the AMS Annual Partnership Topic

15. ACUF • Announcement on June 25, 2007 through AMS Commission on the Weather and Climate Enterprise (CWCE) residing under the Board on Enterprise Communication (BEC) • Charged to engage the weather and climate enterprise in identifying a vision of forecast uncertainty characterization and communication as motivated by the 2006 NRC report • Identify future paths for more thoroughly addressing forecast uncertainty products, services, and information needs of the Nation. • Propose roles and responsibilities of enterprise partners in developing, generating, and providing uncertainty information to the user community. • Reps from government, the private sector, academia, and the user community.

16. NOAA-wide Uncertainty Interests • NOAA Stakeholder Forum – May 2007 • Hazard Mitigation and Resilience Strategies recommendation: • “Develop advanced decision making and risk analysis tools and procedures including available and to be available information on the inevitable uncertainties in data analysis and forecasts.” • Ecosystems: • “Risk/Uncertainty – Ecosystem links are inherently uncertain, need to know the risks and costs of uncertainty.” • NOAA FY07-11 Research Plan • Addresses development of more useful products that convey uncertainty in all environmental information • Weather and Water Goal: • Generation and communication of forecast uncertainty information is a major theme/goal for FY10-14 planning, programming, and budgeting cycle

17. NCAR’s Related Efforts

18. Objective • To effectively communicate uncertainty-explicit forecasts, social science research is needed to better understand people’s perceptions, understanding, use, and preferences • To support provision of this information through survey research • To assess the U.S. public’s attitudes toward forecast and forecast uncertainty information

19. Uncertainty Research Questions • How much confidence do people have in different types of weather forecasts? • Do people infer uncertainty into deterministic forecasts and, if so, how much? • How do people interpret probability of precipitation forecasts? • Do people prefer to receive deterministic or uncertainty-explicit forecast information? • In what formats do people prefer to receive forecast uncertainty information?

20. Do people infer uncertainty into deterministic forecasts and, if so, how much?

21. Suppose the forecast high temperature for tomorrow for your area is 75F. 50% 40% 30% 20% 10% 0% 75 74-76 73-77 70-80 65-85 Other N=1465 What do you think the actual high temperature will be?

22. Do people prefer to receive deterministic or uncertainty-explicit forecast information?

23. Suppose you are watching the local evening news … Channel A Channel B Both Neither I don't know 0% 10% 20% 30% 40% 50% N=1465 • The Channel A weather forecaster says the high temperature will be 76F tomorrow • The Channel B weather forecaster says the high temperature will be between 74F and 78F tomorrow.

24. In what formats do people prefer to receive forecast uncertainty information?

25. All the choices below are the same as a probability of precipitation of 20%. Do you like the information given this way? • Chance of precipitation is 20% • There is a 1 in 5 chance of precipitation • The odds are 1 to 4 that it will rain • There is a slight chance of rain tomorrow  Percent  Frequency  Odds Text Asked this question 3 ways -- using 20%, 50%, and 80% probabilities of precipitation with corresponding descriptions

26. Overall distribution (% yes) 50% 80% 489, 487 100% 20% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Percent Frequency Odds Text N=489,

27. Future Work – Building on our Results • What are the next research questions to pursue? • How do we translate this understanding into improving how weather forecast information, including uncertainty information, is communicated?

28. Discussion Questions • How do we strike a balance between “educating” the users and understanding users’ needs and perceptions? • E.g., hurricane cone of uncertainty • How do we provide good uncertainty-explicit information given the proliferation of weather information, and how do we ensure the information is consistent from public and private sector sources? • How much of the role of social science does/should NOAA take on regarding the provision of uncertainty-explicit forecast information?

29. Resources • AMS, 2002: Enhancing weather information with probability forecasts. • Broad et al., 2007: Misinterpretations of the “cone of uncertainty” in Florida during the 2004 hurricane season. • Gigerenzer et al., 2005: How does the public understand probabilistic weather forecasts? • Murphy, 1998: The early history of probability forecasts: Some extensions and clarifications. • Murphy et al., 1980: Misinterpretations of precipitation probability forecasts. • NRC, 2003: Communicating Uncertainties in Weather and Climate Information: A Workshop Summary. • NRC, 2006: Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. • Roulston et al., 2006: A laboratory study of the benefits of including uncertainty information in weather forecasts. • Many, many more!