Presented to NOAA Science Advisory Board Washington, DC 17 March 2004

1. EXTERNAL REVIEWOF THECOOPERATIVE INSTITUTE FOR MESOSCALE METEOROLOGICAL STUDIES (CIMMS)15-16 OCTOBER 2003UNIVERSITY OF OKLAHOMANORMAN, OKLAHOMA Presented to NOAA Science Advisory Board Washington, DC 17 March 2004

2. External Review Team • Denise Stephenson Hawk, Ph.D., Panel Chair Chairman, The Stephenson Group, LLC; NOAA SAB • Gregory S. Forbes, Ph.D. Severe Weather Expert, The Weather Channel • Eve Gruntfest, Ph.D. Geography and Environmental Studies University of Colorado • Jack Hayes, Ph.D. Director, Office of Science and Technology, NOAA NWS • Robert A. Weller, Ph.D. Director, Cooperative Institute for Climate and Ocean Research Woods Hole Oceanographic Institution • Derek Winstanley, Ph.D. Chief, Illinois State Water Survey

3. OVERVIEW • Guidelines for External Review • CIMMS - Background, Purpose and Scope • Review and Recommendations • Science Plan • Scientific Partnerships • Science Review • Education and Outreach • Science Management Plan • Summary and Conclusions

4. Guidelines for External Reviews • NOAA Office of Oceanic and Atmospheric Research (OAR) provided review format and guidelines to CIMMS and Review Team • Science Plan • Science Review • Education/ Outreach • Science Management Plan • Review conducted under auspices of NOAA SAB

5. CIMMS… • Cooperative program between NOAA and University of Oklahoma (OU) • Established in 1978 to… • Act as research interface between OU, OAR, NWS, NOAA • Assist in transitioning products of research into operational procedures /techniques • Governed by Memorandum of Agreement (MOA) - Last dated June 1995 with addenda December 2000 and August 2002 • Budget for FY 2003: $10,146,626 [60% NOAA; 16% OU; 24% other – NASA, NSF, DOE, ONR, WMO, U.S. EPA…] • Director: Peter Lamb, Ph.D. George Lynn Cross Research Professor of Meteorology

6. CIMMS Research Themes • Basic convective and mesoscale research (1982) • Forecast Improvements (1990) • Climatic Effects of and controls on mesoscale processes (1990) • Socio-economic impacts of mesoscale weather systems and of regional scale variations (200?) • Doppler weather radar research and development (1995) • Climate change monitoring and detection (2001)

7. CIMMS Cooperating Partners • National Oceanic and Atmospheric Administration • Office of Oceanic and Atmospheric Research • National Weather Service (NWS) • University of Oklahoma • Operational Support Facility for WSR-88D Radar • National Environmental Satellite, Data, and Information Service (NESDIS) • Southern Region Headquarters of NWS

8. HIGHLIGHTS OF CIMMS • Significant role in modernization of NWS, particularly through upgrading of radar hardware and software systems • Recent advances in accuracy and lead time of tornado warnings • Extensive partnerships with the Norman, OK scientific community • Department of Commerce Silver Medal in 1999 for Warning Decision Support System (WDSS) • Department of Commerce Gold Medal in 2003 for Weather Event Simulator (WES)

9. Review and RecommendationsScience Plan Science Plan defined – Clearly articulated set of strategies for planning and managing the science portfolio within the framework of the MOA Recommendation 1: To effectively and efficiently serve the needs of NOAA, as defined within the NOAA and line office strategic plans, CIMMS must define a well-articulated strategic science plan that aligns with its established MOA.

10. Review and RecommendationsScience Plan Recommendation 2: CIMMS management should engage the expertise of the NOAA Strategic Planning Office to provide training regarding the tenets and development of strategic science plans.

11. Review and RecommendationsScience Plan What is the scientific (not programmatic) vision for the Institute? Recommendation 3: The leadership of CIMMS must engage the OU and NOAA partners in the development of a vision for the Institute. The vision should establish the rationale behind the six themes agreed upon by NOAA and OU.

12. Review and RecommendationsScience Plan How is the scientific vision related to the NOAA Strategic Plan? Recommendation 4: The leadership of CIMMS must work with NOAA to develop a scientific vision that aligns with the NOAA Strategic Plan.

13. Review and RecommendationsScience Plan What are the goals and objectives [CIMMS]? Recommendation 5: OU and CIMMS must establish a process to coordinate specific thematic research goals and objectives with OAR as part of the NOAA strategic planning and programming process and to use these higher level specific goals and objectives to challenge individual scientists.

14. Review and RecommendationsScience Plan What criteria are used to measure progress in accomplishing these goals and objectives? Recommendation 6: OU/ CIMMS must define more specific indicators of progress in each theme area and for each project for purpose of improving the linkage between OU/ CIMMS research and the NOAA strategic goals and objectives.

15. Review and RecommendationsScience Plan • What are the major scientific themes? • How were they identified? • Which themes/sub-themes are near completion? • What are the emerging thematic areas? Why? Recommendation 7: OU/ CIMMS should evaluate its current research focus relative to the mission, goals and objectives of NOAA and develop a strategic plan to better focus its research in areas that will result in enhanced contributions to the science of NOAA.

16. Review and RecommendationsScientific Partnerships • What is the relationship to OAR Laboratories and other NOAA entities? Thirteen Partnerships • OAR National Severe Storms Laboratory • NWS Warning Decision Training Branch • NWS Storm Prediction Center • Radar Operations Center, Forecast Office in Norman, OK • NWS Southern Region Headquarters • NESDIS National Climatic Data Center • NWS International Activities Office • OAR Office of Global Programs • OU • Other NOAA and non-NOAA organizations within Norman, OK

17. Review and RecommendationsScientific Partnerships • Proposed National Weather Center (NWC) • Will transform the current Oklahoma Weather Center into a larger, more structured organization, which will be led, administered and managed by OU • Responsibilities of NWC • Lead and implement the University Radar Meteorology Initiative • Establish and direct Center for Radar Meteorology and Engineering; Radar Phenomenology Laboratory, Radar Data Portal, and National Mobile Observing Facility • Direct and administer CIMMS

18. Review and RecommendationsScientific Partnerships • What, if any, formal procedures exist for joint planning? Recommendation 8: OU should endeavor to create an environment within the facility [National Weather Center] that fosters and maximizes scientific interactions between OU, CIMMS, NOAA personnel and the broader meteorological community. Recommendation 9: CIMMS must develop operational policies and processes, which will ensure that the conditions of the existing NOAA/OU Memorandum of Agreement, particularly the delineated research themes, are managed in a manner to avoid any apparent or perceived conflicts-of-interest with the mission and mandates of NOAA.

19. Review and RecommendationsScience Review • What are the Institute’s most recent scientific highlights and accomplishments? Recommendation 10: CIMMS should consider assigning a higher priority to certain aspects of its research program where results would more directly impact NOAA operations. Recommendation 11: CIMMS should consider expanding its research initiatives in the theme area: Socioeconomic Impacts of Mesoscale Weather Systems and Regional Scale Variations. CIMMS should consider including in these research initiatives a strategy to quantify the value-added benefits to the nation of weather forecasting and research.

20. Review and RecommendationsEducation and Outreach • What types of educational activities/opportunities (K-12, undergraduate and graduate students) does the Institute offer on an on-going basis? • Fall 2003: Twenty-seven graduate students supported by CIMMS • Average of ten additional undergraduate students supported by National Science Foundation

21. Review and RecommendationsEducation and Outreach • Recommendation 12: It is strongly recommended that CIMMS develop strategic, tactical and operational plans to address education, public affairs, and outreach, and that NOAA funds are appropriated for these activities. • Recommendation 13: It is strongly recommended that CIMMS increase its leveraged funding levels in the areas of education, public affairs and outreach.

22. Review and RecommendationsEducation and Outreach • Recommendation 14: CIMMS should provide statistics regarding the number/diversity of participants educated through outreach programs in terms of gender, age, ethnicity and citizenship disaggregated by participation during the academic year and during the summer and promote diversity in the workplace. • Recommendation 15: It is strongly recommended that CIMMS quantify its education, public affairs and outreach initiatives, in terms of demographics, to demonstrate the value-added contribution of these initiatives to meteorology, to NOAA and to the Nation.

23. Review and RecommendationsEducation and Outreach What are the current and planned outreach efforts? CIMMS … • 10% of CIMMS Fellows are not affiliated with OU • 57% (158 of 278) of refereed publications from 1998-2003 had at least one non-CIMMS affiliated member • Matching support for Vietnam graduate students • Multi-institution program for the Warning Decision Support System II – CIMMS, OU, NSSL, NWS, Australian Bureau of Meteorology and the Georgia Tech Research Institute • Dissemination of software and training for the Weather Event Simulator • Dissemination of current climate research to the meteorological and hydrological services of developing nations

24. Review and RecommendationsScience Management Plan • Science Management Plan • Prepared by managers to articulate to staff, senior officials, and other stakeholders how the organization is managed for purpose of developing and implementing the science plan Recommendation 16: OU/CIMMS should develop a science management plan to state to staff, senior officials and other stakeholders how the organization is managed for purpose of developing and implementing its science plan.

25. Review and RecommendationsScience Management Plan • How does the Institute identify new intellectual opportunities? • What are some recent examples of intellectual opportunities? CIMMS has… • Large and well-accomplished scientific research staff • Collaborated with NOAA and its partners to define intellectual opportunities • Collaborated with OU to project in excess of $200 million in revenue from NWC in its initial decade • Involvement in the OU Meteorological Radar Research Initiative

26. Review and RecommendationsScience Management Plan • What is the strategy for new starts (projects, techniques, campaigns, etc.)? • Scientist-driven; proposals submitted • How much of the Institute resources are reserved for new opportunities or bright ideas? • None identified

27. Review and RecommendationsScience Management Plan • What is the demographic structure of the Institute employees? • Summer : Employees (144); Male (72%); Caucasian or Asian/Pacific Islanders (90%); African Americans (3%); Native Americans (<1%); Hispanic (6%); U.S. Citizens (73%) • What is provided for human resources development? (Recruitment, rewards, training) Recommendation 17: CIMMS should develop a human resource management plan to ensure a diverse workforce.

28. Review and RecommendationsScience Management Plan • What is the state of the financial health of the Institute? (Provide a budget summary and identify imbalances or needed adjustments? • How does the institute intend to work towards accomplishing its financial goals? CIMMS is… • Well-positioned for continued success by virtue of its physical and human capital Recommendation 18: NOAA should require the Joint/Cooperative Institutes to provide to Review Teams briefing materials and presentations that are in direct response to the questions that NOAA requires the Review Teams to assess and evaluate.

29. Review and RecommendationsScience Management Plan • Are there any issues in interacting with NOAA that require attention? • Are there any issues in interacting with the University that require attention? Recommendation 19: NOAA should distribute appropriated funds to the Joint/Cooperative Institutes in a timely manner, thus preventing cash-flow problems between the host institutions and the Joint/Cooperative Institutes. Recommendation 20: NOAA should more fully involve Joint Institutes in its strategic/operational and budget planning sessions to ensure that the value-added contributions of the JIs are included in the final budget preparations and submissions.

30. Review and RecommendationsScience Management Plan • Are there any issues in interacting with NOAA that require attention? • Are there any issues in interacting with the University that require attention? Recommendation 21: The Review Team recommends that NOAA/OAR review its involvement in NOAA’s strategic planning process with a goal of more meaningful involvement on the part of the JIs. Recommendation 22: NOAA/OAR should complete an in-depth review with the assistance of appropriate legal and contracts personnel, and disseminate clear, unambiguous guidance to all Joint Institutes dealing with the Department of Commerce legal concerns over “direct benefit research.”

31. Review and RecommendationsScience Management Plan • Are there any issues in interacting with NOAA that require attention? • Are there any issues in interacting with the University that require attention? Recommendation 23: CIMMS should support and encourage staff to interact with researchers at the NOAA laboratories and Joint Institutes across the country and to share their expertise with NOAA program managers and planners outside the immediate CIMMS/OU area.

32. Summary and Conclusions • CIMMS has… • Extended NOAA expertise for over 25 years • Increased its staff from 38 in FY1991 to over 140 in FY2004 • Provided significant contributions to the evolution of NOAA science • Received awards for its science • Generated over 270 referred publications in the past five years • Grown its budget from $1.4million in FY1991 to over $10 million in FY2003 • CIMMS, with support from NOAA, can … • More strongly position itself to benefit NOAA and the nation