1 / 87

DTC AOP 2011 and Challenges

DTC AOP 2011 and Challenges. Bill Kuo. Outline. Review of DTC AOP 2010 tasks and budgets Highlights of DTC accomplishments Recommendations of SAB and DTC responses Proposed DTC AOP 2011 Future direction and challenges. DTC Organization & AOP 2010 Tasks. WRF : WRF modeling system

arista
Download Presentation

DTC AOP 2011 and Challenges

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DTC AOP 2011 and Challenges Bill Kuo

  2. Outline • Review of DTC AOP 2010 tasks and budgets • Highlights of DTC accomplishments • Recommendations of SAB and DTC responses • Proposed DTC AOP 2011 • Future direction and challenges

  3. DTC Organization & AOP 2010 Tasks WRF: WRF modeling system WRF for Hurricanes: HWRF, HFIP GSI: Grid-point Statistical Interpolation data assimilation system MET: Model Evaluation Tools HMT: Hydrometeorology Testbed collaboration HWT: Hazardous Weather Testbed collaboration DET: DTC Ensemble Testbed NEMS: NOAA Environmental Modeling System • Two major functions of DTC: • Provide support for community systems • Conduct testing and evaluation of community systems for research and operations

  4. DTC FY 2010 Budget Allocations (in $K)

  5. DTC Tasking and FY10 Funding Task areas Funding sources DTC Director’s office: $623K Visitor program: $222K WRF (Wolff): $698K HWRF (Bernardet): $1,123K GSI (Huang): $622K MET (Fowler): $487K HMT (Tollerud): $323K HWT (Jensen): $148K DET (Toth): $651K NEMS (Carson): $173K NOAA Core: $2,900K HFIP: $346K USWRP: $300K AFWA: $825K NCAR: $300K GSD: $300K NSF: $100K Total budget: $5,071K Collaborations with other testbeds

  6. Major DTC milestones: • Feb 8-9, 2010: DRAFT AOP 2010 reviewed by DTC MB • Feb 28, 2010: Received DTC EC approval for AOP 2010 • March 1, 2010: Begin AOP 2010 execution • May 25, 2010: DTC MB teleconference meeting • Aug 26, 2010: DTC EC meeting in WDC • Sep 21-23, 2010: Joint DTC SAB/MB Meeting • Oct 18, 2010: Received SAB written report • Jan 11-12, 2011: DTC MB to review DRAFT AOP 2011

  7. What we will discuss in this meeting: • SAB recommendations and DTC responses: • Do we agree with all their recommendations? • For those that we agree, how do we incorporate their recommendations into AOP 2011 (with potentially reduced budget)? • Continuing resolution and its impact on DTC AOP 2011 & budget and DTC operation: • We don’t have final numbers from all DTC sponsors • May not know the numbers until well into the AOP 2011 • How should DTC operate under these budget uncertainties?

  8. Highlights of DTC Activities • Hired DTC software engineer (Eugene Mirvis) to work on NEMS task at EMC, in collaboration with EMC staff. • Hired DTC scientist (MrinalKantiBiswas) to work on hurricane task. • Established the Community GSI repository, and a process for community to contribute to GSI development. • Started the development of DET modules. • Tested WRF community code for 2011 operational hurricane prediction at NCEP. Additional highlights will be presented by task leads.

  9. GSI R2O Transition Procedure (draft) 1 2 Code development candidate DTC branch Community research Code development candidate DTC branch 3 5 4 DTC trunk EMC trunk EMC branch GSI Review Committee - initial scientific review DTC - developer code merging and testing GSI Review Committee - code and commitment review DTC->EMC GSI code commitment EMC->DTC GSI repository syncing

  10. GSI Transition: O2R and R2O • Past (two years ago?) • Distributed development • Different applications and operational requirement (GFS, RTMA, NAM, RR, AFWA…) • Manual version control • No system documentation • limited system support (for GSI partners and community) • Non-portable system • Current • GSI Review Committee (DTC, NCEP/EMC, NASA/GMAO, NOAA/ESRL, NCAR/MMM-AFWA) • Multiple platform GSI system supported by DTC • SVN version control (Duel repositories with synced trunk and localized development branches) • Completed GSI User’s Guide and website • Annual community release and residential tutorial • Community user support by DTC (gsi_help@ucar.edu) • R2O infrastructure and application Support by EMC (John Derber) was essential!!

  11. HWRF code management: atmospheric model HWRF release 04/2011 WRF V3.3 HWRF for operations (2011) Contributions from EMC and NOAA Research to HWRF (preliminarily tested 3rd nest, new nest motion) 02/2011 WRF V3.2.1+ 09/2010 WRF V3.2.1+ 08/2010 WRF V3.2.1 2011 Baseline “R2 final” Tutorial, HWRF Beta release 03/2010 WRF V3.2 07/2009 2010 baseline “R1” WRF V3.1.1 (…) 05/2004 WRF V2 oper HWRF 2009 oper HWRF 2010 “R2” (…)

  12. Functionally-equivalent T&E suite • Pre-processing (including ability to read binary GFS in spectral coordinates) • Cycled HWRF vortex initialization and relocation • GSI Data Assimilation • Coupled (POM + WRF) model • Post-processing • Tracking • NHC Verification & confidence intervals • Display • Archival • The Developmental Testbed Center • HWRF 2011 Baseline Test Plan • Point of Contact: Ligia Bernardet • December 15, 2010 • Introduction • The DTC will be performing testing and evaluation for the Hurricane WRF system, known as HWRF (Gopalakrishnan et al. 2010). HWRF will be configured as close as possible to the operational HWRF model, employing the same domains, physics, coupling, and initialization procedures as the model used at the NOAA NCEP Central Operations and by the model developers at NCEP EMC. The configuration to be tested matches the 2011 HWRF Baseline, which is the configuration that served as control for all developments at EMC geared towards the 2011 operational implementation.

  13. SAB Recommendations and DTC responses

  14. General Conclusions: • SAB believes DTC currently risks becoming spread over too many projects and directions. While most activities are relevant, some tasks appear to be peripheral to DTC and critical U.S. NWP goals. • We restructured the DTC tasks for AOP 2011 into five focused areas. We examine all tasks to ensure their relevance to the core missions of DTC. New approach should improve the way we communicate the DTC activities to the outside community. • The SAB believes that these accomplishments, although substantial, have not been sufficient to realize DTC’s core objectives, as noted in the introduction. For example, a functionally equivalent operational environment available for extended period tests is not available, although progress has been made in that direction. • AFWA considers DTC to have a functionally equivalent test environment for AFWA related work. • DTC now has a functionally equivalent test environment for HWRF (including full cycling capability) • Efforts on “functionally equivalent test environment” are needed on mesoscale modeling (e.g., NAM, NEMS, and NMM-B).

  15. General Conclusions: • To date, a substantial amount of DTC effort has been placed on the long-term development of infrastructure in a range of areas, such as MET software package. However, it is time to place increased emphasis on infrastructure utilization that directly addresses DTC’s central goals. • SAB asked DTC to increase testing and evaluation efforts that directly contribute to the core missions of the DTC. • With flat budget, the efforts on community support have to be decreased.

  16. Specific Recommendations and Conclusions • (1) The DTC should give priority to building a functionally equivalent operational environment to test and evaluate new NWP methods for extended retrospective periods and significant events using advanced verification tools. Such an environment should include operations-like data assimilation/cycling. Sufficient resources should be provided to insure this is in place within 12 months. • DTC will place greater efforts on this, particularly related to the mesoscale modeling. Assembling a functionally equivalent environmentwill require enhanced collaboration among several task areas, including mesoscale modeling, hurricane, data assimilation and ensemble efforts. • (2) The SAB was in general agreement that a central role for DTC is to be the nation’s model “scorekeeper” that would track mesoscale model forecast improvement and serve as the key benchmark center for U.S. mesoscale NWP. • DTC will develop a “test plan” for Reference Configuration (RC) testing that can help track mesoscale model forecast improvement. RC testing will include EMC and AFWA operational configurations. Concept will provide a framework for tracking the improvements of WRF systems with releases of new versions. The test plan will be reviewed by SAB representatives, EMC, and MMM. • Increased efforts on T&E will require additional resources at the expense of community support services.

  17. Specific Recommendations and Conclusions • The establishment of a suite of model verification tools by DTC (including, but not limited to the MET package) was appropriate, although several enhancements are still required. For example, plan-view spatial verification is a critical tool that is currently missing from MET. Furthermore, DTC verification should support the full range of mesoscale assets, such as ACARS, NEXRAD and profiler data. However, the current lack of comprehensive, actionable verification statistics for major contemporary U.S. modeling systems (see 2 above) is of some concern and higher priority must be given to making this information available more rapidly, even if that requires redirecting some of the resources currently being provided to construct and support more specialized MET capabilities. • Some of these features are already available in MET. Further enhancement will require additional resources. • DTC will make a contribution to actionable verification statistics for major contemporary U.S. modeling systems, through Reference Configuration testing.

  18. Specific Recommendations and Conclusions • (4) The board notes that DTC has supported workshops on some modeling systems that are not widely used in the community (e.g., NMM model). Such activities may be justifiable, but it may be worthwhile to review whether their current frequency reflects optimal use of DTC resources. • DTC does not hold NMM workshops. • DTC and EMC need to decide on NMM-B (and NEMS) tutorials, as well as Joint ARW-NMM tutorials with MMM for AOP 2012. • (5) The committee believes that although the DTC itself must maintain competency and knowledge in the new NWS model infrastructure (NEMS), for the immediate future there is little requirement for any significant NEMS outreach activities to the research community. • DTC has no plan to start a tutorial on NEMS framework. DTC hired a staff to work at EMC, to gain expertise in NEMS, and will have the ability to implement components of other modeling systems into the NEMS-based EMC operational system for testing and evaluation. • Louis Uccellini believes that the most effective R2O is for the research community to use operational systems for their research.

  19. Specific Recommendations and Conclusions • (6) The general sentiment of the SAB is that, although the NOAA HydrometeorologicalTestbed (HMT) and Hazard Weather Testbed (HWT) are important national endeavors, DTC participation in these activities may be a secondary priority with respect to DTC’s core mission. • Testbed collaborations are important to NOAA • HWT and HMT collaborations are no longer identified as DTC task areas. Rather, they will be DTC special projects that are directly linked to a few DTC focused areas. Efforts are made to ensure that these projects contribute to DTC core missions. • (7) The committee believes that the DTC can potentially play a unique role in bringing the research and operational communities together to examine and address key national NWP problems. Dealing with existing deficiencies in NWP physics parameterizations is one such problem. • For AOP 2011, the DTC will organize a Physics Workshop, in collaboration with EMC and the academic community, to be held in WDC area in August 2011.

  20. Specific Recommendations and Conclusions • (8)… there remained concern on the board that, because DET is inevitably a costly long-term project, it may limit DTC’s ability to demonstrate sufficient short-term value and relevancy to its sponsoring agencies and the NWP community at large. Thus, some difficult resource decisions may become necessary. • DET is an important activity for NWS to realize the goals outlined in the “White Paper”. DTC will continue to explore additional resources to augment and support such effort. • DET needs to quickly demonstrate short-term value and relevancy to the sponsoring agencies and NWP community at large. • (9)… Based on this experience and a review of previous cycles of the program, the board believes that the visitor program is an appropriate and important element of DTC activities. However, the board believes that the current program casts too wide a net in its request for proposals…Future calls should be crafted to clearly identify one or two mission-critical areas where collaborations with applicants having particularly relevant expertise can significantly augment DTC capabilities and hasten transition of effective NWP solutions to the broader community. • DTC will improve the process for the visitor program for 2011. • Input from MB will be an important part of this process!

  21. Specific Recommendations and Conclusions • (10) Regarding hurricane-related activities, the board felt that there is a need to establish and maintain a reference configuration and operational testbed for the Hurricane-WRF (HWRF). Such a testbed must include a data assimilation component. Furthermore, there is a need for the development and support of relevant diagnostic tools for hurricanes. • The HWRF system maintained by DTC includes a data assimilation component. DTC will include HWRF as a Reference Configuration for testing, with full cycling data assimilation. DTC will also develop and support relevant diagnostic tools, in collaboration with HFIP partners, pending on HFIP support.

  22. AOP 2011 Tasks Testbed collaborations: HWT & HMT DTC activities are distilled into five focused areas HWT & HMT are cross-DTC special projects, with contributions to DTC focused areas identified NEMS is included as part of mesoscalemodeling

  23. Required budget for proposed AOP 2011 activities Total actual funding for AOP 2010 was $5,070K. Total for AOP 2011 is a 8.7% increase.

  24. Future Directions and Challenges • Mesoscale modeling, NMM-B and NEMS • NCEP is moving forward with NMM-B and NEMS, and expects DTC to entrain the community into these operational systems • SAB does not recommend DTC provide community support for NMM-B and NEMS • DTC needs to have a stronger linkage to mesoscale modeling at NCEP (that will contribute directly to NCEP operations). Physics workshop is a start. • Future NCEP short-range ensemble will include NMM-B and ARW in the NEMS framework. • In 2012, DTC needs to decide whether to continuejoint ARW-NMM tutorials (and whether to do NEMS tutorial).

  25. Future Directions and Challenges • Data Assimilation: • NCEP is moving toward GSI-EnKF hybrid data assimilation for global modeling • Multiple EnKF systems are being developed under HFIP sponsorship • AFWA asked DTC to examine a few regional EnKF systems for possible operational use • DTC needs to work with EMC and AFWA to decide on a community hybrid GSI-EnKF system for both regional and global NWP applications. DTC should not support multiple EnKF systems to the community.

  26. Future Directions and Challenges • Hurricane: • DTC made significant progress in merging the operational HWRF code into WRF repository, and in testing the community HWRF code for operation • Needs to assess the impact of EnKF on hurricane prediction • Needs to facilitate improvements made by HFIP community into operations (i.e., Hurricane Test Plan) • Future of operational hurricane model at NCEP: • Will HWRF migrate toward NMM-B on NEMS? • Should we consider migrating toward AHW?

  27. Future Directions and Challenges • Ensemble: • Very good start on the development of the DET infrastructure and first few modules during AOP 2010 • Need to demonstrate sufficient short-term value to operations and NWP community: • EMC: Next generation SREF • HWT: Collaboration with CAPS and SPC cloud-scale ensemble • HMT: Ensemble verification • Making the DET modules available to the community • DET test and evaluation activities will require significant compute resources • Additional resources required to accelerate development

  28. THANK YOU!

  29. DTC Director’s Office Louisa Nance

  30. Director’s Office Responsibilities • Internal Coordination • Manage and coordinate DTC tasks • Planning, budgeting, execution and reporting • External Communication • Conduct or assist with workshops and tutorials • Interact with DTC partners on collaborative efforts • Create and maintain the DTC Website • Provide administrative support for DTC EC, MB & SAB meetings • Host the DTC Visitor Program

  31. Director’s Office Staff • Management • Bill Kuo – Director (0.5 FTE) • Louisa Nance – Assistant Director (0.5 FTE) • Barb Brown – JNT Director (0.10 FTE) • Bonny Strong - JNT DTC Manager (0.25 FTE) • Steve Koch – GSD Director (0.10 FTE) • TBD - GSD DTC Manager (0.25 FTE) • Administrative Support • Pam Johnson (0.50 FTE) Total FTEs: 2.20 FTEs

  32. Major accomplishments for AOP 2010 • Internal Coordination • DTC staff retreat (Apr 2010) • Monthly staff / task lead meetings • External Communication • DTC Management Meetings • 1st face-to-face meeting of DTC Executive Committee (Aug 2010) • 1st SAB/MB meeting (Sept 2010) • DTC Visitor Program • Announcement of Opportunity (Jun 2010) • Selected 6 proposals from 22 submissions for funding (Sept 2010) • Selected hosts from DTC staff for each project • Hosted initial visits for 3 projects

  33. DTC Visitor Program Withdrawn

  34. Proposed activities for AOP 2011 • Internal Communication • Implement tools to assist with task and staff coordination • Provide a framework for cross-coordination between over-lapping activities of major task areas • Coordinate strategic planning activities • External Communication • Continue making improvements to DTC website • Host MB, EC & SAB meetings and keep open channels of communication • DTC Visitor Program • Host visitors for 5 funded projects • Prepare a new AO & select next round of visitor projects • Provide support for upcoming workshops and tutorials

  35. Challenges for Director’s Office • Meeting the needs/interests of the NWP community during tight funding period • Cross-coordination between task areas • Verification, HMT & HWT • EnKF for DA, Hurricanes & Ensembles • Determining appropriate focus and scope for next DTC Visitor Program Announcement of Opportunity

  36. Mesoscale Modeling Jamie Wolff Collaborators: NOAA/NCEP/EMC NOAA/ESRL/GSD NCAR/NESL/MMM

  37. Mesoscale Modeling Goals • Community Support • Maintain, support and facilitate contributions to the community-released code repositories, currently including: • WRF ** • Post Processing Software * • Support community outreach events ** -in collaboration with MMM* and EMC* • Testing and Evaluation • Extensively test and evaluate a variety of model configurations that will provide benefits to both the operational and research communities • Reference Configurations (RCs) • Provide a functionally equivalent operational environment for testing and evaluating new techniques or capabilities

  38. Major Accomplishments for AOP 2010 • Community outreach and support ** • WRF v3.2 release ,WRF Workshop, WRF Tutorials, wrfhelp email • Significant progress on transition from WPP to UPP * * * • Established community-UPP code repository -in collaboration with MMM*, EMC*, GSD* and AFWA* • Published/updated online docs for past & current activities on the DTC T&E webpage • http://www.dtcenter.org/eval • Designated several DTC RCs and retested two with WRF v3.2.1 • http://www.dtcenter.org/config • Performed evaluation of GFS/NAM Precipitation Forecast Comparison • Presented results at relevant conferences • Began to develop DTC expertise in NEMS software • Software Engineer hire focused on NEMS development (at EMC) • Held a technical information exchange meeting (October 2010) • EMC, GSD, MMM, DTC representatives

  39. Highlight:GFS/NAM Precip Forecast Comparison FSS: NAM15 is consistently higher than the GFS60 across multiple thresholds (12-h lead time shown) • MODE: Counts (left) and size distribution (right) for all objects defined within the NAM4 forecast are more consistent with the observation field than the GFS4 forecast

  40. Community Support - AOP 2011 • Proposed Activities • Maintain/Enhance WRF regression testing * • Continue on-going efforts in community support ** • Finish preparations of community-UPP software package for distribution * • Extensively test, write documentation • Provide community support upon release • Anticipated major accomplishments • WRF release (v3.3 April 2011) ** • Community-UPP version 1.0 release (April 2011) * • Bi-annual WRF Tutorials (July 2011, January 2012) ** • Annual WRF Users Workshop (June 2011) * -in collaboration with MMM* and EMC*

  41. Testing and Evaluation - AOP 2011 • Proposed Activities • Continue expansion of RC testing and evaluation • Strengthen the foundation of DTC expertise with the NEMS software • Continue to contribute to development process in select areas (e.g. portability, inter-operability, I/O layer capabilities) • Co-host physics workshop for mesoscale modeling • Work on publications for DTC methodology and results from select test activities • Anticipated major accomplishments • New RC designations or retests, as appropriate • Functionally equivalent operational environment established at the DTC to assess new NWP methods • White paper outlining short-term and longer-term approaches for making significant progress toward improving physics parameterizations • Manuscript(s) submitted to appropriate peer-reviewed journals

  42. Resource requirements for AOP 2011 *Contributions from testbed collaborations – proposed work presented separately

  43. Hurricanes Ligia Bernardet External collaborators: NOAA Environmental Modeling Center NOAA Geophysical Fluid Dynamics Laboratory NOAA Atlantic Oceanographic and Meteorological Laboratory NCAR Mesoscale and Microscale Meteorology Division University of Rhode Island

  44. Hurricanes Goals • Facilitate transfer of research to operations by creating a framework for NCEP and the research community to collaborate • Support the community in using operational hurricane models • Develop and maintain a hurricane testing and evaluation infrastructure at DTC • Perform tests to assure integrity of community code and evaluate new developments for potential operational implementation

  45. Major Accomplishments for AOP 2010 • Release of HWRF to the community: code management, documentation, support to 150 registered users. • Transition of Community Code to EMC to serve as baseline for 2011 operational implementation. • Creation of a functionally-equivalent infrastructure to run HWRF on jet (including data assimilation and cycling) • Testing and evaluation: • Routine extended regression tests to evaluate integrity of code • Ran 400 cases from the 2008 and 2009 hurricane seasons in preparation to designate a DTC RC.

  46. Functionally-equivalent T&E suite • Pre-processing (including ability to read binary GFS in spectral coordinates) • Cycled HWRF vortex initialization and relocation • GSI Data Assimilation • Coupled (POM + WRF) model • Post-processing • Tracking • NHC Verification & confidence intervals • Display • Archival • The Developmental Testbed Center • HWRF 2011 Baseline Test Plan • Point of Contact: Ligia Bernardet • December 15, 2010 • Introduction • The DTC will be performing testing and evaluation for the Hurricane WRF system, known as HWRF (Gopalakrishnan et al. 2010). HWRF will be configured as close as possible to the operational HWRF model, employing the same domains, physics, coupling, and initialization procedures as the model used at the NOAA NCEP Central Operations and by the model developers at NCEP EMC. The configuration to be tested matches the 2011 HWRF Baseline, which is the configuration that served as control for all developments at EMC geared towards the 2011 operational implementation.

  47. Proposed activities for AOP 2011 • Continue HWRF community support. • Continue HWRF code management, keeping the evolving community and EMC versions of HWRF connected. • Upgrade DTC testing suite by adding HYCOM, UPP and ability to run high-resolution. • Perform extensive testing of HWRF. Actual tests are TBD, but could include changes in resolution, alternate physics and initialization (EnKF). • Diagnostic activities: • Evaluate HWRF to understand weaknesses and sources of error. • Begin assembling a diagnostic toolbox for DTC and the community* *Pending HFIP funding.

  48. Anticipated major accomplishments for AOP 2011 • Hurricane Prediction Test Plan, in collaboration with EMC and HFIP, describing plans for tests to be conducted by DTC in 2011 and a protocol for how future tests will be determined. • Hurricane Tutorial in April 2011. • Publication of a HWRF Reference Configuration. • Expanded WRF Community Code with addition of new developments for HWRF. • Expanded HWRF Testing infrastructure on jet with the addition of new components (HYCOM and high-resolution). • Input to NCEP pre-implementation decisions through HWRF testing and evaluation at DTC.

  49. Resource requirements for AOP 2011 *Includes funding for hurricane diagnostic toolbox

  50. Data Assimilation Xiang-Yu (Hans) Huang *presented by Ming Hu Collaborators: AFWA, NCEP/EMC, NASA/GMAO, NOAA/GSD, NCAR/MMM

More Related