1 / 22

Testbeds and Projects with Ongoing Ensemble Research: Hydrometeorology Testbed (HMT)

DET Collaboration with Other Projects: Past and Future Ed Tollerud 1 and Tara Jensen 2 1 NOAA/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO . Testbeds and Projects with Ongoing Ensemble Research: Hydrometeorology Testbed (HMT) Hazardous Weather Testbed (HWT)

carrington
Download Presentation

Testbeds and Projects with Ongoing Ensemble Research: Hydrometeorology Testbed (HMT)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DET Collaboration with Other Projects: Past and Future Ed Tollerud1 and Tara Jensen 2 1 NOAA/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO • Testbeds and Projects with Ongoing Ensemble Research: • Hydrometeorology Testbed (HMT) • Hazardous Weather Testbed (HWT) • Hurricane Forecast Improvement Project (HFIP) Acknowledgements: Tim Schneider, Marty Ralph, Gary Wick, Rob Cifelli, Dave Reynolds, Steve Weiss, Barbara Brown, Jack Kain, Ming Xue, Fanyou Kong, Adam Clark, Patrick Marsh, Mike Coniglio, Russ Schneider, Evan Kucera, Tom Hamill, Tony Eckel, Louisa Nance, Mark DeMaria, James Franklin, Ed Rappaport, Mike Fiorino and many others

  2. Motivation • Many testbeds now have an ensemble component • Robust and objective evaluation of promising techniques needed • Determine potential utility for operations Model A Operations Testbed Model B Forecast Exercise Ensemble D Research Model C Model E General Format of Many Testbeds

  3. Goals • Leverage off work done in other testbeds • Extend understanding of their results • Help in planning for next year or season • Facilitate faster Research to Ops (R2O) Most effort expended around R2O Forecast Exercise Forecast Exercise There is much work to be done in the off-season - DET might be able to help Effort DET Retrospective Runs Time

  4. Traditional Scores for HWT ensemble members and baselines Recent Activities HWT Participants Evaluate Models Object-Oriented Evaluation Of HMT members

  5. 2010 DTC/HMT Ensemble Activities • High-resolution 9 member WRF regional ensemble for HMT-West Winter Exercises • First DTC ensemble evaluation • Individual deterministic models • Simple ensemble mean calculated and evaluated • 30-day Summary Scoring Displays • Baseline Comparisons (with GFS)

  6. 2010 HMT Ensemble NWP • 9 member ensemble • Grid: 9 km outer - 3 km inner • LBC: GFS Ensemble Members • North American domain • 80 km resolution • IC: LAPS

  7. Real-time QPF verification for HMT-West • January Summary, ETS • 24h forecast, full domain • Verified at 24h gage sites • Segregated by threshold • Dramatic performance hit at higher thresholds • No real ensemble member advantages in statistics but large inter-quartile range

  8. 2011 HMT Ensemble NWP • DET 1st Benchmark (NCEP) will hopefully be run for experiment • North American Domain • 20 km resolution • IC “cycling” option • 9 member ensemble • Grid: 9 km outer – 3 km inner (or potentially finer resolution) • IC: LAPS and/or Cycling • LBC: DET 1st Benchmark (NCEP)

  9. Future DTC/HMT Activities • Use DET Ensemble Configuration Module to set-up and run • First use of MET v3.0 including Ensemble-Stat Tool • Additional Baselines • Ensemble-basedSpatial Verification • Estimate Uncertainty in Verification Statistics

  10. What can HMT do for DET • HMT used for test domain for DET Module 2 (IC/LBC) Benchmark • Workflow Procedures will backbone of DET system • Input data options for initialization and verification (eg., QPF) • Ensemble Products Algorithms for inclusion in Module 5 (Products/Display) • Exploration of Spatial Verification Applications

  11. What can DET do for HMT • Help determine optimal ensemble configuration • Evaluate Ensemble Products • Visualization utilities (Displays) • Probability Products • State-of-the-art probabilistic verification measures

  12. Model Evaluation Tools (MET) DTC Ensemble Testbed (DET) etc…

  13. 2010 DTC/HWT Ensemble Activities • Evaluated 26 member CAPS Storm-Scale Ensemble Forecast System during 2010 Spring Experiment • Baselines: HRRR, SREF, NAM • Ensemble Products generated by CAPS– using 15 memberswith radarassimilation • First plots of BrierScore and Area Under Curve

  14. Evaluation of Simple and Derived Ensemble Products: Probability fields using Neighborhood Method Probability matching method for QPF QPF PQPF

  15. Future DTC/HWT Activities • DET participates in planning of CAPS ensemble constituents potentially with DET Ensemble Configuration Module, IC/LBC, and Physics Modules • Real-time verification performed using DET Verification Module (HWT-DTC Collaboration) • HWT products evaluated in DET Module 5 (Products/Display) • Feedback to model developers RETOP

  16. How DET may help HWT • Module 1 - Work with CAPS to define optimal ensemble member configuration • Module 2 – Work with DTC GSI task to identify GSI-based IC perturbation capabilities for CAPS ensemble • Module 3 – Evaluate individual members in 2010 dataset to provide suggestions for physics improvements • Module 4 – Implement bias correction technique that requires limited historical data • Module 5 – Evaluate ensemble products in 2010 dataset • Module 6 – Used in all evaluation

  17. HFIP Ensemble Work Many institutions involved Many approaches being investigated Global Ensembles • Single Models with IC/LBC and physics perturbations (GFS, NOGAPS, and FIM) Regional Ensembles • Multi-model ensemble made up of regional models run by different organizations statistically post-processed (FSU) • Ensembles initialized from EnKF (PSU) Consensus ensembles (NHC)

  18. How DET might contribute to HFIP • DET could be a testing ground for most promising ensemble techniques • Retrospective testing to address specific questions: • Module 2 – Modify near storm environment or storm position? • Module 3 – Which cumulus schemes provide best spread? • Module 4 – Which is best way to bias correct track and intensity? • Module 5 – How do we best display of uncertainty in track and intensity? • Module 6 – Does spatial (object oriented) verification of ensembles improve understanding of output?

  19. Some Ideas for Thefuture…

  20. Concept of Leveraging DET Modules • Use Near Real-time & Retrospectively • ---------------------------------------- • Immediate Feedback during forecast exercises • ROBUST Test & Eval • of promising techniques uncovered • during forecast exercises MODULE 1: Ensemble Configuration MODULE 2: IC/LBC Perturbations MODULE 3: Physics Perturbations MODULE 4: Statistical Post-Processing Mostly Retrospective -------------------------------------- 1) ROBUST Test & Eval of promising techniques uncovered during forecast exercises MODULE 5: Products/Display MODULE 6: Verification

  21. Questions for Working Group: • Should DET consider capability for realtime (or near realtime) demonstrations? • Where do we place the “overlap area” between DET and other ensemble-related testbeds? What does that area include? • What are the most critical testbed collaboration issues to focus on in the next year or two? • What interactions with other testbeds beyond HMT and HWT should be explored in the next few years?

  22. DTC Ensemble Testbed Model Evaluation Tools (MET) etc…

More Related