220 likes | 362 Views
DET Collaboration with Other Projects: Past and Future Ed Tollerud 1 and Tara Jensen 2 1 NOAA/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO . Testbeds and Projects with Ongoing Ensemble Research: Hydrometeorology Testbed (HMT) Hazardous Weather Testbed (HWT)
E N D
DET Collaboration with Other Projects: Past and Future Ed Tollerud1 and Tara Jensen 2 1 NOAA/GSD, Boulder, CO 2 NCAR/RAL, Boulder, CO • Testbeds and Projects with Ongoing Ensemble Research: • Hydrometeorology Testbed (HMT) • Hazardous Weather Testbed (HWT) • Hurricane Forecast Improvement Project (HFIP) Acknowledgements: Tim Schneider, Marty Ralph, Gary Wick, Rob Cifelli, Dave Reynolds, Steve Weiss, Barbara Brown, Jack Kain, Ming Xue, Fanyou Kong, Adam Clark, Patrick Marsh, Mike Coniglio, Russ Schneider, Evan Kucera, Tom Hamill, Tony Eckel, Louisa Nance, Mark DeMaria, James Franklin, Ed Rappaport, Mike Fiorino and many others
Motivation • Many testbeds now have an ensemble component • Robust and objective evaluation of promising techniques needed • Determine potential utility for operations Model A Operations Testbed Model B Forecast Exercise Ensemble D Research Model C Model E General Format of Many Testbeds
Goals • Leverage off work done in other testbeds • Extend understanding of their results • Help in planning for next year or season • Facilitate faster Research to Ops (R2O) Most effort expended around R2O Forecast Exercise Forecast Exercise There is much work to be done in the off-season - DET might be able to help Effort DET Retrospective Runs Time
Traditional Scores for HWT ensemble members and baselines Recent Activities HWT Participants Evaluate Models Object-Oriented Evaluation Of HMT members
2010 DTC/HMT Ensemble Activities • High-resolution 9 member WRF regional ensemble for HMT-West Winter Exercises • First DTC ensemble evaluation • Individual deterministic models • Simple ensemble mean calculated and evaluated • 30-day Summary Scoring Displays • Baseline Comparisons (with GFS)
2010 HMT Ensemble NWP • 9 member ensemble • Grid: 9 km outer - 3 km inner • LBC: GFS Ensemble Members • North American domain • 80 km resolution • IC: LAPS
Real-time QPF verification for HMT-West • January Summary, ETS • 24h forecast, full domain • Verified at 24h gage sites • Segregated by threshold • Dramatic performance hit at higher thresholds • No real ensemble member advantages in statistics but large inter-quartile range
2011 HMT Ensemble NWP • DET 1st Benchmark (NCEP) will hopefully be run for experiment • North American Domain • 20 km resolution • IC “cycling” option • 9 member ensemble • Grid: 9 km outer – 3 km inner (or potentially finer resolution) • IC: LAPS and/or Cycling • LBC: DET 1st Benchmark (NCEP)
Future DTC/HMT Activities • Use DET Ensemble Configuration Module to set-up and run • First use of MET v3.0 including Ensemble-Stat Tool • Additional Baselines • Ensemble-basedSpatial Verification • Estimate Uncertainty in Verification Statistics
What can HMT do for DET • HMT used for test domain for DET Module 2 (IC/LBC) Benchmark • Workflow Procedures will backbone of DET system • Input data options for initialization and verification (eg., QPF) • Ensemble Products Algorithms for inclusion in Module 5 (Products/Display) • Exploration of Spatial Verification Applications
What can DET do for HMT • Help determine optimal ensemble configuration • Evaluate Ensemble Products • Visualization utilities (Displays) • Probability Products • State-of-the-art probabilistic verification measures
Model Evaluation Tools (MET) DTC Ensemble Testbed (DET) etc…
2010 DTC/HWT Ensemble Activities • Evaluated 26 member CAPS Storm-Scale Ensemble Forecast System during 2010 Spring Experiment • Baselines: HRRR, SREF, NAM • Ensemble Products generated by CAPS– using 15 memberswith radarassimilation • First plots of BrierScore and Area Under Curve
Evaluation of Simple and Derived Ensemble Products: Probability fields using Neighborhood Method Probability matching method for QPF QPF PQPF
Future DTC/HWT Activities • DET participates in planning of CAPS ensemble constituents potentially with DET Ensemble Configuration Module, IC/LBC, and Physics Modules • Real-time verification performed using DET Verification Module (HWT-DTC Collaboration) • HWT products evaluated in DET Module 5 (Products/Display) • Feedback to model developers RETOP
How DET may help HWT • Module 1 - Work with CAPS to define optimal ensemble member configuration • Module 2 – Work with DTC GSI task to identify GSI-based IC perturbation capabilities for CAPS ensemble • Module 3 – Evaluate individual members in 2010 dataset to provide suggestions for physics improvements • Module 4 – Implement bias correction technique that requires limited historical data • Module 5 – Evaluate ensemble products in 2010 dataset • Module 6 – Used in all evaluation
HFIP Ensemble Work Many institutions involved Many approaches being investigated Global Ensembles • Single Models with IC/LBC and physics perturbations (GFS, NOGAPS, and FIM) Regional Ensembles • Multi-model ensemble made up of regional models run by different organizations statistically post-processed (FSU) • Ensembles initialized from EnKF (PSU) Consensus ensembles (NHC)
How DET might contribute to HFIP • DET could be a testing ground for most promising ensemble techniques • Retrospective testing to address specific questions: • Module 2 – Modify near storm environment or storm position? • Module 3 – Which cumulus schemes provide best spread? • Module 4 – Which is best way to bias correct track and intensity? • Module 5 – How do we best display of uncertainty in track and intensity? • Module 6 – Does spatial (object oriented) verification of ensembles improve understanding of output?
Concept of Leveraging DET Modules • Use Near Real-time & Retrospectively • ---------------------------------------- • Immediate Feedback during forecast exercises • ROBUST Test & Eval • of promising techniques uncovered • during forecast exercises MODULE 1: Ensemble Configuration MODULE 2: IC/LBC Perturbations MODULE 3: Physics Perturbations MODULE 4: Statistical Post-Processing Mostly Retrospective -------------------------------------- 1) ROBUST Test & Eval of promising techniques uncovered during forecast exercises MODULE 5: Products/Display MODULE 6: Verification
Questions for Working Group: • Should DET consider capability for realtime (or near realtime) demonstrations? • Where do we place the “overlap area” between DET and other ensemble-related testbeds? What does that area include? • What are the most critical testbed collaboration issues to focus on in the next year or two? • What interactions with other testbeds beyond HMT and HWT should be explored in the next few years?
DTC Ensemble Testbed Model Evaluation Tools (MET) etc…