Overview

1. Joint Center for Satellite Data AssimilationOverview of activitiesLars Peter Riishojgaard, JCSDA Director

2. Overview • JCSDA Overview • Background, structure, accomplishments • Current activities • Future plans • Summary JCSDA Science Workshop, NCWCP College Park

3. NOAA/NESDIS NOAA/NWS NASA/Earth Science Division NOAA/OAR US Navy/Oceanographer and Navigator of the Navy and NRL US Air Force/Director of Weather JCSDA Partners, Vision, Mission Vision: An interagency partnership working to become a world leader in applying satellite data and research to operational goals in environmental analysis and prediction Mission: …to accelerate and improve the quantitative use of research and operational satellite data in weather, ocean, climate and environmental analysis and prediction models. JCSDA Science Workshop, NCWCP College Park

4. JCSDA Science Priorities • Radiative Transfer Modeling (CRTM) • Preparation for assimilation of data from new instruments • Clouds and precipitation • Assimilation of land surface observations • Assimilation of ocean surface observations • Atmospheric composition; chemistry and aerosol Overarching goal: Help the operational services improve the quality of their prediction products via improved and accelerated use of satellite data and related research Driving the activities of the Joint Center since 2001, approved by the Science Steering Committee JCSDA Science Workshop, NCWCP College Park

5. JCSDA Mode of operation • Directed research • Carried out by the partners • Mixture of new and leveraged funding • NOAA appropriation comes through NESDIS/STAR • JCSDA plays a coordinating role • External research • Grants or contracts awarded to by one of the JCSDA parent agencies on a rotating basis • Open to the broader research community • Funding awarded competitively, based on peer-reviewed proposals • Visiting Scientist program JCSDA Science Workshop, NCWCP College Park

6. JCSDA accomplishments • Common assimilation infrastructure (NOAA, GMAO, AFWA) • Community radiative transfer model (all partners) • Common NOAA/NASA land data assimilation system (NOAA, GSFC, AFWA) • Numerous new satellite data assimilated operationally, e.g. MODIS (winds and AOD), AIRS and IASI hyperspectral IR radiances, GPSRO sensors (COSMIC, GRAS, GRACE), SSMI/S, Windsat, Jason-2,… • Advanced sensors tested for operational readiness, e.g. ASCAT, MLS, SEVIRI (radiances),… • Ongoing methodology improvement for sensors already assimilated, e.g. AIRS, GPSRO, SSMI/S,… • Improved physically based SST analysis • Adjoint sensitivity diagnostics JCSDA Science Workshop, NCWCP College Park

7. JCSDA accomplishments (II) • OSSE capability in support of COSMIC-2, JPSS, GOES-R, Decadal Survey and other missions • Comprehensive suite of data impact experiments for all major observing systems using NCEP GFS • New supercomputer at GSFC (jointly funded by NASA and NOAA, installed and operated by NASA for the Joint Center) • Part of NOAA/NESDIS-funded supercomputer (S4) located at UW Madison available for JCSDA investigators • Hand-off to NCEP of ATMS data assimilation capability (collaboration between EMC, NESDIS, NASA, JCSDA); implemented in operations on May 22 2012 JCSDA Science Workshop, NCWCP College Park

8. JCSDA Computing • Lack of JCSDA computing “major obstacle to success” (JCSDA Advisory Panel Jan 2009) • Review of past JCSDA-funded external projects revealed lack of computer resources as significant limitation • R2O requires O2R • Research community needs access to operational codes and adequate computer resourcesin order to help • Some resources available on NOAA R&D computer • No projected growth for JCSDA • NASA made initial investment in JCSDA supercomputer to address this problem • IBM Linux cluster; 576 Intel Westmere processors • Immediately augmented by NOAA (GOES-R) to 3456 processors • Located at Goddard, operated by NCCS for the Joint Center • NOAA/NESDIS provides scientific software support JCSDA Science Workshop, NCWCP College Park

9. Jibb (“Joint Center in a Big Box”)Total System Capabilities • Compute – IBM iDataPlex • 3,456 total cores; 37.8 TF Peak Computing • 288 Compute Nodes • Dual-socket, hex-core 2.8 GHz Intel Westmere with 24 GB of RAM • Quad Data Rate Infiniband Network (32 Gbps) in a 2-to-1 blocking fabric • Storage • 8 IBM x3650 Storage Servers • 2 IBM DS3512 Storage Subsystems • 400 TB Total • IBM GPFS File System • Ancillary Nodes • 2 Login Nodes • 2 Management Nodes JCSDA Science Workshop, NCWCP College Park

10. Jibb (II) • System first open to JCSDA users 01/2011 • As of 08/2012 ~50 JCSDA users; many JCSDA applications have been ported on to this system • GDAS porting completed late 2011; NCEP verified scientific integrity of forecast result (comparable to to forecasts run on NOAA R&D platform “vapor”) • Hybrid DA system ported immediately after initial GDAS port • Code management plan between NCEP and JCSDA under development to facilitate two-way code transfers (“R2O and O2R”) JCSDA Science Workshop, NCWCP College Park

11. Examples of Jibb experimentation • NPP transition to operations impact experiments (Casey, Kleist, Collard, …) • General data impact experiments (Jung, Garrett, …) • Next generation NOAA hybrid data assimilation system port (Jung, …) • Data impact experiments (OSE’s) (Jung, Garrett, …) • Hyperspectral IR methodology (Zavodsky,…) • Hyperspectral IR cloudy radiances (Casey, …) • Satellite wind impact experiments (Jung, Santek, Daniels, Nebuda…) • Lightning assimilation methodology (Fuelberg, …)

12. S4 in Support of JCSDA Activities • Overview: • NESDIS, in collaboration with University of Wisconsin is making S4 available to scientists involved in satellite data assimilation and other activities related to it. • Conceptually, half of S4 is dedicated to OSSE (global &regional). • One of the purposes is to consolidate data assimilation activities performed in the research community and to encourage the usage of NOAA operational models/tools/codes installed on it. • Open policy for users (since hosted in academic institution) including JCSDA scientists. • Only open source/non-restricted data allowed on S4. • Brief Technical Description • The S4 system is a Linux cluster (Dell hardware), • 3072 CPU cores in 64 compute nodes with 8TB of total RAM • 520 TB in 26 storage nodes • Quad data rate (40 Gbps) Infiniband interconnects between all compute and storage nodes, • Lustre high performance filesystem for scratch space (4 x 80TB) and data storage (200 TB). • S4 targets primarily the following activities: • Undertake satellite data assimilation experiments at global and/or regional scales and the assessment of their impacts on forecast models skills, using currently flying satellite sensors and • In support of the activity above, undertake all data simulations, calibration, algorithms development/improvement, radiative transfer modeling and validation, quality control (QC) procedures, etc • Perform Observing System Simulation Experiments (OSSEs) for new sensors (such as GOES-R and JPSS). The S4 system, hosted by University of Wisconsin.

13. Examples of S4 Projects • Research of Impact Assessment of SNPP/ATMS and CrIS data on the new hybrid data assimilation system • Readiness for GOES-R data assimilation (ABI, GLM), for High-Impact weather events • GOES-R AMV assimilation readiness • OSSE simulations for DWSS and JPSS • Development of DA capability for SEVIRI volcanic ash • Development of GOES total column ozone assimilation • Community Multi-purpose Formatting Tool (CMFT) • MODIS-and AVHRR-Derived Polar Winds Experiments • Satellite Ocean Data Assimilation Capability Using HYCOM • Assimilation of Microwave and Thermal Infrared Observations of Soil Moisture in LIS

14. R2O requires O2R • In order to facilitate R2O, JCSDA has to make “operational” assimilation systems available to the research community • GDAS/GFS ported to Jibb and S4, skill is benchmarked against NCEP operational platform and NOAA R&D machine • Available to internal and external researchers • Currently T-574 hybrid system being benchmarked on jibb and S4 JCSDA Science Workshop, NCWCP College Park

15. JCSDA/NCEP intercomparison VAPOR – NOAA R&D (Identical to NCEP Operations) S4 – JCSDA / NESDIS / UW Linux cluster JIBB – JCSDA / NASA Linux Cluster From Jung et al 2012, submitted to NESDIS Technical Report, pp 66 JCSDA Science Workshop, NCWCP College Park

16. 500 hPa Anomaly Correlations15 Aug – 30 Sep 2010 No Satellite / No Conventional Data Northern Hemisphere Southern Hemisphere Example of OSE diagnostics (J. Jung, 5th WMO Impact Workshop, Sedona 2012) JCSDA Science Workshop, NCWCP College Park

17. GSI-WRF Integration • Establish working GSI prototype on AFWA HPC • Currently running on Dev 8 • Producing a 48-hr forecast at 12Z cycle • 6-hr init from 6Z cycle • Full DA for 6Z and 12Z run • GSI based WRF initial satellite data assimilation • Initially planned: COSMIC and AMSU-A -- completed • Additionally added: HIRS3, HIRS4, AIRS – completed • Actively working: IASI, WINDSAT, and ASCAT (expect inclusion before IOC)

18. GSI Integration Schedule IOC Working Prototype WRFVAR/GSI Comparison Configure Parallel Runs Conduct Parallel Runs Statistical Analysis DTC Community Capability Cost Analysis Community Funding Decision Phase 2 Decision GSI Operational Transition Optimize AFWA Pre-Processing Configure Code for ops efficiency GSI Sustainment Participate in GSI Review Committee Establish New Satellite Data Flows Phase 1 Dec 11 – Mar 12 Apr 12 – Fall 2012 Apr 12 – 15 May 12 15 May 12 – 1 Aug 12 Aug 12 – Sep 12 Mar 12 – May 12 Mar 12 – May 12 Sep 12 Sep 12 Phase 2 Sep 12 – Jan 13 Sep 12 – Jan 13 Sep 12 – Jan 13 LEGEND Major Activity Planned/In progress Complete Jan 12 Mar 12 May 12 Jul 12 Sep 12 Nov 12 Jan 13 Mar 13 May 13 Jul 13 Sep 13

19. FNMOC and GMAO Observation Impact Monitoring Current Operations Slide courtesy of Ron Gelaro (Fifth WMO Impact Workshop, Sedona, May 2012) http://www.nrlmry.navy.mil/obsens/fnmoc/obsens_main_od.html http://gmao.gsfc.nasa.gov/products/forecasts/systems/fp/obs_impact/ Much larger relative impact of AMVs in Navy system

20. Research to Operations – Suomi NPP ATMS Courtesy of Kevin Garrett, NESDIS/STAR ATMS Ch. 10 AMSU-A Ch. 9 Observed brightness temperatures for NOAA-19 AMSU-A Ch. 9 (left) and ATMS Ch. 10 (right). Coverage for one GDAS cycle. First Guess Departure Tb - Sea (K) Scan dependent brightness temperature bias (observed-simulated) for ATMS Ch. 10 (top) and NOAA-19 AMSU-A Ch. 9 (bottom) Satellite Zenith Angle (degrees) • JCSDA ATMS OSE • Port GSI Hybrid to S4/JIBB supercomputers (with help from NCEP) • High resolution OSE • ATMS data impact on forecast is neutral • ATMS data available since 11/8/2011 from NDE (NESDIS/ESPC support) • OSEs run from Dec 2011-May 2012 • ATMS data assimilation capability delivered with GFS/GDAS update to NCO May 22, 2012. OSE Results (TOP) 500 mb Height Anomaly Correlation for 00z forecasts between Jan. 16, 2012 – April 28, 2012 in the Tropics. (RIGHT) Equitable Threat Score over CONUS 36-60 hr forecast • NCEP/STAR/JCSDA agreement for ExpeditedR2O of Suomi NPP ATMS in the GSI • Management support for collaboration, resource and responsibilities allocation • NCEP/EMC integration of ATMS data in GSI Hybrid • ATMS data processing, quality control, spatial averaging • Low-resolution OSE JCSDA Science Workshop, NCWCP College Park

21. Community Radiative Transfer Model (CRTM) Joint Center for Satellite Data Assimilation Courtesy of Quanhua Liu, NESDIS/STAR CRTM Mission Satellite radiance simulation and assimilation for passive MW, IR, and Visible sensors of NOAA,NASA,DoD satellites, and others(200 sensors) CRTM Applications Data assimilation in supporting of weather forecasting Physical retrieval algorithm for satellite products Stability and accuracy monitoring of satellite observations Education and Research: reanalysis, climate studies, air quality forecasting, and a radiative tool for students CRTM Future Development SBUV for ozone data, JPSS ATMS,CrIS,VIIRS,OMPS, GPM,Fy3,Studying feasibility of active sensors including Radar and Lidar space measurements Derber et al., 2007

22. Example of data impact study (CRTM v. 1 to v. 2 upgrade) Courtesy of Fuzhong Weng, NESDIS/STAR 500 hPa geopotential height anomaly correction from 01/09/2008-02/22/2008 New implementation meets basic “do no harm” criterion Large impact in the SH (more sensitive to satellite data) JCSDA Science Workshop, NCWCP College Park

23. Impact of COSMIC methodology update Courtesy of Cuccurul, NCEP/EMC • 500 hPa height AC scores as a function of the forecast for the 500 hPa heights in the Southern Hemisphere • 40-day experiments: • expx (NO COSMIC) • cnt (old RO assimilation code - with COSMIC) • exp (updated RO assimilation code - with COSMIC) COSMIC provides 8 hours of gain in model forecast skill starting at day 4 JCSDA Science Workshop, NCWCP College Park

24. Preparation for data from new sensors Goal is to have operational users ready to assess data from new sensors from day 1 assimilate data from new sensors within one year from launch Current activities include NPP (ATMS, CrIS, VIIRS, OMPS) in close collaboration with NESDIS/STAR, NCEP/EMC Aquarius SMAP GPM GOES-R (ABI, GLM) … JCSDA Science Workshop, NCWCP College Park

25. Colloquia and Workshops • Fifth WMO Workshop on the Impact of Various Observing Systems in NWP • Major international venue for data impact studies • Hosted by JCSDA for WMO in Sedona, AZ, May 22-25 2012 • 59 participants from 13 countries(strong US/JCSDA representation) • 40 scientific presentations, ample discussion of results • Recommendations captured in Final Report from Workshop to be published by WMO • Second JCSDA Summer Colloquium, July 23 – Aug 2 2012 • Major training event in satellite data assimilation • Two weeks, 19 participants, 16 lecturers • Particpants were Ph.D. students and post-docs, mostly from US educational institutions or employed in the US • Formal lectures by invited lecturers and informal presentation by participants JCSDA Science Workshop, NCWCP College Park

26. Future plans (I) • Hyperspectral IR water vapor radiances in NCEP operations • Additional impact experiments • CrIS implementation • Better utilization of SATWINDs • MetOp-B preparation • SMAP • GPM • … JCSDA Science Workshop, NCWCP College Park

27. Future Plans (II) • Additional OSSEs • New high-resolution Nature Run required • External research announcement by NOAA in FY2013 • New model for rotation of External Research Announcement • Examine possibilities for updating jibb/S4 to new high-resolution DA systems • JCSDA Symposium at 2013 AMS Annual Meeting JCSDA Science Workshop, NCWCP College Park