在科研中更好的使用美国业务资料分析系统（ GSI ）

1. 在科研中更好的使用美国业务资料分析系统（GSI）在科研中更好的使用美国业务资料分析系统（GSI） Ming Hu*Developmental Testbed Center, NCAR-NOAA/GSD**Cooperative Institute for Research in Environmental Sciences, Colorado University at Boulder

2. Outline • General introduction to the GridpointStatistical Interpolation (GSI) system • Development Testbed Center and Community GSI • History, repository, test, and review committee • Available GSI resources for community • O2R: website, document, tutorial, and help desk • R2O: community contributions • Current and future development • Community GSI and operational GSI

3. General introduction to the GridpointStatistical Interpolation system This section is mainly based on John Derber’s talk in 2013 summer GSI tutorial

4. Overview of GSI John C. Derber NOAA/NWS/NCEP/EMC

5. History • The Spectral Statistical Interpolation (SSI) analysis system was developed at NCEP in the late 1980’s and early 1990’s. • Main advantages of this system over OI systems were: • All observations are used at once (much of the noise generated in OI analyses was generated by data selection) • Ability to use forward models to transform from analysis variable to observations • Analysis variables can be defined to simplify covariance matrix and are not tied to model variables (except need to be able to transform to model variable) • The SSI system was the first operational • variational analysis system • system to directly use radiances

6. History • While the SSI system was a great improvement over the prior OI system – it still had some basic short-comings • Since background error was defined in spectral space – not simple to use for regional systems • Diagonal spectral background error did not allow much spatial variation in the background error • Not particularly well written since developed as a prototype code and then implemented operationally

7. History • The Global Statistical Interpolation (GSI) analysis system was developed as the next generation global/regional analysis system • Wan-Shu Wu, R. James Purser, David Parrish • Three-Dimensional Variational Analysis with spatially Inhomogeneous Covariances. Mon. Wea. Rev., 130, 2905-2916. • Based on SSI analysis system • Replace spectral definition for background errors with grid point version based on recursive filters

8. History • Used in NCEP operations for • Regional • Global • Hurricane • Real-Time Mesoscale Analysis • Rapid Refresh (ESRL/GSD) • Operational at AFWA • GMAO collaboration • Modification to fit into WRF and NCEP infrastructure • Evolution to ESMF

9. General Comments • GSI analysis code is an evolving system. • Scientific advances • situation dependent background errors -- hybrid • new satellite data • new analysis variables • Improved coding • Bug fixes • Removal of unnecessary computations, arrays, etc. • More efficient algorithms (MPI, OpenMP) • Bundle structure • Generalizations of code • Different compute platforms • Different analysis variables • Different models • Improved documentation • Fast evolution creates difficulties for slower evolving research projects

10. General Comments • Code is intended to be used Operationally • Must satisfy coding requirements • Must fit into infrastructure • Must be kept as simple as possible • External usage intended to: • Improve external testing • Reduce transition to operations work/time • Reduce duplication of effort

11. Simplification to operational 3-D for presentation • For today’s introduction, I will be talking about using the GSI for standard operational 3-D var. analysis. Many other options available or under development • 4d-var • hybrid assimilation • observation sensitivity • FOTO • Additional observation types • SST retrieval • Detailed options • Options make code more complex – difficult balance between options and simplicity

12. Basic analysis problem J = Jb + Jo + Jc J = (x-xb)TB-1(x-xb) + (H(x)-y0)T(E+F)-1(H(x)-y0) + JC J = Fit to background + Fit to observations + constraints x = Analysis xb = Background B = Background error covariance H = Forward model y0 = Observations E+F = R = Instrument error + Representativeness error JC = Constraint terms

13. Analysis variables • Background errors must be defined in terms of analysis variable • Streamfunction (Ψ) • Unbalanced Velocity Potential (χunbalanced) • Unbalanced Temperature (Tunbalanced) • Unbalanced Surface Pressure (Psunbalanced) • Ozone – Clouds – etc. • Satellite bias correction coefficients

14. Analysis variables • χ = χunbalanced + A Ψ • T = Tunbalanced + B Ψ • Ps = Psunbalanced + C Ψ • Streamfunction is a key variable defining a large percentage T and Ps (especially away from equator). Contribution to χ is small except near the surface and tropopause.

15. Background fields • Current works for following systems • NCEP GFS • NCEP NMM – binary and netcdf • NCEP RTMA • NCEP Hurricane (not using subversion version yet) • GMAO global • ARW – binary and netcdf – (not operationally used yet RR - GSD) • FGAT (First Guess at Appropriate Time) enabled up to 100 time levels

16. Background Errors • Three paths – more in talk by D. Kleist • Isotropic/homogeneous • Most common usage. • Function of latitude/height • Vertical and horizontal scales separable • Variances can be location dependent • Anisotropic/inhomogeneous • Function of location /state • Can be full 3-D covariances • Still relatively immature • Hybrid

17. Observations • Observational data is expected to be in BUFR format (this is the international standard) • Each observation type (e.g., u,v,radiance from NOAA-15 AMSU-A) is read in on a particular processor or group of processors (parallel read) • Data thinning can occur in the reading step. • Checks to see if data is in specified data time window and within analysis domain

18. Regional AMSU-A NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 METOP-A Channels1-6, 8-13, 15 AQUA Channels 5, 8-13 Thinned to 60km AMSU-B/MHS NOAA-15 Channels 1-3, 5 NOAA-18 Channels 1-5 METOP-A Channels 1-5 Thinned to 60km HIRS NOAA-17 Channels 2-15 METOP-A Channels 2-15 Thinned to 120km AIRS AQUA 148 Channels Thinned to 120km  IASI METOP-A 165 Channels Global all thinned to 145km GOES-15 Sounders Channels 1-15 Individual fields of view 4 Detectors treated separately Over ocean only AMSU-A NOAA-15 Channels 1-10, 12-13, 15 NOAA-18 Channels 1-8, 10-13, 15 NOAA-19 Channels 1-7, 9-13, 15 METOP-A Channels 1-6, 8-13, 15 AQUA Channels 6, 8-13 ATMS NPP Channels 1-14,16-22 MHS NOAA-18 Channels 1-5 NOAA-19 Channels 1-5 METOP-A Channels 1-5 HIRS METOP-A Channels 2-15 AIRS AQUA 148 Channels IASI METOP-A 165 Channels Input data – Satellite currently used

19. Radiosondes Pibal winds Synthetic tropical cyclone winds wind profilers conventional aircraft reports ASDAR aircraft reports MDCARS aircraft reports dropsondes MODIS IR and water vapor winds GMS, JMA, METEOSAT and GOES cloud drift IR and visible winds GOES water vapor cloud top winds Surface land observations Surface ship and buoy observation SSM/I wind speeds QuikScat and ASCATwind speed and direction SSM/I and TRMM TMI precipitation estimates Doppler radial velocities VAD (NEXRAD) winds GPS precipitable water estimates GPS Radio occultation refractivity and bending angle profiles SBUV ozone profiles and OMI total ozone Input data – Conventional currently used

20. Simulation of observations • To use observation, must be able to simulate observation • Can be simple interpolation to ob location/time • Can be more complex (e.g., radiative transfer) • For radiances we use CRTM • Vertical resolution and model top important

21. Quality control • External platform specific QC • Some gross checking in PREPBUFR file creation • Analysis QC • Gross checks – specified in input data files • Variational quality control • Data usage specification (info files) • Outer iteration structure allows data rejected (or downweighted) initially to come back in • Ob error can be modified due to external QC marks • Radiance QC much more complicated. Tomorrow!

22. Useful References • Wan-Shu Wu, R. James Purser and David F. Parrish, 2002: Three-Dimensional Variational Analysis with Spatially Inhomogeneous Covariances.Monthly Weather Review, Vol. 130, No. 12, pp. 2905–2916.  • R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part I: Spatially Homogeneous and Isotropic Gaussian Covariances.Monthly Weather Review, Vol. 131, No. 8, pp. 1524–1535.  • R. James Purser, Wan-Shu Wu, David F. Parrish and Nigel M. Roberts, 2003: Numerical Aspects of the Application of Recursive Filters to Variational Statistical Analysis. Part II: Spatially Inhomogeneous and Anisotropic General Covariances.Monthly Weather Review, Vol. 131, No. 8, pp. 1536–1548.  • McNally, A.P., J.C. Derber, W.-S. Wu and B.B. Katz, 2000: The use of TOVS level-1B radiances in the NCEP SSI analysis system.  Q.J.R.M.S., 126, 689-724. • Parrish, D. F. and J. C. Derber, 1992: The National Meteorological Center's spectral statistical interpolation analysis system. Mon. Wea. Rev., 120, 1747 - 1763. • Derber, J. C. and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 2287 - 2299. • Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Wu, Wan-Shu; Lord, Stephen , Introduction of the GSI into the NCEP Global Data Assimilation System, Weather and Forecasting. Vol. 24, no. 6, pp. 1691-1705. Dec 2009 • Kleist, Daryl T; Parrish, David F; Derber, John C; Treadon, Russ; Errico, Ronald M; Yang, Runhua, Improving Incremental Balance in the GSI 3DVAR Analysis System, Monthly Weather Review [Mon. Weather Rev.]. Vol. 137, no. 3, pp. 1046-1060. Mar 2009. • Kazumori, M; Liu, Q; Treadon, R; Derber, JC, Impact Study of AMSR-E Radiances in the NCEP Global Data Assimilation System Monthly Weather Review,Vol. 136, no. 2, pp. 541-559. Feb 2008. • Zhu, Y; Gelaro, R, Observation Sensitivity Calculations Using the Adjoint of the Gridpoint Statistical Interpolation (GSI) Analysis System, Monthly Weather Review. Vol. 136, no. 1, pp. 335-351. Jan 2008. • DTC GSI documentation (http://www.dtcenter.org/com-GSI/users/index.php)

23. DTC comments • GSI is an operation system • Well-tested for operation applications • Very strict data QC • Advanced operator for radiance • Bias correction is very important radiance data analysis • Model top and channel selection • Data coverage for regional application • PrepBUFR and NCEP QC procedure • The operation system supported by DTC for community users

24. DTC and Community GSI Who is DTC Community GSI: Goal and Building Community GSI repository Test and evaluation GSI review committee

25. Who is DTC ? • Developmental Testbed Center (DTC) • http://www.dtcenter.org/ • Bridge between research and operational community • NWP systems supported by DTC • WRF, HWRF, GSI, MET, …

26. Community GSI: Goals • Provide current operation GSI capability to research community (O2R) • Provide a pathway for research community to contribute operation GSI (R2O) • Provide a framework to enhance the collaboration from distributed GSI developers • Provide rational basis to operational centers and research community for enhancement of data assimilation systems

27. Community GSI: Building • O2R: • Code portability: well-tested GSI code package for release • Documentation: GSI User’s Guide • GSI User’s Webpage • Annual release • On-site Tutorial (lectures and Practical cases) • Help desk • R2O: • Repository: Build, maintain, sync, test • Commit community contributions • GSI Review Committee The important role of GSI repository and review committee

28. GSI Code Repository Branch GSI Trunk NDAS GDAS Community release HWRF • Use tags or branches for: Release, new development, bug fix … • Applications may use different revisions in the trunk (“snapshot”). Tag Branch Branch • Which GSI should I use ? • There is no “DTC GSI”, “EMC GSI” or “global GSI”. There is only one GSI! For a researcher, community release should be sufficient to use. If you are interested in getting new development back to the GSI trunk, contact GSI helpdesk (gsi_help@ucar.edu) get access to the developmental version of GSI. Branch

29. GSI Review Committee Branch GSI Trunk NCEP/EMC NASA/GMAO NOAA/ESRL NCAR/MMM AFWA DTC NESDIS GSI Review Committee • Represent research and operational community • Coordinate distributed GSI development • Conduct GSI Code Review Tag Branch Branch Branch • DTC is working as the pathway between the research community and operational community. • Provide visitor program for potential contributions to the operational code. • Provide equivalent-operational tests of community contributions and provide rational basis for operational implementations.

30. History • June, 2006:North American Mesoscale (NAM) System, NCEP • May, 2007: Global Forecast System (GFS), NCEP • 2007: DTC started to document GSI. • 2009: • Created GSI code repository over NCEP and DTC • First GSI release V1.0 • Started user support • 2010: • First Community GSI tutorial • Created GSI Review Committee • 2011 • First Community GSI workshop

32. Available GSI resources for community: O2R User’s Webpage Documentation Annual release package Annual residential tutorial Help desk

33. Community GSI – User’s Page • Support mainly through User’s Page and help desk: http://www.dtcenter.org/com-GSI/users/index.php

34. Community GSI Release and Tutorial The GSI User’s Guide and the on-line tutorial match with official release code Summer 2013 GSI Tutorial: NCEP, Aug. 5-7, 2013

35. Community GSI Release Source code and fixed files were based on: the GSI EMC trunk r19180 (May 10, 2012) the community GSI trunk r826 The GSI User’s Guide and the on-line tutorial cases are updated with each official release.

36. Community GSI - Documents • User’s Guide • Matching with each official release • Tutorial lectures • Code browser • Calling tree • Publications

37. Community GSI- User’s Guide V3.2

38. Community GSI- User’s Guide V3.2

39. Community GSI - Practice • On-line tutorial for each release • Residential tutorial practice cases

40. Community GSI – tutorial and workshop

41. Community GSI - Help desk • gsi_help@ucar.edu • Any GSI related questions • Most of questions were answered by DTC staff • Forward complex questions to GSI developers

42. Available GSI resources for community: R2O Commit the community contributions to trunk Help desk

43. GSI R2O Transition Procedure (2011 Implementation) • GSI Review Committee Scientific Review • Development, testing and merging • GSI Review Committee code and commitment review Community research Code development candidate 1 Code commitment candidate (Branches) 2 Code in repository trunk 3

44. GSI R2O applications • DTC contributes to R2O transition, coordinate & assist code commitment: • MariuszPagowski: Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model • DTC: Portability issues from trunk head testing • MariuszPagowski: Add WRF-Chem model file as background for PM2.5 assimilation • Tom Augline: Add control and state variables for Cloudy analysis (triggered more work on the bundles of GMAO) • Zhiquan Liu: aerosol work has been merged to the trunk • GSD: Recent GSI developments for RR

45. Collaboration • HFIP program: BUFR processing training • AMB branch: GSI application for Rapid Refresh • MariuszPagowski (AMB): Assimilation of surface PM2.5 observations in GSI for CMAQ regional air quality model • Don Birkenheuer (FAB): Thorpex work using GSI • EMC GSI group (feedback from community GSI users) • NCAR/MMM DA group (hybrid and cloud analysis,, chemistry analysis ) • Space Weather Prediction Center /NOAA (Houjun Wang ) (whole atmosphere data assimilation using GSI) • Other GSI users (OU regional ENKF, PSD (Jeff Whitaker) global ENKF …)

46. R2O Summary • DTC also works with researchers to bring research community contributions back to the GSI operation repository (R2O) • Create Trac to keep community researchers in developing loop • Help researchers to review and merge the code on top of the trunk • Test the code with regression test suite and initial the code review for committing • Send your questions to gsi_help@ucar.edu

47. Current and future development Community GSI Operational GSI August 9th, 2013 Committee Meeting Agenda 8:45 GMAO 
9:30 NESDIS 
10:15 break 
10:25 ESRL 
11:10 NCAR 
11:55 lunch
12:40 DTC 
1:10 NCEP 
3:00 Adjourn

48. Community GSI • Maintaining the current support • GSI review committee • Community GSI trunk: sync, test • Annual release with code and updated document • Improving GSI user’s Website • Holding on-site tutorial • Maintaining Helpdesk • Help committing community contributions • Application tools and new instructions to GSI (bundle, hybrid, …) • Build operation-similar test system (ongoing)

49. Ongoing and future EMC GSI development work EMC GSI development team Update Aug. 8, 2013

50. Use of observations • Prepare for METOP-B and CrIS implementation - Andrew Collard • Implementation Aug. 20 to WCOSS • Includes SEVIRI from Meteosat-10 • Use of AURA MLS ozone profiles – Haixia Liu • Testing of new version of real time data underway • Cloudy Radiance Assimilation – Emily Liu, Yanqiu Zhu, Z. Zhang, Will McCarty (GMAO) • Updates to trunk – Significant progress – Testing without convection • Improved structure of radiance diagnostic files – Andrew Collard - TBD • Satellite wind usage upgrade –XiuJuan Su • GOES hourly wind project • Ground based Doppler radar data – Shun Liu • Conversion to dual-pol • Aircraft based Doppler radar data – MingjingTong • In operations on WCOSS • SSMI/S data – Andrew Collard, Emily Liu, and Ellen Liang (NESDIS) • To be included in Spring 2014 implementation • Mesoscale surface observation – Manuel Pondeca, Steve Levine • GLERL surface reduction for lakes under testing • GOES-R/SEVERI radiances – Haixia Liu • Additional quality control under development • Channels used above low clouds • CRTM – Paul VanDelst, David Groff, Quanhua Liu(NESDIS), Yong Chen(NESDIS) • Improvements continue • Improve use of significant levels in RAOB profiles – Wan-shu Wu • Bug fixed