1 / 22

Data Preparation for Atmospheric and Geosciences Research

Access high-quality observational datasets and research data archives for atmospheric and geosciences research. Comprehensive services and support from the NCAR Data Support Section.

briannar
Download Presentation

Data Preparation for Atmospheric and Geosciences Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Plans and Progress Data Preparation CFS Reanalysis

  2. CISL Research Data ArchiveData for Atmospheric and Geosciences Research CISL Research Data ArchiveData for Atmospheric and Geosciences Research CISL Research Data ArchiveData for Atmospheric and Geosciences Research CISL Research Data ArchiveData for Atmospheric and Geosciences Research NCAR Data Support Section (and Roy Jenne) supplied high quality observations for R1, R2, ERA40, JRA25, NARR, MERRA, & CFSRR Services Services Services Services ftpService ftpService ftpService ftpService Datasets Datasets Datasets Datasets OtherResources OtherResources OtherResources OtherResources SpecialProjects SpecialProjects SpecialProjects SpecialProjects Staff Staff Staff Staff Documentation Documentation Documentation Documentation Software Software Software Software Left to Right: Chi-Fan Shih, Will Spangler, Joey Comeaux, Bob Dattore, Steven Worley, Dave Stepaniak, Hua Ji, Doug Schuster, Gregg Walters, Cecilia Banner

  3. CFSRR Input Datasets Available from NCDC Archives TOVS, SSM/I GOES-IR FROM CLASS COADS A number of NCDC datasets provided via NCAR

  4. The History of Conventional Data Preparation for Reanalyses at NCEP, 1994-2007 • TEMP PILOT AIREP PIREP ACARS SATOB SYNOP MARINE BOGUS • R1/R2 merged data sources from NCAR, NCDC, NCEP, JMA, and ECMWF • R1/R2 merged qc’d datasets were packaged and delivered to the JMA-25 project • R1/R2 datasets were reprocessed and enhanced by NCEP/NCAR for ERA-40 • NCEP obtained the ECMWF MARS archive of conventional data, 1979-1994 • NCAR improved versions of key datasets, ie, COADS, ON29/RAOBS, ON124 • R1/R2 and new datasets were merged for the North American Regional Reanalysis • The latest versions of all source datasets were merged for the MERRA project • Much effort was made by MERRA people to find data gaps and other problems • CFSRR will use conventional observations prepared for MERRA, 1978-2000 • CFSRR will use NCEP/NCO operational archives to overlay and complete 1997-2007

  5. List of Radiosonde/Pibal Datasets Collected at NCEP for Reanalysis 1920 1930 1940 1950 1960 1970 1980 1990 2000 --------------------|----|---------|---------|---------|---------|---------|---------|---------|---------| basua-----|1948-2005|----|----|----|----|----|----|--***************************************************** cards-----|1946-1966|----|----|----|----|----|----|*********************---|----|----|----|----|----|----| china-----|1954-1962|----|----|----|----|----|----|----|---*********--|----|----|----|----|----|----|----| countries-|1946-1993|----|----|----|----|----|----|************************************************-|----| france----|1948-1979|----|----|----|----|----|----|--********************************|----|----|----|----| gate------|1974-1974|----|----|----|----|----|----|----|----|----|----|----|---*|----|----|----|----|----| jma-------|1978-1994|----|----|----|----|----|----|----|----|----|----|----|----|--*****************|----| lie-------|1967-1967|----|----|----|----|----|----|----|----|----|----|-*--|----|----|----|----|----|----| mars------|1979-1994|----|----|----|----|----|----|----|----|----|----|----|----|---****************|----| mit-------|1958-1963|----|----|----|----|----|----|----|----|--******-|----|----|----|----|----|----|----| navyship--|1985-1996|----|----|----|----|----|----|----|----|----|----|----|----|----|----************---| navyua----|1966-1973|----|----|----|----|----|----|----|----|----|----|********-|----|----|----|----|----| on20------|1962-1972|----|----|----|----|----|----|----|----|----|-***********--|----|----|----|----|----| on29------|1973-2000|----|----|----|----|----|----|----|----|----|----|----|--**************************** permships-|1973-1980|----|----|----|----|----|----|----|----|----|----|----|--********----|----|----|----| ptarmigan-|1950-1961|----|----|----|----|----|----|----************---|----|----|----|----|----|----|----| ruships---|1947-1978|----|----|----|----|----|----|-********************************-|----|----|----|----| rusice----|1954-1978|----|----|----|----|----|----|----|---*************************-|----|----|----|----| rusnp-----|1950-1991|----|----|----|----|----|----|----******************************************---|----| rusua-----|1960-1990|----|----|----|----|----|----|----|----|----*******************************----|----| safrica---|1961-1967|----|----|----|----|----|----|----|----|----|*******--|----|----|----|----|----|----| scherhag--|1954-1962|----|----|----|----|----|----|----|---*********--|----|----|----|----|----|----|----| td52------|1922-1971|----|-**************************************************---|----|----|----|----|----| td53------|1919-1969|---***************************************************|----|----|----|----|----|----| td54------|1943-1974|----|----|----|----|----|--********************************|----|----|----|----|----| td90------|1943-1962|----|----|----|----|----|--********************--|----|----|----|----|----|----|----| twerle----|1975-1976|----|----|----|----|----|----|----|----|----|----|----|----**---|----|----|----|----| usafua----|1973-1980|----|----|----|----|----|----|----|----|----|----|----|--********----|----|----|----| usctrl----|1946-1969|----|----|----|----|----|----|************************|----|----|----|----|----|----| --------------------|----|---------|---------|---------|---------|---------|---------|---------|---------| 1920 1930 1940 1950 1960 1970 1980 1990 2000

  6. A quantitative estimate of duplication in source archives Conventional observation counts. R3 is current collection of source archives

  7. Except for TOVS, these datasets form the basis of the CFSRR conventional obs 1979-1997 After 1997 the NCEP CDAS and OPS archives are used as conventional data sources VTPR TOVS TEMP/PILOT SYNOP AIRCFT COADS SATOB PAOB

  8. A BUFR data structure was designed for R1 to record feedback information in the observations which is still used in operations and the CFSRR as well analysis 1st guess ob error “event” stack The diagram illustrates how processing information such as QC results, and observation modifications are recorded in the BUFR files during the execution of the reanalysis. One temperature level is shown as an example.

  9. TOVS, ATOVS, AQUA Archives for CFSRR TOVS HIRS/2 and MSU 1-B orbital files were obtained from NESDIS/SAA spanning all satellites for the years 1978-1997 for the NARR project. Synoptic 1-C BUFR brightness temperature files were created from the 1-B archive. The NESDIS/NCDC CLASS server has since become operational and is a great resource for satellite data. Available data is constantly being expanded. Some data not available from CLASS may be obtained directly from NCDC. For CFSRR, the NARR HIRS/2 and MSU 1-B archives are enhanced, and new holdings extended through to the present. The complete set of SSU orbit files were added to the archives, as well as AMSU-A, AMSU-B, HIRS/3, HIRS/4, MHS, AIRS, and AMSR-E data. Data sets were obtained through CLASS and NCEP archives. New codes to convert raw orbit files to calibrated synoptic BUFR BT files are created for MSU, and SSU. Improvements include new calibrations for MSU NOAA-10-14 (Cheng-Zhi Zou), and more use of the imbedded QC data found in the orbital files.

  10. CFSRR TOVS-ATOVS-AQUA Satellite Data Timelines NCEP CLASS AIRS AMSR-E AMSU-A AMSU-B HIRS/2 HIRS/3 HIRS/4 MHS MSU SSU 1980 1990 2000 2010

  11. Defense Met Sat Program Special Sensor Microwave Imager SSM/I Sensor Data Records SDR orbits ssmit synoptic BUFR files spssmi ocean surface wind speed BUFR files

  12. Availability of GOES data in CLASS is shown in the following table.  High density SNDR and IMGR data is available in CLASS starting with GOES-8 in 1994. With some help from NESDIS/NCDC with cloud processing, we could hope to go at least that far back. There was an IR sounder (VAS) on GOES-4-7 satellites, but whether anything can be done with them for reanalysis is currently undetermined.

  13. NESDIS V8 SBUV TOTAL AND PROFILE OZONE RETRIEVALS, 1978-2007 V8 is the most recent version of the backscattered ultraviolet (BUV) algorithms that have undergone three decades of progressive refinement. Its predecessors, V7 for total ozone, developed about in 1998, and V6 for profile ozone, developed in about 1990, have been used to produce the acclaimed TOMS total ozone and SBUV(/2) ozone profile time series. V8 will correct several small errors in V7 and V6 that were discovered by extensive error studies using radiative transfer models and by comparison with ground-based instruments. The V8T uses only two wavelengths (318 nm and 331 nm) to derive total ozone. Other wavelengths are used for diagnostics and error correction. Experience with TOMS and SBUV/2 suggests that the algorithm is capable of producing total ozone with rms error of about 2%, though these errors are not necessarily randomly distributed over the globe. The errors typically increase with solar zenith angle and in presence of heavy aerosol loading.

  14. Subset of the SBUV ozone retrievals selected for CFSRR a – ascending node d – descending node

  15. This shows when data types enter the CFSRR stream TEMP/PILOT SYNOP AIRCRAFT MARINE SATOB BOGUS VADWND PROFILR METAR ERSWND MSONET SSMIWND V8 OZONE QKSWND ACARS 1980 1990 2000 2010 AIRS GOESND 5X5 AMSRE HIRS4 MHS HIRS MSU SSU GPSIPW HIRS3 AMSUA AMSUB GEOIMR GOESND 1X1

  16. Tropical Cyclone Synthetic Wind Data 1978-2002 CSFRR will use Mike Fiorino’s TC Bogus wind dataset as an aid to locating storm vortices in the GSI analysis. The synthetic winds consist of a central location surrounded by four wind profiles.

  17. SEA ICE for CFSRR NASA GSFC recomputed (for instrument drifts) from Oct 1978 to approx 1996, when it will turn over to the NCEP ice. The NCEP ice will be recomputed with present analysis methods, land masks, algorithms, etc. from 1995/10/01 to 2000/02/25 from prior grids of brightness temperatures.  From 2/25/2000 to present using archived SSM/I passes. The GLERL Great Lakes analysis will be used for the historic coverage of those lakes.  NIC Great Lakes will continue this towards the present. GSFC analysis does not cover all ice covered areas of our interest. Those (great lakes, peripheral (to the arctic) lakes and seas) will be filled by similar method to NARR -- identifying proxy points and filling from there.  Proxies will be found from the 2/25/2000 to present period, where there is full data available and the procedure can be continued into the future. Half degree ice will be produced consistently throughout the period (vs. Reanalysis 1 switching from Nomura 1 degree to NCEP 1 degree to NCEP 0.5 degree).  Starting with 2/25/2000, a 5' lat-long grid will also be available. Where there are temporal gaps, they'll be filled by linear interpolation in time.  This is particularly a problem for the Great Lakes.  It is not currently a concern, with 3 SSM/I instruments flying, and AMSR-E also providing comparable data for any area which can be analyzed by automated means.

  18. Ingest of Snow Analyses for CFSRR Current Ops T382 GDAS ingests 2 daily snow analyses global Air Force analysis (Air Force non-grib format, one file per hemisphere) N. Hemisphere NESDIS 4-km analysis (known as IMS) in grib format ops job-step (known as SNOW2MDL) by George Gayno of EMC merges the two snow analyses above and maps merged anal toT382 GFS grid (or any GFS grid) CFS RR will assimilate both Air Force and IMS (as in Ops) for those years that both are available (1997-present) & only Air Force analysis before 1997 Using SNOW2MDL job step, modified to account for data availability below 23-km resolution until about 2005 (date TBD), then 4-km resolution thereafter IMS 23 km analysis available from Feb 1997 through the present IMS 4 km analysis available from Feb 2004 through the present George Gayno has modified his job SNOW2MDL to be able to use only Air Force analysis prior to the 1997 start of NESDIS IMS analysis Air Force analysis available for all years needed 1979 to 1998 in Grib format from COLA M. Fennessy of COLA obtained Air Force analyses for above period from NCDC/ETAC and then gribbed it with help from EMC 2002 to present in Air Force non-grib format used in present ops (via EMC archive of Jesse Meng 1999-2001 (to be extracted from NCO run history files) Jan-Oct 1999 period may need special attention due to CRAY fire impact on archives Option 1: Use archived GDAS “snogrb” file Option 2: Obtain 1999 Air Force analysis from NCDC/ETAC archive and have COLA grib it

  19. Some SST Options for CFSRR • OI 1/4 Degree Daily Analysis from NCDC • AVHRR only satellite source thru May 2002 • Option of including AMSR June 2002 onward, greatly improving satellite coverage in cloudy regions(especially important for daily analysis) • May not be available in time: • Currently 1985-present, undergoing revision (perhaps available by October 2007, per Dick Reynolds) • Nov 1981-Dec 1984, would need to first compute daily AVHRR SST retrievals. (Pathfinder Project if funded, or Reynolds himself) • OIv2 1-degree Weekly Analysis • Lower resolution and no AMSR satellite data • More consistent • Available Nov 1981 – present (AVHRR period only) • If necessary, consider spinning backwards to required start date, using satellite quality field as initial first guess to retain some gradients

  20. Some data sets we still hope to get for CFSRR Reprocessed GMS winds used in JRA-25 (NCAR DSS) ERS-1 and ERS-2 winds prior to mid-1997 (NCAR DSS) Reprocessed GPS Occultation data from COSMIC (NCAR) GOES SOUNDER 1x1 radiances (CLASS)

More Related