Service Oriented Atmospheric Radiances (SOAR)

1. M. Halem(1),Ye. Yesha(1), C.Tilmes(2), D.Chapman(1),P. Nguyen(1), J. de La Beaujardiere(2), N. Most(3),K. Stewart(3),A. Bertolli(3) University of Maryland, Baltimore County, (2) NASA Goddard Space Flight Center, (3)Innovim halem@umbc.edu Service Oriented Atmospheric Radiances (SOAR) AGU Presentation December 11, 2007

2. OVERVIEW • Motivation • Gridding Problem • SOAR Services • SOAR AIRS/MODIS Grid Products • Summary/Comments ---------------------------------------------------------------------------------------------- Acknowledgements: Francis Lindsay, NASA ACCESS Program Manager Andy Rindos, IBM Center for Advanced Studies, Manager Pat Gary, GSFC and Jerry Sobieski, UMCP for 10Gb/s fiber network Wen Li Yang, GMU for installing WCS on the UMBC compute cluster Chris Lynnes, GSFC DAAC Manager for enabling AIRS data transfer

3. Motivation • NASA, NOAA and DOD have been collecting and archiving satellite IR radiance data from temperature and moisture sounding instruments for the past 38 years. • Radiance data are assimilated operationally as daily input to weather forecasting models nationally and internationally to improve their forecast accuracies and range of prediction. • Radiance data have been used directly to determine observed global trends in temperature, ozone, albedo, desertification, aerosols, etc.(Spencer et. al. 1990) • Radiance data are used in reanalysis of the past 1/2 century to infer the historical record of global atmospheric states and to validate climate models derived fields (Kalnay et.al. 2000, Charney et. al. 1969) • Gridded data sets for IR spectral radiance measurements of the AIRS and MODIS instruments are not products produced operationally nor are they available for other satellite measurements of IR radiances. • IR Atmospheric Radiance Data are considered one of the most important and versatile climate measurements acquired from space ( Goody & North, 1998).

4. Why should we have gridded data sets on demand? • More than a petabyte of satellite sounding radiance data collected over 3 decades and these data are expected to double again by 2012. • Atmospheric radiance data products are stored at different spatial resolutions, spectral frequencies, formats, units, and archives • Providing averaged gridded data sets is a form of data compression seamlessly transformed from orbital granules or tracks enabling scientists to easily subset, analyze and compare arrays of long records with model outputs and observations from multiple satellites • The simultaneity of AIRS and MODIS IR channels on same platform is a unique opportunity for long term spectral radiance inter comparisons. NPP and NPOESS will provide similar multi-sensor radiance data from CrIS & VIIRS for the foreseeable future • The SOAR system, sponsored by the NASA ACCESS program, provides web based access and tools for multi-sensor gridded AIRS/AMSU and MODIS data records on demand and has demonstrated that these instruments have shown remarkable IR spectral radiance consistency and stability extending more than 5 years • Still need to implement radiometric measurement algorithms for performing convolutions, limb scan adjustments, and statistical analysis tools for multi sensor data on demand • SOAR can simulate what a multi satellite IR spectral radiance mission (CLARREO) might produce as an absolute climate record measurement (NRC Earth Science Decadal Report)

5. What is Gridding problem for AIRS/AMSU FOV?

6. Gridding MODIS radiances within single AIRS FOV

7. Gridding as Data Reduction • AIRS produces ~3*10**6 obs/day for each of 2378 spectral sounding channels or ~7*10**9 Bytes/day • MODIS produces ~6*10**8 obs/day for 16 spectral sounding channels or ~10*10**9 Bytes/day • A 1.0 x 2.0 deg. grid with one average obs. for ~30 ch. per grid element reduces AIRS by ~10**4 and MODIS daily data volume of 16 IR ch by ~10**4. A 0.5 X1.0 deg. grid yields AIRS reduction of ~10**3 and MODIS ~10**3 resp. or ~ 10 MB/day • A 0.5 X0.5 deg. grid is poor man’s AIRS/AMSU/MODIS footprint collocation scheme • Gridded radiances enables scientists to manipulate decades of atmospheric radiance data on demand climate analysis studies on their own laptops. (i.e.~ 40 GB per decade at 0.5 X 1.0 deg. is quite manageable today) .

8. SOAR Gridding Services • SOAR is a publicly accessible web service that provides complex gridding services on-demand for AIRS, AMSU and MODIS radiance data • SOAR removes tedious transformations needed for users to conduct the following climate analysis: • Maps orbital data sets into user specified spatial/ temporal/ spectral lat-lon arrays (I.e. gridded data sets) • Performs arbitrary spatial-temporal subsetting of pre-gridded radiance arrays, • Offers pre-gridded daily arrays of average radiance in grid bin, first footprint in bin, min/max value in bin, brightness temperatures for asc. & desc, orbits • Can deliver higher resolution regional gridding of spectral radiances on demand • Calculates scatter plots, difference fields,multi-sensor radiance plots, and statistics of arbitrary spatial/temporal averaged channels • Creates displays, animations and data delivery services of radiances, BTs, difference fields and other requested products. • SOAR is modular and extensible, so it can add additional services and other IR sensors to the repository building on a common framework • Still needs to include services for spectral convolution matching and limb scan adjustments on demand before gridding.

9. AIRS radiances gridded at 1.0 x 0.5 deg res • On left, AIRS Asc gridded daily averaged radiance data for Jan1, 2005 for ch 2333 • On right, AIRS Desc gridded daily averaged radiance data for Jan 1, 2005 for ch 2333

10. MODIS gridded radiance data at 1.0 x 0.5 deg res. • On left, asc MODIS daily averaged gridded radiance data for Jan 1, 2005 for channel 32 • On right, desc MODIS daily averaged gridded radiance data for Jan 1, 2005 for channel 32.

11. SOAR V1.0 status: Subsetting & Gridding of AIRS and MODIS Data bluegrit.cs.umbc.edu/soar/ id: curt password: this • Developed a manual processing service that remotely accesses level 1B AIRS/AMSU and MODIS data on line over a 10Gb/s fiber optic net from UMBC through UMCP (Dragon Systems) to the G/DAAC and MODAPS (avg rate 440Mb/s). • Processing service averages all granule observations lying in an arbitrary grid box resolving granules crossing poles and longitude boundaries. • Generated 3 years of AIRS global gridded averaged radiances for 324 ch’s, BTs, min/max grid value, asc. & desc., at 0.50 lat x 1.00 lon using IBM blade cluster at UMBC. (Grids 6 days in ~70 min/day in parallel on 12 processors or 12min/day) • Generated 2 months of AQUA MODIS global gridded averaged data at 0.50 x 1.00. and 1 week of TERRA MODIS ( Grids 6 days in ~ 90 min on 12 processors in parallel or 15min/day) MODIS keeps only last month of data on line. • Employed GMU WCS and WMS system to display a daily instrument resolution of AIRS and MODIS BT fields and their differences for processed gridded data fields • SOAR v1.0 has full subsetting, averaging, anomaly creation and display capability for AIRS/AMSU and being extended to deliver MODIS data sets as well.

12. Global fields of AIRS and MODIS BTs • Global AIRS BT’s for Channel 2333(~3.8um) averaged for week of Jan. 1-7,2007 on left. • Global MODIS BT’s for channel 20 (~3.8um) averaged for same week of Jan. 1-7, 2007 on right.

13. Global BT fields of AIRS and MODIS • Global AIRS BTs for channel 528 (~12um) for week of Jan. 1-7, 2005 • Global MODIS BTs for channel 32(~12um) for week of Jan. 1-7,2005

14. Sample Channel Difference Fields for MODIS-AIRS • Difference of MODIS ch 20(~3.8um) -AIRS ch 2333(~3.8um) on left for Jan1-7, 2005. • Difference of MODIS ch 32(~12um) -AIRS ch 528 (~3.8um) on right for Jan1-7,2005. • Color Scale on right

15. AIRS/MODIS ch Radiances and BTs plotted vs Spectral wavelengthfor Jan. 2005 showing spectral consistency for two distinct instruments • Plot on left shows the radiances averaged over 20N to 20S for each of the 16 MODIS IR channels (wavelength) in red and the 324 AIRS operational channels (wavelength) in green for Jan 1 2005. The MODIS value is at the center of the red box. • Plot on right shows the brightness temperatures for the same averaged radiances, for the same region and same channels on the same day and same resp. color codes.

16. AIRS/MODIS Channel Plots • AIRS/MODIS Radiance and brightness temperatures resp. for OCT. 1, 2007 • Consistent stable instrument measurements after 5+ years!

17. 3 Yr Feb 05,06,07 mean and monthlyanomalies: Ch 528

18. 3 Yr Feb 05,06,07 mean and monthlyanomalies: Ch 2333

19. SOAR V2.0 inclusive of Science Servicesalpha.softwarereuse.net/~nsgerard/soar/ any ID & Password • SOAR v2.0 provides gridded AIRS, AMSU and MODIS data in a bounding box for each specified channel averaged over a user specified time window • SOAR v2.0 remotely accesses L1B data on demand from G-DAAC or MODAPS and calculates all data lying in grid element for each element of granule and performs various statistics on data forming the grid. • SOAR v2.0 provides a set of scientific web based services for conducting multi-sensor gridded analysis with arbitrary spectral and spatial resolutions • SOAR v2.0 includes WMS as a mapping service for overlaying and differencing AIRS and MODIS data on global mapped Earth dataset on demand • SOAR v2.0 provides animations of anomalies, difference-maps, trend-lines, and other statistical derivations and maintains a list of the user's requested visualizations • SOAR v2.0 will capture a complete provenance documentation during data ingest and processing that is easily searched and retrieved and provides scientific reproducibility for on demand data sets

20. WMS web server AIRS/MODIS options for displays

21. MODIS ch 32 (12um), AIRS ch 528(12um), & Diff. for single day Diff

22. The table gives the mean MODIS -AIRS BT difference and the extrema. TIME MODIS ch 32 AIRS ch 528 Global MIN AV BT MAX AV BT MinAV BT MAXAV BT Mean(M-A) 2005-01-01T00 185.209 297.947 182.843 298.529 -0.107417 2005-01-01T12 191.348 330.617 194.016 332.876 -0.103135 2005-01-02T00 186.753 298.444 184.100 299.589 -0.182602 2005-01-02T12 188.018 329.164 186.023 332.400 -0.0748267 2005-01-03T00 184.950 297.238 187.726 298.225 -0.107340 2005-01-03T12 191.853 332.165 191.066 334.149 -0.132432 2005-01-04T00 223.608 280.481 183.096 300.198 -0.166676 2005-01-04T12 184.690 331.859 182.096 334.090 -0.0385201 2005-01-05T00 186.180 297.299 191.723 298.239 -0.168761 2005-01-05T12 192.457 331.788 192.568 333.873 -0.0628987 2005-01-06T00 193.218 300.229 191.847 300.873 -0.220639 2005-01-06T12 196.028 331.110 195.323 334.096 -0.0816157 2005-01-07T00 193.182 298.362 192.672 299.223 -0.203587 2005-01-07T12 191.641 331.556 193.265 334.224 -0.0958175 2005-01-08T00 188.707 298.699 189.957 299.496 -0.274076 2005-01-08T12 190.769 329.309 191.787 332.531 -0.159123

23. Global Image Request One-Day Radiance Average Submit Request View Results

24. Global Scattergram Request Monthly Radiance Average (MODIS vs AIRS) Submit Request View Results

25. Global Image Request One-Day Radiance Difference (MODIS-AIRS) Submit Request View Results

26. Summary SOAR Services: Provides multiple gridding options for arbitrary spatial/spectral resolutions Incorporates a variety of instrument science analysis tools as services Provides transparent computing, universal data access and compression SOAR System Framework: Offers interagency interfacing (NASA, NOAA, UMBC) to access multi-sensor data Flexible framework for integrating future missions, e.g. NPP,NPOESS, IASI Generates long term multi-instrument climate data records; AIRS. MODIS. Etc. An educational and scientific tool for climate analytical studies. SOAR Scientific Findings: Relative AIRS and MODIS IR spectral radiance measurements have not degraded in over five years AIRS and MODIS gridded IR spectral radiances have potential to provide long term (10 year) fundamental climate data record Maintains complete provenance for on demand scientific reproducibility

27. Backup

28. Compute & Storage Req. of a 1.0 x 0.5 deg gridded global data set

29. Visualization Services

30. AIRS radiances gridded at 1.0 x 0.5 deg res. • AIRS daily averaged gridded radiance data for asc and desc orbits combined for ch 2333 on Jan 3 2005.