The Land, Atmosphere Near-real-time Capability for EOS (LANCE)

1. The Land, Atmosphere Near-real-time Capability for EOS(LANCE) Ed Masuoka Code 614.5 GESDIS Combined Branch Meeting 2-2-2010

2. What is LANCE? • Building on existing EOSDIS elements (MODAPS/LAADS; OMI SIPS, GES DISC), provides data from MODIS, OMI, AIRS, MLS, and AMSR-E instruments in near real-time (< 3 hours from observation) • Utilizes PGE Code from Standard Science Products, but: • All products generated with a “nrt” extension to distinguish them from standard products • Requirements for ancillary data inputs have been relaxed (e.g., MODIS cloud mask, land surface reflectance, aerosols, and clouds) • High operational availability achieved by: • Extra compute and ftp servers • Redundant power sources • Redundant production strings on separate networks (in progress) • Applications of LANCE data include: • Numerical weather & climate prediction/forecasting • Monitoring of Natural Hazards • Disaster Relief • Agriculture • Air quality • Homeland Security

3. LANCE Architecture LANCE Architecture Protocols: HTTP/FTP Users GES DISC EDOS Level Zero Processing Facility Data Available: -AIRS L1 and L2 -MLS L2 Format: RBD AIRS GMAO Format: RBD MLS NRL Protocols: HTTP/FTP MODAPS SPoRT Data Available: -MODIS L1, L2, L2G and selected L3 -AMSR-E L1, L2 and L3 Format: S-PDS MODIS UMBC Format: RBD AMSR-E USGS Protocols: HTTP/FTP Others OMI SIPS Data Available: -OMI L1B and L2 Format: RBD OMI Transfer latency: bandwidth dependent Observation to availability latency: 30-180 min. LANCE Element Under Development Operating LANCE Element RBD: Rate Buffered Data S-PDS: Session Based Production Data Set

4. LANCE Product Categories

5. LANCE-MODIS Architecture ftp server nrt1 Scheduling host modnrt1 Gigabit Ethernet switch Database server modnrt1db RAID 6 Storage modnrt1 Compute servers nrt1min101-115 2nd string to deployed in Spring, 2010 DOORS EMSnet Mission Network Science and Engineering Network nrt2 PDR server PDR server modpdr01 Browse Server • High Operational Availability achieved by: • 1. Redundant power: • 2 sources of conditioned power (UPS 9 and 10) for each component • 2. Extra compute and ftp servers (Dell 2950) • 15 servers (2 can fail w/o impact to production) • 2nd ftp server (currently manual fail-over) • 3. Redundant production strings on separate networks

6. LANCE-MODIS Operational Data Flow 6

7. Example Latencies for LANCE-MODIS and Standard Forward Products

8. Comparison of MODIS Standard and NRT Products Land Surface Reflectance over Midwest. Slightly more haze is visible in NRT view West of Great Lakes. Standard Processing Near-Real-Time Processing

9. Comparison of MODIS Standard and NRT Products Cloud top temperature over the Midwest. Thin clouds over lake Superior show sensitivity to GDAS ancillary data Standard Processing Near-Real-Time Processing

10. Comparison of MODIS Standard and NRT Products • For LSR the Match is the percentage of NRT data with <1% error margin when compared to the operational Collection 5 codes • For Snow, Sea Ice, and Fire, the Match is the exact pixel to pixel match between NRT and operational Collection 5 codes • Omission and Commission errors are computed as a percentage of the snow,sea ice, fire in the operational Collection 5 products

11. AIRS L1 Product Quality AIRS Level1-B VIS/NIR Quick Browse Image Standard Product Near-Real Time 11

12. AIRS NRT Data Product Quality • Same PGEs used to produce Standard Data Products used in NRT processing • NRT L1 data nearly identical to L1 Standard Product • Some minor differences in geolocation due to use of predicted ephemeris • Occasional minor differences in granules along contact session boundaries (~0.1 K) • Processing option: retain data at session boundaries to ensure data overlap when generating L1B • To date data latency has superseded minor quality improvement • NRT L2 products compare favorably to the L2 Standard Data Products. • NRT system uses surface climatology in place of dynamic ancillary data (GFS) as input to science data processing • Surface Air Temperature differences within 1 K for areas where sfc pressure is approximated by climatology, otherwise differences can be as much as 10 K in selected areas • Total Water Vapor differences within 5% over most of the globe; as much as 15 -20% where large surface pressure differences are observed, particularly at high latitudes

13. LANCE-MODIS Data Access and Services • LANCE-MODIS Web Site • URL: http://LANCE-MODIS.eosdis.nasa.gov • Provides access to a wide variety of information including; MODIS ftp site, registration, latencies, metrics, PGE versions and production rules, data outages, and access to other LANCE systems • Provides access to land and atmospheres browse products • 7 days of browse products are maintained in rolling archive • User Registration • Registration is required before data may be accessed • Present registration process is temporary pending the development of an EOSDIS-wide user registration system • URL: https://LANCE-MODIS.eosdis.nasa.gov:8080 • User Data Access • Subscriptions • Provide data directly as available (push) • Direct access to public • Download from directories (pull) • Hostname: nrt1.modaps.eosdis.nasa.gov; Directory Path: allData/1/esdtx/year/dataday • 7 days of data products are archived in a rolling archive

14. LANCE-AIRS/MLS User Access and Data Services • 5 day rolling archive available • FTP • Pull (anonymous, subscription) • Push (subscription) • Near Real Time Data Web Site • http://disc.gsfc.nasa.gov/nrt • Currently no format conversion or subsetting offered for NRT data products • Spatially qualified subscriptions available • Web Map Service (WMS) offered for AIRS NRT products • L1 visible radiances • CO concentration • SO2 indicator

15. Present Users of the LANCE-MODIS System

16. eMODIS System USGS/EROS Example Application: eMODIS Africa Expedited Product Flow T+12hrs USGS FEWS NET Processing System (stack, smooth, create products) Terra MODIS T+9hrs EW&EM NDVI and Anomaly Maps EDOS FEWS NET Decision Support System (USAID FFP) MODIS L0 Data T+3hrs MODIS L2 Data Input Data Deadline: Tuesday 10:00 a.m. FEWS Impact Assessment Maps LANCE EW&EM/FEWS NET Desired/required 24 hour or less data delivery schedule November 2009

17. Proposed Modifications to LANCE-MODIS 17 • Following the LANCE Workshop in December, the following changes to LANCE-MODIS are being planned/considered: • Addition of a redundant system using different power and on a different network • Adding a Land Surface Reflectance product • Adding the AMSR-E products • Adding a variety of sub-setting capabilities • Adding re-projection and mosaicing capabilities • Adding a variety of data formats (e.g. GeoTiff) • Integration of the Rapid Response System with LANCE-MODIS • Generate products derived from Land Surface Temperature to support the Flood Mapping program

18. Backup

19. NRT Products from LANCE-MODIS

20. NRT Products from LANCE-MODIS

21. NRT Datatypes available from GES DISC 1All datatypes appended with _NRT to denote difference from production datatypes

22. Product Dependencies on Ancillary Data • The L2 codes search back for ancillary data a maximum of the times indicated on the arrows • The L2 Snow, Sea Ice, and Fire do not use ancillary data products Cloud Mask L2 Aerosols 84h 120h(optional) 14d 72h 21d 36h GFS REYNSST SeaIce NISE GDAS TOAST 120h 14d 14d 8d 7d 60h L2 LSR L2 Cloud Ancillary Data UnchangedFrom Standard PGE