Tools Developed For AVHRR Data Production and Analysis

1. Tools Developed For AVHRR Data Production and Analysis • The AVHRR modular framework can produce three output datasets in NetCDF format when AVHRR NetCDF inputs are used • AVHRR observations, cloud property retrieval, and cloud mask with necessary ancillary fields required for clear sky OH albedo updating • AVHRR observations, cloud mask, and debug parameters • Ancillary fields (i.e. snow mask, surface elevation, IGBP, etc…) required for NOAA-9/10 ERBE radiation CDR • Tools suggested by team members and developed for product production and analysis • Scripting infrastructure on AMI to 1) pull AVHRR LV1B files from cloudsgate1 NAS, 2) convert to NetCDF, 3) farm out to a user-defined maximum number of jobs, 4) each job process orbit in 1000 scanline chunks, 5) write NetCDF file types 1 and/or 2 above, and 6) push output back to cloudsgate1 NAS • - Files internally compressed and most parameters written as short integers • IDL quicklook image creator for file types #1 and #2 above • Web-based quicklook image viewer with animation, fade, and toggle tools • Fortran monthly average gridding software for 8 parameters (cloud fraction, cloud effective/top height/pressure/temperature, optical depth, particle size), separated by day/night, water/ice, land/water/snow or ice land surface • IDL monthly average image viewer for modular, CERES, CALIPSO, and PATMOS-X outputs • Fortran cloud fraction analysis software where cloud fraction is computed for each IGBP type, sunglint yes/no, and day/night

2. AMI Processing With Standard NetCDF-4 DAY 1 DAY 2 ORBIT 1 ORBIT 1 ORBIT 2 ORBIT 2 ORBIT 3 ORBIT 3 …. …. ORBIT 15 ORBIT 15 Standard NetCDF does not allow 2 jobs to simultaneously write to same NetCDF file We cannot continue to process Day 3 until previous 2 days are completed and clear sky OH albedo maps are updated 30 Jobs Processing Orbit 15 Times In 1000 Scanline Chunks Update Clear Sky Reflectance Map Process Days 3 and 4

3. AMI Processing With Parallel NetCDF-4 DAY 1 DAY 2 Chunk 1 Chunk 1 Chunk 2 Chunk 2 ORBIT 1 ORBIT 1 Chunk 3 ….. Chunk 15 Chunk 3 ….. Chunk 15 ORBIT 2 ORBIT 2 ORBIT 3 ORBIT 3 …. …. ORBIT 15 ORBIT 15 Parallel NetCDF allows simultaneous writes We distribute jobs in 1000 scan line chunks across processors, greatly accelerating our processing Parallel NetCDF has been installed on AMI, but I have not yet figured out how to use it Up to 450 Simultaneous Jobs Processing 2 Days Worth of Orbits Update Clear Sky Reflectance Map Process Days 3 and 4

4. Monthly Mean Datasets To Be Shown In The Following Slides 1) Modular AVHRR framework using late 2011 mask and pre-Ed4 cloud retrievals from October 2008 and October 1986 2) CERES MODIS Edition 4 spring 2012 delivery gridded output from October 2008 3) PATMOS-X AVHRR produced by UW-CIMSS for 2010 GEWEX cloud assessment from October 2008 and October 1986 4) CALIPSO 5-degree cloud fraction from October 2008 via Qing

5. October 2008 Results

6. Modular Using CERES October 2008 Clear Sky OH Albedo Map

7. Modular Using AVHRR October 2008 Snow and Non-Snow Clear Sky OH Albedo Map

13. Modular Using CERES October 2008 Clear Sky OH Albedo Map NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to significantly impact optical depth retrieval

14. Modular Using AVHRR October 2008 Snow and Non-Snow Clear Sky OH Albedo Map NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to significantly impact optical depth retrieval

15. NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to significantly impact optical depth retrieval

16. NOTE: Modular AVHRR does not include O3 up to 0.1 hPa which has been shown to significantly impact optical depth retrieval

21. October 1986 Results