Framework Details All products may be run from one program Coordination of input data:

1. ABIAOT Land pixel? No Yes Declaration and initialization Internal check for unfavorable land condition Internal check for unfavorable water condition Read Configuration file Pass water internal check? Pass land internal check? No No Lookup tables Read Lookup tables Yes Yes Read input data Retrieval over water 1.) Initialize output structure. 2.) Interpolate aerosol and sunglint lookup table to current geometry. 3.) Calculate aerosol independent atmospheric parameters. 4.) Estimate surface reflectance parameters. 5.) Retrieve AOT550, fine mode weight and compute residual for each aerosol model combination. 6.) Select the optimal solution. 7.) Calculate the AOTs at all ABI bands. Retrieval over land 1.) Initialize output structure. 2.) Interpolate aerosol lookup table to current geometry. 3.) Calculate aerosol independent atmospheric parameters. 4.) Estimate surface reflectance. 5.) Retrieve AOT550 and compute residual for each aerosol model 6.) Select the optimal solution. 7.) Calculate the AOTs at all ABI bands. ABI reflectance data ABI cloud mask Ancillary model data Determine number of pixels No Process each pixel Yes Suitable for retrieval? No Yes Assign pixel input structure Output retrieval results • Infrastructure: Hardware and Software • Hardware • One IBM Power5+ machine with 16 CPUs • 75 TB of disk space on the SAN for all simulated data, proxy data and products. • Rack of Dell Processors (33 machines, 144 CPUS) for product development. • Software • Compilers • Intel compiler for Linux • Portland group compiler for Linux • Compiler on IBM AIX machine (xlf90, xlC) • Code Checkers • Lint • Forcheck • Valgrind • Home grown Standards Checkers • Revision Control • Clear Case • Clear Quest ABIAOT End Abstract The GOES-R Algorithm Working Group (AWG) Product Processing System Framework is under development at NOAA/NESDIS/STAR. The framework has been put in place for the development of the AWG geophysical product processing systems. This development is to produce and test the scientific algorithms in a scalable operational demonstration environment -- STAR collaborate environment. The framework includes accommodations for all the Level 2 Advance Baseline Imager (ABI) products. The dependence of one product on the output of another product (product precedence) has been taken into account during the development of the framework. Data common to all the product systems have been read into the framework and passed to the appropriate product systems. These design features and the current status of the framework will be discussed. • Framework Details • All products may be run from one program • Coordination of input data: • Model Forecast data • Emissivity Data • Instrument Data • Unified Data Structures • Data structures for input – instrument and ancillary data • Data structures for output products • Production rules determine how the system is run • Run any product in order determined by the production rules • Product runs may automatically trigger product precedence • Radiative transfer model calculations performed within the framework for all algorithms • One data format for all products • Common software libraries • Fully integrated C/C++ and Fortran 90/95 program Organized Ancillary Data • Ancillary data used by more than one algorithm is read into the AIT framework and then passed to the individual product systems • This reduces the I/O time and removes unwanted redundancies • Current ancillary data that has been moved from product systems to the framework are: • NWP • Emissivity • STAR Collaborative Research Environment • To develop and test scientific algorithms in a scalable operational demonstration environment • Hardware consists of operating systems that are within OSDPD • Multiple compilers are available for maximum code debugging capability • Develop algorithms using large quantities of simulated, proxy, and/or real data • The GOES-R AWG work is conducted within the STAR Collaborative Research Environment • Migrate and test AWG product systems • GOES-R AWG product validation. • Integration of AWG product systems into the AIT framework • Interface between Framework and the Product Algorithms • For each algorithm, have to create a common interface that is used both by the AIT framework and the algorithm research system • Common interface enables a plug and play capability between the research system and the AIT framework: • All data required by the algorithm from the framework is passed through this interface • Reduces time that it takes to roll back the algorithm changes to the product team that the AIT makes • Reduces integration time for future deliveries • Input Configuration Files • Determine how the framework is run • Run all products or a single product • Modify the dependencies used to create the products • Multiple settings (flags, constants, coefficient files), different dependencies • Enable backup dependencies • Run using multiple configuration files or a single configuration file • Run using default settings for each product or may customize • Run multiple satellites and instruments • Test different versions/updates of an algorithm against each other • Test different algorithms against each other • AWG Algorithm Flowchart: • Description of the Product System • Flowcharts are delivered by the product teams to the AIT: • Describes the Algorithm Processing System • Identify the inputs and outputs • List Function/Subroutine calls • Describes how the products are generated • Preparation of Framework for algorithm software • AIT has received system flowcharts of each of the product systems being created within the AWG. • AIT has/will breakdown each of the product systems to find the commonality between them. • Commonality between the systems resides within the framework. Current Status • Framework is in place • Completed parts: • NWP model ingest • Radiative Transfer Model installation • Configuration File Ingest • Current Work • Installation of the following product systems: • Cloud Mask • Sounds • Fire • Land Surface Temperature • NDVI • Aerosols • Algorithm Working Group (AWG) Integration Team (AIT) • Consists of: • System Lead • Quality Assurance Lead • Four Full Time Programmers • Documentation Specialist • Configuration Manager • Roles • Interface with the AWG Product Teams • Integrate AWG Product Systems • Quality Assurance Check of the AWG Product Systems • Interface with the GOES-R Project Office • Packaging of deliverables to the GOES-R Project Office • Supply the standards for the AWG product teams: • Coding Standards • Code examples • Documentation Standards • Documentation Templates • Delivery Standards • Configuration Management • Coordinate between AWG product development teams • Provide technical and software assistance to AWG product teams • Receive code deliveries from product teams • Verify and Review code deliveries • Distribute updated software package to product teams • Overall Strategy • Develop a processing system where GOES-R algorithms can be developed and tested is a well defined and organized manner • Code will adhere to NESDIS Operational Standards • All pre-operational code is thoroughly tested within the STAR Collaborative Research Environment Development of the GOES-R AWG Product Processing System Framework Walter Wolf1, Lihang Zhou2, P. Keehn2, Q. Guo3, S. Sampson3, S. Qiu2, and Mitchell D. Goldberg1 1NOAA/NESDIS/ORA, Camp Springs, MD 20746 2PSGS, Alexandria , VA 22311, USA 3IMSG, Kensington, MD 20895, USA Detailed Aerosol Optical Thickness Flowchart