Climate Data Records CDRs and CICS

1. Climate Data RecordsCDRs and CICS J.J. Bates, J.L. Privette, E.J. Kearns, NOAA National Climatic Data Center Asheville, NC

2. Outline NOAA Climate Goal reaction to NRC reports Summary and status of NOAA’s CDR Program Impact of CICS on CDR Program Progress in JPSS-specific CDR issues and strategic planning F11 FFO

3. NOAA’s CDR Program is in response to an operational need • A Climate Data Record (CDR) is a time series of sufficient length, consistency, and continuity to determine climate variability and change • FCDR: Calibrated signals for a family of sensors together with the ancillary data used to calibrate them. (e.g. brightness temperatures, TOA radiances) • TCDR: Geophysical variables derived from FCDRs; May be generated by blending satellite observations, in-situ data, and model output (e.g., Aerosol Optical Depth, SST) • A Climate Information Record (CIR) is derived from CDRs and related data tand provides specific information about an environmental phenomena of importance to science and society (e.g., Hurricane tracks, Arctic Sea Ice Extent, Incidence of disease)

4. CDRs Provide Scientifically Corrected Long Time Series for Climate Analysis Uncorrected Data Time Series Contain Both Environmental Information and Satellite-induced Artifacts Operational Climate vs Operational Weather Operational weather and hazard products are produced rapidly to potentially save life and property New satellite launched Top of Atmosphere Vegetation Index Vegetation Greenness Index Climate Data Records (CDRs)provide long term product consistency through rigorous reprocessing with advanced algorithms, ancillary data and evolved instrument understanding. Climate Information Records (CIRs)provide specific information about environmental phenomena of particular importance to science and society (e.g., hurricane trends, drought patterns) Top of Canopy Vegetation Index Time (year)

5. CDR Program functionally located within RSAD at NCDC

6. NOAA NESDIS & Climate Goal reaction to NRC “Ensuring the Climate Record…” • Clear Agency Roles and Responsibilities • Regular quarterly NOAA-NASA meetings and NOAA participation in CERES and SORCE science team meetings are helping • International Coordination • NOAA provides international leadership through WMO GSICS & SCOPE-CM as well as CEOS Climate ad hoc Group • Community Involvement in the Development of Climate Data Record

7. CDR Program Functional Framework Management BoardUSGCRP Observations Management Structure Intl Data Programs Product Development Teams (CDRs, CIRs and Support) Experts in Instrument Characterization, Algorithms, Validation, Data Management, Applications, and Observing System Performance Monitoring Improved CDR and CIRs Leveraging Resources / Collaboration Observing System Performance Monitoring Production of Near Real-time CDRs Processing of CDRs for Long-term Records Climate Information Records Other Agencies GEOSS, CEOS Design for Future Systems CDR Stewardship Engaging User Community Modeling Monitoring Prediction Research

8. End-to-End Stewardship NOTIONAL SAMPLE

9. CDR Activities Address 3 Epochs Prepare for products from “Remanifested” climate sensors and future missions Establish operational CDRs and CIRs from NOAA’s archive of four decades of heritage data Ensure climate quality data from operational systems, build Raw Climate Data Records to facilitate future reprocessing, ensure continuity with heritage CDRs

10. Long-Term Planning • NOAA is initiating long-term planning by initiating community requirements meetings • Continuity of Earth Radiation Budget (CERB), July 2010 • NOAA is working with the international community to coordinate efforts • The aim of the WMO Sustained, Co-Ordinated Processing of Environmental Satellite Data for Climate Monitoring (SCOPE-CM) is to establish a network of facilities ensuring continuous and sustained provision of high-quality satellite products related to the Essential Climate Variables (ECV), on a global scale, responding to the requirements of the Global Climate Observing system (GCOS). • Committee on Earth Observing Satellites Climate Ad hoc Group (CEOS-CAG) - The group should ensure the more coherent product of FCDR/ECVs among agencies in response to the defined needs of GCOS

11. Preparing for CERES - ISCCP Reprocessing Data Flow Multi-National Geostationary Satellite Data B1 10 km pixel data every 3 hours Perform Mapping Operation B4 Run Cloud Detection Algorithm BX DX Perform Radiative Retrievals Calculate Gridded Statistics DS 1 deg res every 3 hours Merge the Various Satellites D1 D2 Calculate Monthly Averages 1 deg monthly NCDC has processed to this level to SW and LW Flux calculations

12. CDR Program Funding Profile * To Be Determined

13. FY10 Investment Allocation* *Expected **Anticipated

14. FY10 Organization Allocation* 48% * planned

15. NOAA NESDIS & Climate Goal reaction to NRC “Ensuring the Climate Record…” Microwave CDRs funded Altimeter focus in FY11 FFO Clouds and radiation (ISCCP) reprocessing and R2O underway Hyperspectral being used to correct filtered radiometers Solar irradiance CDR funded Global aerosol CDR R2O Ocean color currently not funded Ozone CDR team funded

16. CDR Product Discipline Teams • Goal • Have community experts coordinate activities within discipline/sensor • Develop coherent and self-consistent product suites • Ultimately seek consistency across disciplines (e.g., cloud detection) • Possible Issues for Team to Address • Algorithms – adopt common cloud/land/water masks, FCDR and ancillary data inputs, consistent QA flags and metadata for quality control, begin algorithm chain production and testing in development phase • Products – identify gaps or resolve apparent redundancies within product suite; adopt common and appropriate grid projections, resolutions, data formats and packaging, etc.

17. Current CDR Development Portfolio(FY09 & FY10 Awards) • AMSU (ATMS) • Hydro Cycle (Ferraro) • Upper Air Temp (Ho) • Water Vapor (Luo) • Temp. Profile (Zou) • AVHRR (VIIRS) • Cloud Properties (Kato) • Snow/Ice (Key) • VNIR Cal./Clouds Minnis) • Thermal Calibration (Mittaz) • Land/Carbon (Vermote) • Ocean Fluxes (Clayson) Other Satellites • GOES: Imager (ABI) • VNIR Cal./Clouds (Minnis) • SORCE, Glory (TSIS) • Solar Irrad. (Pilewskie) • HIRS (CrIS) • FCDR/Intersensor calibration (Cao) • Water Vapor (Luo) • Cloud Properties (Menzel) • DMSP: SSM/I, SSMIS (MIS) • Calibration (Kummerow) • Snow/Ice (Key) • Water Vapor (Luo) • ERBS: ERBE (CERES) • Radiation Budget (Kato) • SBUV (OMPS) • Ozone (Flynn) • GPS RO (Various) • Temp. Profiles (Ho) Arrows identify key climate instruments

18. Coordination Activities:Annual CDR Team Meeting • 2-day meeting (Aug. 4-5) in Asheville • Attended by more than 50 participants from academia, industry, NOAA and other agencies • Featured 14 competitively-selected CDR investigators describing algorithm and product plans • CDR Program rolled out research-to-operations plans and expectations from team • Impact: CDR investigators identified intra-team product dependencies & synergies; CDR Program developed detailed understanding of CDR algorithm formulation and maturity

19. New CDR Teaming at NCDC

20. Public Posting of Prototype CDRs in Operations • Maturity Matrix adopted by GCOS (1 of 12 guidelines for CDR Production)

21. CDRs in Development • Online Data • Investigator Teams/ Affiliations • Abstracts • Progress Reports • Community Workshop URLs • Team Meeting Presentations • Impacts • Used by EUMETSAT’s Satellite Application Facility on Climate Monitoring (CM-SAF) • Interest in coordinating work

22. CDR Program Management: Staffing • Federal Program Manager vacancy announcement open • FAC-P/PM Senior certification • Closes 9/10 • RFP for Program Management Support released by ERAD • 2 year base + 1 year option • 4 Key Personnel positions • Technical review completed Aug. 20 • Expected award date: this week?

23. Interaction with CICS • Several selected CDR grants were awarded as Type III funding • Support for key internal work • NEXRAD, extreme events, SST • Securing science and scientific programmer support in RSAD • Postdocs and students • Assembling IT Development Environment • Summer ‘11 climate workshop

24. Impact of Interaction with CICS • Introduction of academic flavor to an operational federal workplace • Part of paradigm shift at NCDC • Enabling CDR processing • Development environment • Network connections • Bring another dimension to NCS

25. JPSS-Specific Issues • NASA missions are end-to-end and NOAA are broken into segments • NOAA has mapped one to the other and found gaps that are being addressed • Sensor integration and testing • Instrument scientist and analyst • Sensor cal/val teams • CDR legacy processing not portable

26. CDR Information Flow Data (Direct & Remotely Sensed- RDRs) Time-tagged Geo-Referenced Raw Climate Data Records (R-CDRs) Sensor DataRecords (SDRs) Homogenization and Calibration EnvironmentalData Records(EDRs) Fundamental Climate Data Records (FCDRs) Climate Data Records or Homogenized Time Series Converted to Bio-Geophysical Variables Fundamental Climate Data Record (FCDR): Time series of calibrated signals for a family of sensors together with the ancillary data used to calibrate them. Thematic Climate Data Record (TCDR): Geophysical variables derived from FCDRs, often generated by blending satellite observations, in-situ data, and model output. Thematic Climate Data Records (TCDRs) Climate Information Records (CIRs)

27. Climate Raw Data Record(R-CDR) Status Recent Accomplishments Major Milestones • Hired 2 new programmers; 1 more sought • Successfully ported and adapting NPP/VIIRS and CrIS software as used by NASA PEATES • System Definition Review (Completed: Mar. 2010) • Critical Design Review (Scheduled: Sept. 16, 2010) • “R-CDR CDR” • External reviewers from GSFC & Univs. Miami, Wisconsin & Colorado NPP Operational Software Reuse Project Risks • Complexity of the NPP/NPOESS RDRs and short schedule • Staffing needs for ATMS, and OMPS NP Raw CDRs and system execution coding Interface Data Processing Segment Land PEATE R-CDR NPP/JPSS NCDC GSFC Project Schedule Updated: 03/12/10

28. FY11 FFO • http://www.federalregister.gov/a/2010-17294/p-579 • Initially focused on Fundamental CDRs and Thematic CDRs related to Earth's water and energy cycles and sea level • Funding Availability: For the present grants competition, the CDR Program expects to select proposals over a two year period (FY 2011 and FY 2012) for funding. The total anticipated Federal funding in FY 2011 is $2.5M for new awards. The anticipated number of new awards is from 5 to 15. • The CDR Program anticipates new funding availability in FY 2012 for additional awards from the present grants competition. • The total anticipated Federal funding in FY 2012 and the number of additional awards will be dependent on the enacted budget. • Application Deadline: Letters of Intent (LOI) should be received at the CDRP Office no later than 5 p.m. Eastern Time, September 15, 2010 • Full proposals must be received no later than 5 p.m. Eastern Time, November 10, 2010

29. Summary The CDR Program has recurring resources and is actively engaging the U.S. and international climate communities to operationalize CDRs for NOAA. CICS has already proven to be a key cooperator for the CDR Program JPSS gaps in climate information stewardship have been identified and are being addressed FY11 FFO is out -- and responses due soon!

30. Questions?

31. Cal/Val Program NPP Data Flow Diagram Science Data Segment (NASA) ERB CARS CERES CDRs Research Community NPP/NPOESS Data Processing Segment (IDPS) RDRs PEATEs PEATEs OMPS-Limb SDRs, EDRs PEATEs PEATEs SDRs, EDRs, IPs, Ancillary & Auxiliary Data Correlative Data & Results xDRs, IPs NESDIS Central AFWA Central CERES CDRs Archive & Dist Segment (CLASS; NCDC and NGDC) Ingest xDRs, IPs, Ancillary & Auxiliary Data RoboticStorage RDRs Datastore CDR Program (NCDC) Inventory CDRs,CIRs xDRs, CDRs, NUPs SDRs, TDRs, IPs, EDRs CIRs, IPs, Ancillary & Auxiliary Data KEY: RDR Raw Data Record SDR Sensor Data Record TDR Temperature Data Record EDR Environmental Data Record xDR RDR, SDR, TDR, EDR CDR Climate Data Record CIR Climate Information Record IP Intermediate Product NUP NOAA Unique Product NDE (NPOESS Data Exploitation) NUPs Users V1.6 NCDC NPP 16 July 2009

32. Climate Raw Data Record (C-RDR)System Interfaces NESDIS Central xDRs, IPs, Ancillary & Auxiliary Data Interface Data Processing Segment (IDPS) xDRs, IPs, Ancillary & Auxiliary Data CLASS PoP National Climatic Data Center (NCDC) CLASS NCDC R-CDR Processing RDRs, Ancillary & Auxiliary Data R-CDRs CDR/CIR Processing Archive Climate User Community

33. Outline • About NOAA’s new CDR Program at the National Climatic Data Center • Why NOAA is interested in this activity for oceans, land, & atmosphere • What is meant by an operational CDR • How it will engage the science community to meet program goals

34. FY10 CDR Investments

35. Maturity Matrix Identifies Milestones and Research-to-Operations Transition Points

36. Sustained, Coordinated Processing of Environmental Satellite Data for Climate Monitoring (SCOPE-CM) The aim of SCOPE-CM is to establish a network of facilities ensuring continuous and sustained provision of high-quality satellite products related to the Essential Climate Variables (ECVs) global scale respond to the requirements of the Global Climate Observing system (GCOS)