National Climate Assessment Data Management and External Data Requirements

1. National Climate AssessmentData Management and External Data Requirements

2. Overview • National Climate Assessment (NCA) • Version 3.0 • Interagency climate portal • NCA Data Management Planning • Considerations • Requirements • Approach • End-to-end data management and connecting in the middle • Earth System Grid Federation • Challenge and Next Steps

3. Tiered policy for use of external use of data? More or different requirements have to be met Increasingly stringent criteria for use External data policy Reproducible ------ Use cases ------ operational NCA severe weather forecast

4. Tiered policy for use of external use of data? More requirements have to be met long-term retention requirement Increasingly stringent criteria for use Reproducible ------ Use cases ------ operational NCA severe weather forecast

5. About the National Climate Assessment Section 106 of GCRA: Scientific Assessment On a periodic basis (not less frequently than every 4 years), the Council, through the Committee, shall prepare and submit to the President and the Congress an assessment which – • integrates, evaluates, and interprets the findings of the Program and discusses the scientific uncertainties associated with such findings; • analyzes the effects of global change on the natural environment, agriculture, energy production and use, land and water resources, transportation, human health and welfare, human social systems, and biological diversity; and • analyzes current trends in global change, both human- induced and natural, and projects major trends for the subsequent 25 to 100 years.

6. NOAA’s Role in the National Climate Assessment • NOAA has played consistent leadership role in previous National Assessments • NOAA ‘hosting’ and supporting Federal Advisory Committee for next assessment • Web-framework, data policy - quality and transparency, publication – editorial and graphics, modeling/”downscaling”, scientific leadership, climatologies, regional partners and engagement

7. About the National Climate Assessment • A product AND a sustained process • More stakeholder engagement and contributions • “Request for Information” will be sent out shortly • More transparency, traceability, accessibility and usability • More data, more authors, more users • A suite of Indicators (discrete and composite, physical and societal) • Next report due in 2013, draft due in 2012, Federal Advisory Committee just formed • Snapshots plus frequent updates • More frequent interim reports, updated indicators and fresh web-content are expected… BUT • An assessment is necessarily a snapshot in time and should be reproducible • Regional and Sectoral chapters, cross-sectoral and inter-regional, adaptation and mitigation

8. Calls for an Interagency Climate Portal • National Climate Adaptation Summit, John Holdren, Director of the White House Office of Science and Technology Policy, : “Creating a federal climate information portal. This would provide single-point access to data from all relevant federal agencies and programs and would evolve over time into a more ‘national’ portal with information about relevant non-federal efforts.” • Also, Executive Office of the President, Committee on Environment, Natural Resources, and Sustainability (CENRS) Roundtable on Climate Information Services called for: an interagency climate information portal plan, focusing on National Climate Assessment Reflects broad calls from stakeholders – real motivation

9. Portal: Where to Begin? • A National Climate Portal as envisioned is an immense task • Use the National Climate Assessment (NCA) to focus the initial scope • The NCA can be… • A starting point, extensible to interagency data • A driver of requirements for Data Management Planning: transparency, access, usability, data quality, and business rules • A test case for connecting robust and federated data structures with appropriate user experiences • 2009 Assessment is being redeployed online as we speak and explores better transparency and data access

10. NCA Data Management Planning Considerations • External Data (At least 2 tiers): • External to NOAA, but still federal • (even here, egs of citizen science (eg climate effects network – USGS) • Clear need to move beyond government data: • peer reviewed scientific literature as information source, • Non-peer reviewed, non-traditional • but who guarantees access and a continuous/permanent archive? • Tiered “acceptance”? Core vs. supplemental? • Using external data in NCA implies that quality and access are guaranteed • Quality • Information Quality Act requirements (highest level) • Peer review requirements (highest level)

11. Quality Requirements in NCA • NCA is classified as ‘A Highly Influential Scientific Assessment’ (HISA) - must meet the highest level of quality in the Information Quality Act (NOAA IQA) • is a scientific assessment that: (i) has a potential impact of more than $500 million in any one year on either the public or private sector (the economic test); or (ii) is novel, controversial, or precedent-setting, or of significant interagency interest (the narrative test).

12. NCA Data Management Planning Considerations Example of projected growth in “Physical” climate data • Volume • Enormous volume growth • E.g. physical data growth driven by satellite/radar and model data • Kinds of data • Physical climate data • Model data, reanalysis, satellite… • e.g. downscaled and right-scaleddata needed and new model runs (CMIP5) • Also socioeconomic, biophysical,ecological… • Original data (e.g., satellite and in-situ observations) • Synthesized data (e.g., model data, indicators, GIS application) • Interpreted products (e.g., regional assessments) • Also will need to consider ‘new’ types of data such as: State and local government, public utilities, traditional knowledge, operational data, insurance risk data J T Overpeck et al. Science 2011;331:700-702

13. NCA Data Management Requirements • NCA will likely not ‘host’ basic data, but will provide central access to federated system (critical questions remain for external data) • For example, when external data is of high quality but permanent access not guaranteed – do we agree to host? Which agency? • Highly-processed, derived data used in the report may be centrally hosted • Use of data.gov? • Must accommodate huge data sets • Too big to move, too big to host all data in one location • Implies distributed data architecture, federated storage • Metadata must adhere to common standards • Uncertainty must be documented • Quality and original source of data must be clear and peer reviewed • Data system must be interoperable • Use of data.gov must have clear guidelines • Narrative assessment and graphics must link to source data and information

14. Approach: End-to-end Data Management • What data sets? • What formats? • What time periods? • How to cite data consistently? • How to document data reviews? • How to ensure traceability from key findings to data? • How will we curate the data used on the assessment? • How will we deal with non-traditional data? User requirements should drive business rules User Interface Missing Layer Making the connection between user interface and complex data system – different communities of effort Business Rules Federated data system Data Stewardship

15. Approach: The Earth System Grid Federation (ESGF) • Too much data to move, must leave in place • Promote sharing of knowledge, software and tools among partners • Define APIs and protocols for interoperability among data centers, • PROTOCOLS FOR ACCESS OF EXTERNAL DATA? • Collaborative development of some software components • Deployment of common software infrastructure

16. Data Management Planning Challenge • Develop an implementable strategic plan for the immediate data management demands of the NCA, and the longer-term interagency/ National Climate Portal • Guidance to external contributors: how to meet the requirements – near term need! • Focus on the business rules • SUGGESTION: Case study/use cases to illustrate approach • Work with NCA to co-develop NOAA guidance • If we can meet rigorous IQA standards of NCA, then we likely meet quality for other uses, but too restrictive for some uses? Points to tiered approach. If one policy is preferred, must incorporate different uses

17. Thank you Anne.Waple@noaa.gov 828-257-3000