The NASA Standards Process for Earth Science Data Systems

1. The NASA Standards Process for Earth Science Data Systems Richard Ullman, NASA Yonsook Enloe, SGT Inc IGARSS 2010

2. Technology Diffusion Time 100% Source: Geoffrey Moore, Crossing the Chasm, 1999modified after Everett Rodgers, 1962 Adoption 0% Innovators Early Early Late Laggards Adoptors Majority Majority Innovative Pragmatic Technology Demonstration Mission Reliability/Stability NASA’s Chasm NASA Earth Science Data Systems Standards - IGARSS - August, 2010

3. Standards Diffusion NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • Chasm between: • early adopter - technology for strategic advantage • early majority - pragmatic focus. • A practical path across chasm … “Community-led” • successful practice in specific community • broader community adoption • community-recognized “standards” • Community-led relies on: • trusted endorsements • strong leadership • “whole product”

4. ESDS Standards Process – Seeking Pragmatic Recommendations NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • A specification or practice is recommended as a standard … Only after practices have been shown to: • (1) have demonstrated implementation and • (2) benefit to operation will they be endorsed for preferential use. • Ideas come from innovators or mission planners and are tempered by the significant demands of writing an RFC. • Review process permits adoption only after “significant” community endorsement. • THE RFC PROCESS PROVIDES PRAGMATIC REFERENCES

5. Crossing our Chasm – ESDS and the Pragmatic Decadal Survey NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • SPG has been successful as a pragmatic voice: NASA HQ has endorsed SPG recommendations for Decadal Survey Mission use. Mission planners have embraced SPG as subject matter experts. • NASA HQ recognize the benefit of choosing standards that are already recommended by the community • The frustrating thing about standards from a HQ perspective is that everyone seems to want a different one. • Program-focused HQ planners want agency investments to have broad return. i.e, for data: effective use for the primary mission and multiple secondary uses • Decadal survey mission planners recognize the benefit of choosing standards that are already recommended by NASA. • The frustrating thing about standards from a mission is that there are so many to choose from. • Mission-focused pragmatic planners want to make the right choice. Standards that work and standards that won’t get them in hot water with their stakeholders.

6. Decadal Survey [2] NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • SPG process has engendered: • Greater confidence that mission products will be interoperable with heritage and current NASA data holdings • Stakeholder community benefits because their preferences and opinions are available to the mission planner • Lower barriers to entry and use of NASA data by external discipline communities within NASA and outside NASA because community standards are used. • NASA Earth Science Data Systems Program is actively evolving the Earth Science data “system of systems” architecture for the Decadal Survey missions and beyond

7. Tier 1 Decadal Survey Missions NASA Earth Science Data Systems Standards - IGARSS - August, 2010 Tier 1 Decadal Survey missions are the first group of missions to be formulated since NASA chartered the SPG Soil Moisture Active Passive (SMAP) Mission Ice, Cloud, and Land Elevation Satellite 2 (ICESat 2) Climate Absolute Radiance and Refractivity Observatory (CLARREO) Deformation, Ecosystem Structure, and Dynamics of Ice (DESDynI) The Tier 1 missions are actively working with the SPG in order to explicitly assure that the mission data systems solutions draw on the consensus-built lessons of agency experience. Tier 1 missions have a level 1 requirement to use SPG-endorsed standards

8. Crossing our Chasm – ESDS and Innovative Technologists NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • It is a challenge to encourage technology demonstration projects to use RFC process. • Technologist are often more interested in the technology puzzle rather than in practical dissemination. • Early adopters are often more interested in strategic advantage rather than wide use. • System Engineering focused on mission requirements makes “outreach” of writing RFC for community good “out of scope” • “Whole Product” may be missing. • Considerable systems engineering is still required when abstract standard is to be embedded into a mission implementation.

9. Process Basics NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • 3 types of reviews and all reviews can be done simultaneously – some reviewers perform multiple reviews. • Technical specification review • Usefulness for purpose review • Operational readiness • If the proposed standard has been adopted by an another standards organization, then do not need a technical specification review • For a mature defacto standard, we will perform a subset of the three types of reviews. May not need an operational readiness review or usefulness for purpose review • The SPG will determine which reviews will be conducted for each proposed candidate standard. Can be a subset. • Encourage strong community leader – essential to the process

10. Responsibilities NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • Community Leader • Identify someone in their community who will document standard according to SPG guidelines. • Work with the community to get an extended review of the proposed standard. • SPG • Assign “RFC editor” to advise on RFC document. • Publish and publicize RFC • Assign “TWG”, technical working group to organize community review and evaluate responses. • Recommend action to NASA HQ.

11. Endorsed Standards and Tech Notes Standards Technical Notes OGC Web Map Service V1.3 AURA Guidelines Backtrack Orbit Search Algorithm Interoperability Between OGC CS/W and WCS Protocols Lessons Learned Regarding WCS Server Deisgn and Implementation Creating File Format Guidelines: The Aura Experience • Data Access Protocol (DAP) V2.0 • OGC Web Map Service V 1.1.1 • HDF 5 • HDF EOS 5 • NetCDF Classic • GCMD DIF • ICARTT NASA Earth Science Data Systems Standards - IGARSS - August, 2010

12. Near Term & Long Term Future RFCs RFCs Under Review Tech Areas for Future RFCs Web Services (e.g. REST, SOAP, ..) Catalog services (e.g. OGC CSW,…) Metadata standards (e.g. ISO 19115,…) Visualization standards (e.g. KML,..) File Formats (e.g. NetCDF4,..) Interface standards (e.g. DAP v4, …) Data modeling Data Fusion • CF Metadata Conventions • ECHO Metadata Model • Mapping HDF5 to DAP2 • NetCDF-4/HDF5 File Format NASA Earth Science Data Systems Standards - IGARSS - August, 2010

13. Back-UP NASA Earth Science Data Systems Standards - IGARSS - August, 2010

14. Decadal Survey Background NASA Earth Science Data Systems Standards - IGARSS - August, 2010 • At NASA’s request, the National Research Council (NRC) looks out ten or more years into the future and prioritize research areas, observations, and notional missions to make those observations • NRC completed its first decadal survey for Earth science in Jan 2007 at the request of NASA, NOAA, and USGS. • For the next decade, the decadal survey identified 15 missions for NASA and 3 for NOAA including 1 joint NASA/NOAA • The 17 total (NASA & NOAA) missions are presented as the result of prioritization methodology designed to achieve a robust, integrated program. • The 15 missions for NASA are presented in 3 time-phased blocks. • The 17 missions form a minimal, yet robust, observation component of an Earth information system that is capable of addressing a broad range of societal need.

15. TWG • Review Questions: • Technical Specification • Operational Readiness • Suitability for Use Process Diagram • Initial Screening • Initial review of the RFC • Provide RFC submission support • Form TWG; set schedule RFC SPG Proposed Standard Technical Note Reject Community Review Stakeholders Evaluate Proposed Standard TWG Evaluate Proposed Standard Implementations and Community Response SPG Recommendation Reject Technical Note Recommended Standard NASA Earth Science Data Systems Standards - IGARSS - August, 2010

16. NASA ESDS RFC List NASA Earth Science Data Systems Standards - IGARSS - August, 2010

17. NASA ESDS RFC List NASA Earth Science Data Systems Standards - IGARSS - August, 2010