Vision for an Earth System Data Environment (ESDE)

1. Vision for anEarth System Data Environment (ESDE) Kelvin K. Droegemeier Presented to the NWC Directors 20 August 2012

2. Background • Oklahoma is home to a wide variety of geographically distributed, ground-based environmentalobserving systems • They… • Are operated by several different Federal and State agencies as well as a few private entities • Serve a wide variety of purposes but in general produce high quality data • Span the spectrum from highly reliable, longstanding operational platforms to new and highly experimental mobile platforms • Have strong linkages to research and educational programs at OU and OSU

3. Example: Radars and OK Mesonet

4. Other Environmental Data in Oklahoma • Satellites • Stream gauges • Seismic arrays • Lightning mapping arrays • CO2 sensors • Pollen and particulate sensors • Road weather sensors • Bio, Geo, Arc, Water, Climate Surveys • Video and image capture systems • Unmanned aerial systems (around the corner) • Water quality sensors • Other…

5. Broader Types of Data • A wide variety of other static or slowly-changing data – both physical and social – are available for Oklahoma • Population (density, race and ethnicity, unemployment, religion, income, social class, mortality, migration, etc) • Land cover • Terrain • Surface assets (e.g., pipelines, refineries, hazardous material locations, munitions depots) • Drilling core samples • Other…

6. Oklahoma as a Data Leader • Oklahoma also has … • A strong history of environmental sensing innovation and proof-of-concept testing (e.g., JDOP, Oklahoma Mesonet, CASA, RaXpol, AIR, PX1000, OU-PRIME, lightning mapping arrays) • Pioneered the utilization of environmental data in climate statistics (Lamb and Richman), data assimilation and numerical prediction (CAPS/ARPS), decision support tools (WDSS-II), and social sciences • Been a national leader in the collection and real time disseminationof environmental data and related products (e.g., CRAFT, NSSL National Mosaic and Multi-Sensor QPE) • Led major initiatives in data access portals, federated repositories, and service-oriented architectures (e.g., LEAD, CyberCommons) • Hired (at OU) a new Dean of Libraries who is a data expert and visionary and will move the Library forward as a focal point for data collection, stewardship and access

7. Key Points • The atmosphere, ground surface, and sub-surface of Oklahoma are among the most heavily instrumented and data-rich regions on the planet • These environmental data are supplemented by a wide variety of other data • Collectively, these Earth System data are vital for research, development, education, and business activities in Oklahoma, and represent a unique strategic asset that also has value well beyond our borders • Historically, we have thought about, promoted, and utilized these data somewhat piecemeal, even for major field programs such as VORTEX, IHOP, etc. • We have before us an important opportunity to both showcase and utilize Oklahoma’s Earth System data assets in their totality, as a single, integrated system that is structured and marketed as such

8. Motivation for a New Approach • Many recent developments suggest value and opportunity in taking a new approach to Earth System data in Oklahoma • Establishment of the NWC and its emerging national role • Earth System research that requires a broad array of data • Creation of new organizations such as the Climate Science Center and Oklahoma Water Survey • Huge advances in data collection, dissemination and storage technologies (e.g., clouds, federated repositories) and major national initiatives such as EarthCube and DataOne • Decision support systems utilized in risk decision making by a wide array of industries – from energy to insurance to aviation • Geographic Information Science as a pervasive capability to merge vast amounts of highly disparate data • System-of-systems concepts nationally and globally and new Federal funding of national observing facilities (e.g., NEON, Earthscope)

9. Motivation for a New Approach • A new strategy for cyber support of research and development at OU that tightly integrates research, IT and libraries • External Blue Ribbon Panel visited OU two weeks ago and affirmed this strategy • OU can be a unique, national role model – with the library as a focal point for data stewardship • The new EPSCoR Track-1 proposal provides a unique organizing element and a framework for supplemental funding • Oklahoma can become a change agent for other EPSCoR jurisdictions in its data frameworks and policies

10. Current State of Affairs at OU Research IT Libraries

11. The Future at OU Research IT Libraries

12. A Bold Concept • Under the auspices of the National Weather Center as a natural integrator and facilitator, create the Earth System Data Environment (ESDE) • ESDE will join data from ALL of Oklahoma’s key Earth System sources – both operational and experimental – into a single cyber framework/portalfor research, education, applied development, and even commercial application. • ESDE is both a cyber framework as well as a virtual organization that brings together disparate data providers with a common purpose and shared benefits – though of course preserving current independent administrative structures of these providers • Although ESDE will focus on Oklahoma, it can serve as an important framework for national leadershipin Earth System data, including and especially in the social sciences

13. Value Proposition of ESDE • Easy access, within a single portal framework, to a wide array of important data and tools for research, education, and applied development • A strong marketing tool and competitive advantage for Oklahoma in proposing or hosting field campaigns, seeking major testing facilities (e.g., FAA UAS initiative), and developing research proposals to utilize the data (e.g., Climate Science Center) • EISE also is a mechanism for greatly enhancing • Oklahoma’s participation in major activities such as the Big Data Initiative, innovative STEM education, NSF’s EarthCube and DataOne projects, and efforts to broaden participation by underrepresented groups in STEM • Strengthening capabilities in, and attracting outstanding students to work on, innovative and exciting problems in data, IT and Earth System research • Attracting and supporting private businesses that depend upon Earth System data, processed information, or advanced decision support environments • Improving collaboration across disciplines as well as interaction across organizations that provision data (e.g., NOAA, USGS, USDA, OU, OSU, DOT, DOE)

14. Foundational Components • #1: A Light-Touch Administrative Framework • Simple MOUs, as necessary, between OU and participating data providers, both for access to data and use of organization names and logos in “marketing” • Note that if data generally are available to the broad community (e.g., satellite, physiography, population), formal agency participation or approval will not be necessary • A small governance group, drawn from participating organizations that self-selects a chair,to oversee the effort and maintain communication with all stakeholders • A technical coordination group, with time volunteered from participating organizations, to conceive the architecture/design and manage the development process

15. Foundational Components • #2. A web-based portal that provides the following (this is only a partial list) • A complete physical and technical description of each data set, along with meta data • Instructions, as well as code, for accessing each data set in its native format, applicable to both archived and streaming data • An interactive GIS-based map of all data assets • Graphical displays in the portal of live data • Hourly assimilated data set based upon all available data • A 12-month archive of all data with a user-friendly retrieval system • Information about how to use each data set for educational purposes, including examples and sample problems (could be tested in OU courses) • Access points to other data resources nationally

16. Foundational Components • #3: Funding and Personnel • Participating organizations will contribute technical and administrative personnel time as in-kind support • OU will provide hardware and initial funding for programmers to develop and manage the data portal as this project can be a key starting point for an institutional repository • External funding will be sought from a variety of sources to develop and support the system, including a supplement to the new EPSCoR Track-1 proposal • Seek additional partners (NIST may be very interested as it works more on environmental observation standards)

17. Initial Steps (some in parallel) • Obtain agreement from NWC leaders and others (e.g., Director of new Climate Science Center, Dean of Libraries, CIO) regarding the concept • Refine the concept and present to other potential participants to establish a relatively small Phase I cohort (e.g., OU, NOAA, USGS) • Create a comprehensive inventory of available data • Develop a set of functional requirements for the data portal and create a preliminary design, leveraging previous work (LEAD, CyberCommons) and new initiatives (EarthCube, DataOne) • Create a project roadmap with phases, specific goals, milestones, cost estimates, and staffing requirements • Develop an Alpha version of the portal, beginning with a few data sets most readily available (e.g., NEXRAD, OK Mesonet, ARM) • Create a branding and marketing strategy • Deploy to a limited set of test users, collect information about experiences, and proceed with the design cycle