U.S. Department of the Interior U.S. Geological Survey

1. OPeNDAP Services – Present and Future at LP DAAC Brian Davis1, Rob Quenzer1, Jason Werpy2 , David Meyer3, Chris Doescher3, Chris Torbert3 1Stinger-Ghaffarian Technologies, contractor to U.S. Geological Survey (USGS) Earth Resources Observation and Science (EROS) Center, Sioux Falls, SD,2Information Dynamics, contractor to the USGS EROS Center, 3 USGS EROS Open-source Project for a Network Data Access Protocol (OPeNDAP) services allow Earth Science data users of the LP DAAC to directly access data in a flexible, open manner that lets them customize data selection and delivery. Users can focus more on what they do with the data and less on how to find, download, arrange, manipulate, and alter it to fit their needs. OPeNDAP Prototype Environment The LP DAAC began offering OPeNDAP services to select science stakeholders in 2013. This has allowed us to evaluate OPeNDAP as a data access protocol and investigate what types of data and access methods LP DAAC users will require in the future. All LP DAAC MODIS tile data was made available and some of the LP DAAC’s MODIS tile data was aggregated in to full time series coverage data sets. Future Production Environment The production environment that is being planned to provide OPeNDAP services to the public will utilize a cluster of servers to maintain performance. The servers will be configured with access to the products on the LP DAAC Data Pool. • Coding/Scripting with OPeNDAP services • Using R and Shiny, the LP DAAC has developed a prototype application to interactively extract Normalized Difference Vegetation Index (NDVI) values derived from a single tile of 500m resolution 16-day composite MODerate resolution Infrared Spectrometer (MODIS) data from both the Terra and Aqua platforms, and graphically plot the variations by date. • Shiny applications have two components: a user-interface definition and a server script. These components define the input controls and reactive functions which generate textual and graphical data descriptions. The R libraries incorporate the connectivity to OPeNDAP services which provide interactive data archive exploration. LP DAAC Prototype OPeNDAP Environment OPeNDAP Aggregation Selection Form Dataset: MOD17A2.005.h10v04.ncml Gpp_1km[0][0], 0.0006, 0.0008, 0.0008, 0.0004, 0.0004, 0.0005, 0.0007, 0.0012, 0.0012, 0.0007, 0.0007, 0.0006, 0.0009, 0.0013, 0.0013, 0.0013, 0.0011, 0.001, 0.001, 0.001 Gpp_1km[0][1], 0.0012, 0.0012, 0.001, 0.0008, 0.0005, 0.0004, 0.0009, 0.0009, 0.0009, 0.0005, 0.0005, 0.0006, 0.0009, 0.0013, 0.0011, 0.001, 0.001, 0.0006, 0.001, 0.001 Gpp_1km[0][2], 0.0012, 0.0012, 0.0011, 0.0007, 0.0008, 0.0006, 0.0009, 0.001, 0.0006, 0.0005, 0.0007, 0.0007, 0.0008, 0.0008, 0.0007, 0.0008, 0.0007, 0.0006, 0.0007, 0.0009 Gpp_1km[0][3], 0.0011, 0.0012, 0.0012, 0.0012, 0.0008, 0.0007, 0.0009, 0.0009, 0.0006, 0.0008, 0.0006, 0.0008, 0.0007, 0.0007, 0.0008, 0.0008, 0.001, 0.001, 0.0013, 0.001 Gpp_1km[0][4], 0.0006, 0.0012, 0.0012, 0.0011, 0.0006, 0.0005, 0.0007, 0.0006, 0.0004, 0.0005, 0.0006, 0.0005, 0.0005, 0.0006, 0.0007, 0.0013, 0.001, 0.0012, 0.0013, 0.0014 Gpp_1km[0][5], 0.0006, 0.0009, 0.0012, 0.0012, 0.0012, 0.0011, 0.0008, 0.0005, 0.0004, 0.0006, 0.0006, 0.0006, 0.0006, 0.001, 0.0012, 0.0012, 0.0012, 0.0012, 0.0012, 0.0014 Gpp_1km[0][6], 0.0009, 0.0012, 0.0011, 0.0011, 0.0007, 0.0006, 0.0005, 0.0004, 0.0006, 0.0006, 0.0005, 0.0005, 0.001, 0.0012, 0.0007, 0.0012, 0.0013, 0.0014, 0.0014, 0.0014 Gpp_1km[0][7], 0.0009, 0.0007, 0.0007, 0.0011, 0.0009, 0.0006, 0.0005, 0.0006, 0.001, 0.001, 0.0007, 0.0006, 0.001, 0.001, 0.0013, 0.0013, 0.0012, 0.0012, 0.0012, 0.0009 Gpp_1km[0][8], 0.0007, 0.0007, 0.0006, 0.0013, 0.001, 0.0006, 0.0006, 0.0009, 0.0009, 0.0008, 0.0005, 0.0006, 0.001, 0.0008, 0.001, 0.0012, 0.0011, 0.0011, 0.0011, 0.0007 Gpp_1km[0][9], 0.0008, 0.0007, 0.0007, 0.0009, 0.001, 0.0007, 0.0006, 0.0009, 0.001, 0.0007, 0.0007, 0.001, 0.0007, 0.0008, 0.0012, 0.001, 0.0007, 0.0007, 0.0009, 0.0016 Gpp_1km[0][10], 0.0008, 0.0008, 0.0007, 0.0011, 0.0009, 0.0009, 0.0007, 0.001, 0.0008, 0.001, 0.001, 0.0007, 0.0009, 0.0008, 0.0007, 0.0007, 0.0011, 0.0015, 0.0019, 0.0018 … Proposed LP DAAC OPeNDAP Cluster Scripting Integration with OPeNDAP through a custom UI R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows, and MacOS. Shiny is an open source R package which provides an elegant and powerful web framework for building web applications using R. Shiny transforms analytical scripts into interactive web applications without requiring HTML, CSS, or JavaScript knowledge. Technologies like R and other scripting languages enable users of LP DAAC data to dynamically interact with datasets and increase the utilization of them for scientific analysis. Production LP DAAC OPeNDAP Use Cases LP DAAC MODIS OPeNDAP Subset Example CIDA GDP Giovanni OPeNDAP access through other tools and partners OPeNDAP services allow other interfaces from outside of the LP DAAC to make LP DAAC data available. The USGS Center for Integrated Data Analytics (CIDA) group’s Geo Data Portal (GDP) can access OPeNDAP datasets from the LP DAAC and provide subsets across time. The Giovanni interface from the Goddard Earth Sciences (GES) Data and Information Services Center (DISC) is planning on using LP DAAC OPeNDAP data access to pull LP DAAC data into its system for viewing and analysis by its users. National Snow and Ice Data Center (NSDIC) is working on accessing LP DAAC data through OPeNDAP to develop an Earth Data Cube. Example R/Shiny Application U.S. Department of the Interior U.S. Geological Survey July 2014