HydroShare: An online collaborative environment for the sharing of hydrologic data and models

1. Flow Time Time IN11A-1510 HydroShare: An online collaborative environment for the sharing of hydrologic data and models Models Observers and instruments Visualization Tools Analysis Tools Web Browser HydroDesktop Resource Repository David G. Tarboton, Ray Idaszak, Jeffery S. Horsburgh, Daniel Ames, Jonathan L. Goodall, Lawrence E. Band, VenkateshMerwade, Alva Couch, Jennifer Arrigo, Richard P. Hooper, David W. Valentine, David R. Maidment t RESTful Interface Data Loaders Data Discovery Tools Contact: dtarb@usu.edu Learn more: http://hydroshare.cuahsi.org Drupal GeoAnalytics Framework y OCI-1148453 OCI-1148090 2012-2017 x Resource Centric Paradigm for Modeling and Analysis Collaborative integrated modeling use case Vision and Goal We envision that HydroShare will enable more rapid advances in hydrologic understanding through collaborative data sharing, analysis, and modeling. HydroShare will provide a community collaboration site that enables users to easily discover and access data and models, retrieve them to a desktop computer or perform analyses in a distributed computing environment that includes grid, cloud, or high performance computing model instances as necessary. Understanding will be advanced through the ability to integrate information from multiple sources. Outcomes (data, results, models) can then be published as new resources that can be shared with collaborators. Our goal is to make sharing of hydrologic data and models as easy as sharing videos on YouTube or shopping on Amazon. RESTful Interface iRODS Workflow Structured Objects Policy Enforcement Point Implemented Modeling Services (e.g. SWATShare) P Pre-processing and model linking Flow Indexing Data Grid Computation Time Collaboration Data: Links to national and global data sets of essential terrestrial variables (e.g. NASA NEX, HydroTerre) Tools to preprocess and configure inputs (EcoHydroLib) Preconfigured models and modeling systems as services (SwatShare) Standards for information exchange for interoperability (OpenMI, CSDMS BMI, DataNet Federation Consortium, DataOne) Tools for visualization and analysis Automated reasoning to couple models based on purpose, context, data and resources Coming Soon • Enable multiple models to use common “best practice” tools • Each model interacts with information in the common data store • The modeler does not need to be concerned with and can take advantage of standardized analysis, visualization loading and discovery tools 5 3 4 6 Analysis 7 Collaborative data analysis and publication use case Resource Data Model All content is represented using the Resource Data Model that separates system and science metadata and has elements common to all resources as well as elements specific to the types of resources HydroShare will support. Data Models Architecture and Implementation 1 HydroShare Resource • Open Archives Initiative – Object Reuse and Exchange (OAI-ORE) - defines standards for the description and exchange of aggregations of Web resources • BagIt – hierarchical file packaging format designed to support disk-based or network-based storage and transfer of generalized digital content 2 Publication, Archival, Curation Drupal Content Management System • Extensible Open Source Content Management Framework for Publication written in PHP • Over 14,000 user contributed modules • Themed and Styled Presentation of HydroShare Resources with in page visualization • Off the shelf modules provide a Social Experience surrounding Hydrologic Data: Comments, Ratings, Group Behavior • Custom module development supports HydroShare Data Model, GeoAnalytics and iRODS Integration iRODS • Storage of HydroShare Resources Replicated across multiple institutions • Rule engine for policy driven data management • Extensibility via microservices Resource Map 1 .. * Aggregated Object 1 .. * Aggregation Data and metadata files 1 .. 1 1 .. * Example 1: Time Series as WaterML Example 2: Hydrologic Model Resource Map • Resource map • Science metadata file • Model executable file • Source folder • Many source files • Resource Map • Science Metadata • WaterML file Model.exe Science Metadata Collaborative open source open development using GitHUB and Codeplex • http://github.com/hydroshare/ • http://hydrodesktop.codeplex.com source Data are observed and then loaded into CUAHSI HydroServer. HydroServer stores data in a relational database and publishes it using web services. Metadata is harvested and supports geographic and context based data discovery. A HydroDesktopclient user discovers, downloads and analyzes the data, or uses it in a model. The user next posts the results (data and model) to HydroShare as resources (social objects) retaining provenance information on the original data source. Upon ingestion background actions (microservices) parse metadata and enable analysis based on rules and policies. The user shares posted resources with colleagues (access controlled). Group collaboration and analysis on refining the analysis, model or result. After iteration the result is finalized and submitted for publication. Upon publication the resources produced (data, model, workflow, paper) are made immutable, access is opened and permanent persistent identifiers assigned. source1.f90 source2.f90 source3.f90 sourcen.f90 . . . HydroShare is currently in beta testing. The HydroShare Beta Site located at http://beta.hydroshare.org/ is open for you to use, recognizing the sometimes buggy nature of in development beta systems. So please keep backups of any content in beta.hydroshare.org that is important to you. Please provide us suggestion or feedback on problems you experience so we can improve HydroShare. Your participation as a beta tester is greatly appreciated. http://beta.hydroshare.org Resource Map • Attributes common to all resources • Attributes of Model WaterML Science Metadata • Specifying a resource data type requires • Data content and logical relationships • Name and type of all data and metadata elements • Indicate whether elements are required or optional • Indicate whether elements are from a vocabulary • File formats selection or definition • Attributes of time series (WaterML)