Barbara Minsker, U of Illinois, Urbana, IL David Maidment, U of Texas, Austin, TX August 30, 2014

1. The WATERS (WATer and Environmental Research Systems Network) Network: A Joint CLEANER and CUAHSI Venture Barbara Minsker, U of Illinois, Urbana, IL David Maidment, U of Texas, Austin, TX August 30, 2014

2. The Need…and Why Now? Nothing is more fundamental to life than water. Not only is water a basic need, but adequate safe water underpins the nation’s health, economy, security, and ecology. NRC (2004) Confronting the nation’s water problems: the role of research. ● Water use globally will triple in the next two decades, leading to increases in erosion, pollution, dewatering, and salinization. ● Major U.S. aquifers (e.g., the Ogallala) are being mined and the resource consumed. ● Only ~55% of the nation’s river and stream miles and acres of lakes and estuaries fully meet their intended uses; ~45% are polluted, mostly from diffuse-source runoff. ● From 1990 through 1997, floods caused more than $34 billion in damages in the U.S. ● Of 45,000 U.S. wells tested for pesticides, 5,500 had harmful levels of at least one. ● Fish consumption advisories are common in more than 30 states because of elevated mercury levels (source: mostly fossil fuel combustion; mercury is a neurotoxin).

3. WATERS Network Grand Challenge from the November Joint CUAHSI/CLEANER Workshop How are water quantity, quality, and related earth system processes affected by natural and human-induced changes to the environment? • How do we detect and predict the effects of natural phenomena and human activities on the quantity, distribution, and quality of water across a range of scales? •  How do we manage, engineer, and adapt to aspects of the urban water cycle to achieve sustainable water use and availability for humans and ecosystems? •  How do hydrologic, biologic, geomorphic and chemical transformations in the atmosphere, surface and subsurface affect water quality over multiple space and time scales?

4. CLEANER Grand Challenge Questions(from All-Hands Meeting, Sept. 20-21, 2005) • How do we detect and predict waterborne hazards in real time? • How do we predict the effects of human activities on the quantity, distribution, and quality of water? • How do we improve water cycle engineering management strategies to provide water quantity and quality to sustain humans and ecosystems?

5. WATERS Network Infrastructure • These questions cannot be adequately addressed without observatory infrastructure: • Network of field sites (15-20 nationwide) • Large spatial scale to study complex environmental systems • Equipped with state-of-the-art sensors and other instrumentation • Technical staff to help with experiments and IT • Linked to national community via cyberinfrastructure • Computer hardware, networks, and software • Creates a “collaboratory” for interdisciplinary teams in different universities to collaborate on large-scale research

6. Investigators Education & Outreach Science & Applications Communities SYNTHESIS WATERS Network Infrastructure CYBERINFRASTRUCTURE & Modeling OBSERVATORIES & Environmental Field Facilities MEASUREMENT FACILITY & Sensor Development

7. The WATERS Network will feature: (a) sites with gradients across the range of human impacts (b) where possible, co-location of minimally impacted sites with other EO field sites (c) nested watersheds ranging from local catchments to major river basins to improve understanding of environmental processes across scales

8. WATERS Network CI Planning • CLEANER Project Office (http://cleaner.ncsa.uiuc.edu; Minsker PI, Jerry Schnoor & Chuck Haas co-PIs) • Cyberinfrastructure Committee (Chairs Jeanne Van Briesen & Tom Finholt) is creating a CI program plan in collaboration with CUAHSI • Two groups are creating CI demonstrations for WATERS Network: • CUAHSI Hydrologic Information System project (Maidment, PI) • NCSA Environmental CI Demonstration (ECID) project (Minsker and Jim Myers co-leads) • We have proposed a draft common environmental CI architecture

9. Environmental CI Architecture: Research Services Integrated CI ECID Project Focus Supporting Technology Data Services Workflows & Model Services Knowledge Services Meta-Workflows Collaboration Services Digital Library HIS Project Focus Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Create Hypo-thesis Obtain Data Discuss Results Publish Research Process

10. Knowledge Services • Help users find information they need quickly and effectively • Includes information from: • Archives of community documents, data, workflows, models, collaboration transcripts, etc. • Metadata, including automatically generated metadata collected from other users’ activities and relationships among the metadata (“knowledge networks”) • Web crawls to create dynamic databases on web sites & topics of interest to the community

11. Knowledge Services (cont’d.) • Includes both “information push” and “pull” • Information push – make referrals to users based on their interests and preferences • Information pull – standard user searches • To provide comprehensive knowledge services, provenance of all user activities must be stored in metadata • History and origin of all products (data, workflows, documents, etc.) stored in a flexible and expandable metadata scheme

12. Knowledge Services: ECID Technology Development • Metadata harvesting from all CI activities enables comprehensive knowledge services • Using RDF & Kowari to log “provenance” (source & links) of all objects in “triples” (subject, object, property) • E.g., graph shows that data2 came from data1 using a meta-workflow

13. Knowledge Services: ECID Technology Development • CI-KNOW (CI Knowledge Networks on the Web) uses metadata to make referrals to users • Built with social networking algorithms (Contractor)

14. Environmental CI Architecture: Research Services Integrated CI Supporting Technology Data Services Workflows & Model Services Knowledge Services Meta-Workflows Collaboration Services Digital Library Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Create Hypo-thesis Obtain Data Discuss Results Publish Research Process

15. CUAHSI Hydrologic Information System A multiscale web portal system for accessing, querying, visualizing, and publishing water observation data and models for any location or region in the United States North American Scale (e.g. North American Regional Reanalysis of climate) 1:1,000,000 scale Multiscale data delivery Continental US Scale (coast to coast data coverage, HIS-USA) 1:500,000 scale 1:100,000 scale Regional Scale (e.g. Neuse basin) Watershed Scale (e.g. Eno watershed ) 1:24,000 scale Site Scale (experimental site level) Site scale Point Point Observation Scale (gage, sampling location)

16. Observatories LTER Ameriflux NCAR NCDC Storet NWIS CUAHSI Web Services Excel Visual Basic ArcGIS C/C++ Matlab Fortran Access SAS Some operational services

17. CUAHSI Web Services Web application: Data Portal • Your application • Excel, ArcGIS, Matlab • Fortran, C/C++, Visual Basic • Hydrologic model • ……………. • Your operating system • Windows, Unix, Linux, Mac Internet Simple Object Access Protocol Web Services Library

18. Direct and Indirect Web Services • Direct web service • The data agency provides direct queryingability into its archives through SOAP or OpenDAP (NCDC) • Indirect web service • CUAHSI constructs a “web page mimic” service, housed at SDSC, that programmatically mimics the manual use of an agency’s web pages (USGS, Ameriflux)

19. Observation Site Files Ameriflux Towers Automated Surface Observing System Climate Reference Network USGS NWIS Stations

20. Observation Site Map for US USGSNWIS ASOS Climate Research Network Ameriflux + others…….

21. Neuse Basin with all points NWIS Streamflow and Water Quality ASOS Ameriflux NWIS Groundwater NARR

22. Filtered Site Map NWIS Streamflow and Water Quality ASOS Ameriflux NARR

23. HydroObjects Library User Application (Excel, ArcGIS, …..) • CUAHSI has developed a HydroObjects Library with web service wrappers that know where to access each web service and how to interpret its output HydroObjects Library with web service wrappers for NWIS, Ameriflux, NCDC, … Direct or Indirect web services Web data

24. Transfer of research results • CUAHSI web services for NWIS were announced at a cyberseminar on Friday Oct 28 • On Wednesday Nov 2, Jason Love, from a private firm, RESPEC, in Sioux Falls, South Dakota, posted on the EPA Basins list server:“Occasionally one comes across something that is worth sharing; the CUAHSI Hydrologic Information Systems - Web Services Library for NWIS is a valuable tool for those of us interested in rapidly acquiring and processing data from the USGS, e.g., calibrating models and performing watershed assessments.” • He provided a tutorial on how to use the services from Matlab (which CUAHSI had not developed) • Technology transfer took less than 1 week!

25. Environmental CI Architecture: Research Services Integrated CI Supporting Technology Data Services Workflows & Model Services Knowledge Services Meta-Workflows Collaboration Services Digital Library Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Create Hypo-thesis Obtain Data Discuss Results Publish Research Process

26. Series and Fields Features Series– ordered sequence of numbers Point, line, area, volume Discrete space representation Time series – indexed by time Frequency series – indexed by frequency Surfaces Fields– multidimensional arrays Continuous space representation Scalar fields – single value at each location Vector fields – magnitude and direction Tensor fields – several vectors Random fields – probability distribution

27. Digital Watershed Hydrologic Observation Series Geospatial Data Digital Watershed Remote Sensing Fields Weather and Climate Fields A digital watershed is an overlay of observation series and fields on a geospatial framework to form a connected database for a hydrologic region

28. ArcGIS ModelBuilder Application for Automated Water Balancing Fields Series Geospatial

29. Environmental CI Architecture: Research Services Integrated CI Supporting Technology Data Services Workflows & Model Services Knowledge Services Meta-Workflows Collaboration Services Digital Library Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Create Hypo-thesis Obtain Data Discuss Results Publish Research Process

30. Meta-Workflow • Many of the observatory efforts involve: • Studying complex environmental systems that require coupling analyses or models of different components of the systems, often created by different people, • E.g., water flows drive contaminant transport in multiple media, and both affect ecological flora and fauna • Real-time, automated updating of analyses and modeling that required diverse tools • E.g., spreadsheets, scripts, GIS tools, models • “Meta-workflow” tools enable heterogeneous workflows to be coupled and run on the desktop, on remote servers, or across the Computational Grid

31. CyberIntegrator: ECID Project’s Meta-Workflow Tool

32. Environmental CI Architecture: Research Services Integrated CI Supporting Technology Data Services Workflows & Model Services Knowledge Services Meta-Workflows Collaboration Services Digital Library Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Create Hypo-thesis Obtain Data Discuss Results Publish Research Process

33. Collaboration Services • Environmental observatories will require: • Tools for easy remote communication, such as: • Wikis for collaborative editing (integrated with word processors) • Instant messenger and chat rooms • Voice over IP and videoconference connections • Screen sharing • Numerous technologies exist, but are not: • Easily integrated with data and analyses for technical discussions • Scalable to large groups across different platforms • Integrated with knowledge services to support collaborative knowledge sharing within & across communities

34. CyberCollaboratory is the ECID project’s prototype collaboration service. To check it out, create an account at http://cleaner.ncsa.uiuc.edu/cybercollab

35. A New Paradigm for Research • WATERS Network will create a new paradigm for environmental research • Shared infrastructure at large scales • Interdisciplinary teams collaborating remotely to address complex environmental issues • New paradigm will enable improved understanding & management of large-scale natural environmental systems • Cyberinfrastructure can create a nationwide knowledge network for environmental researchers • Supports all research & education, regardless of their focus

36. Acknowledgments • Contributors: • CLEANER project office & planning grant teams • NCSA ECID team • CUAHSI HIS team • Funding sources: • NSF grants BES-0414259, BES-0533513, and SCI-0525308, EAR-0413265 • Office of Naval Research grant N00014-04-1-0437