CUAHSI and CLEANER Overview: Infrastructure and Informatics

1. CUAHSI and CLEANER Overview:Infrastructure and Informatics Jon Duncan Program Manager

2. Collaborative Large Scale Engineering Analysis Network for Environmental Research (CLEANER) • Project Office funded, based in Arlington, VA • Science, Cyberinfrastructure, Social Science, Sensors, Organization committees • Major infrastructure planning process and more

3. Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) • A consortium of 105 research universities, 6 affiliate members, and 8 int’l affiliates • Incorporated June, 2001 as a non-profit corporation in Washington, DC

4. What is Hydrologic Science? • Expands beyond traditional hydrology • Focus on “why” the earth works as it does, like other earth sciences, moving beyond traditional problem-solving orientation • Embraces parts of hydrology, geomorphology, hydrogeology, biogeochemistry, … • Hydrologic cycle is central organizing principle

5. Purpose • Science Objective: To further predictive understanding of the terrestrial hydrologic cycle and its linkages with climate and biogeochemical cycles • Societal Need: Will there be enough water for the next century? • …of appropriate quality • …to meet society’s needs • …to maintain the integrity of our ecosystems

6. Need for CUAHSI • Larger-scale, longer-term research to advance science • Enable research at disciplinary boundaries • Support of larger research teams • Improve efficiency and effectiveness of data collection and dissemination

7. What CUAHSI is doing? • Hydrologic Information Systems • Observatories • Hydrologic Measurement Facilities • Education and Outreach • Hydrologic Synthesis Center

8. Geomorphologist Biogeochemist Aquatic Ecologist Glaciated Valley DOCQuality? Perifluvial Oligotrophic? Carbon source? Hyporheic exchange? Backwater habitat Redox Zones? Substrate Size, Stability? Benthic Community Thalweg? Mineralogy? Chemistry? Well sorted? Abstractions in Modeling Real World “Digital Observatory” DNA Sequences Water quantity and quality Meteorology Hydrologist Remote sensing Vegetation Survey Conceptual Frameworks Snowmelt Processes? Groundwater Contribution? Physical World Model Representations -Mathematical Formulae -Solution Techniques Geographically Referenced Mapping Q, Gradient, Roughness? Data Representation • Theory/Process Knowledge • Perceptions of this place • Intuition Validation Wetted Perimeter Measurements

9. Time, T North American and Global A data value D 1:1,000,000 scale 1:500,000 scale United States Space, L 1:100,000 scale River Basin 1:24,000 scale Watershed Variable, V 1:1200 scale River reach Point scale A plot Data Representation • Four-dimensional {x,y,z,t} • Continental scope • Multi-scale, multi-resolution • Points, coverages, dynamic fields

10. Digital Continent • Integrating monitoring and research data yields a single body of information for the country • Observatories contribute intensive information to this body • Observatories are placed within context of climate, geology, soils, etc. but are not assumed to be representative of an area. • Digital observatories may be watersheds, aquifers, river reaches, or any region that is part of the continent

11. Observation Stations Map for the US Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System USGS National Water Information System NOAA Climate Reference Network

12. Inference Space • “Transcending place” means testing hypotheses in areas thought to be similar (in some attributes). • Digital continent will enable identification of “similar” areas and (some) data about that spot. • Observatories will enable inference about similar regions (e.g., presumably one can infer more about Delaware R. from Potomac than about Rio Grande).

13. DOs are the foundation of EOs • Collaboration of Mission and Science Agencies • DO contains both monitoring and research data • DO supports hypothesis test, decision support systems, mgmt models • Interdisciplinary communication • Scientists can access multiple conceptualizations to improve understanding • Everyone benefits from context provided • Incentives must exist for people and agencies to want to contribute (and they do!)

14. Water Resource Regions and HUC’s

15. NHDPlus for Region 17E

16. NHDPlus Reach Catchments ~ 3km2 Average reach length = 2km 2.3 million reaches for continental US About 1000 reach catchments in each 8-digit HUC

17. Slope Elevation Mean annual flow Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order Reach Attributes

18. HIS 1.0 (1 Nov 2006) • Point Time Series • Discovery and Publication • Agencies • USGS NWIS • NCDC • EPA Storet • [LTER Trends] • Static Federation to Observatory Test Beds

19. CUAHSI Hydrologic Information System Architecture National HIS – San Diego Supercomputer Center Map interface, observations catalogs and web services for national data sources HIS Server Workgroup HIS – river authority, research centre or univ. Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Personal HIS – an individual scientist or manager HIS Analyst Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

20. Supports data discovery, delivery and publication Data discovery – how do I find the data I want? Map interface and observations catalogs Data delivery – how do I acquire the data I want? Use web services or retrieve from local database Data Publication – how do I publish my observation data? Use Observations Data Model HIS Server

21. National and Workgroup HIS National HIS Workgroup HIS Workgroup HIS has local observations catalogs for coverage of national data sources in its region. These local catalogs are partitioned from the national observations catalogs. National HIS has a polygon in it marking the region of coverage of a workgroup HIS server For HIS 1.0 the National and Workgroup HIS servers will not be dynamically connected.

22. Data Sources NASA Storet Ameriflux Unidata NCDC Extract NCAR NWIS Transform CUAHSI Web Services Excel Visual Basic ArcGIS C/C++ Load Matlab Fortran Access Java Applications http://www.cuahsi.org/his/ Some operational services

23. How Excel connects to ODM Excel HydroObjects CUAHSI Web service • Obtains inputs for CUAHSI web methods from relevant cells. • Available Web methods are GetSiteInfo, GetVariableInfo GetValues methods. parses user inputs into a standardized CUAHSI web method request. converts standardized request to SQLquery. SQL query Observations Data Model Response converts response to a standardized XML. imports VB object into Excel and graphs it converts XML to VB object

24. Data Types Hydrologic Observation Data Geospatial Data (GIS) (Relational database) Digital Watershed Weather and Climate Data Remote Sensing Data (NetCDF) (EOS-HDF)

25. HIS Extensions • Integration of Weather Data • Work with NCAR; prototype on Ohio • Move from gridded to watershed-based delivery of data • Hydrogeology • Constructing stratigraphy for continent • Geologic Framework • Geomorphic and geologic history • Incorporation of human dimension • Transportation; structures • Permits, Toxic Release Inventory, etc. • Flood plain (contribution from real estate sector?) • FEMA Lidar products • Explicit development of AK, HI, PR beyond CONUS

26. Digital Observatories • Data Representation • Need to develop integrated 4-dimensional data base • Add coverages (easy) and fields (more complicated) • Conceptual Frameworks and Modeling • Can single data representation fulfill diverse set of science needs? • Data assimilation techniques to guide sensor deployment and operation

27. Environmental Observatories Two Paths: • Bottom-up- Critical Zone Observatories (CZO). EAR- Hydrologic Science, Geochemistry/Biogeochemistry, Geomorphology • Top-Down- WATERS Network- CLEANER/CUAHSI

28. Observatory Development: Bottom-up EAR Research funds: CZO Solicitation • Two sites selected in 2007 • CUAHSI will push for public data access • If successful, these first two hopefully initial pieces of a prototype observatory network

29. Observatory Development: Top Down Major Research Equipment and Facility Construction (MREFC) • WATERS Network- • Currently 11 testbeds have been either awarded or recommended for award

30. HydroView WATERS Network An initiative of the U. S. National Science Foundation Engineering and Geosciences Directorates Presenter Name, Affiliation Presented at …. Date

31. Environmental Observatories: a new approach for integrated, field-oriented collaborative research at regional to continental scales Rely on advances in: sensors and sensor networks at intensively instrumented sites shared by the research community cyberinfrastructure with high bandwidth to connect the sites, data repositories, and researchers into collaboratories distributed modeling platforms From USGS

32. WATERS NetworkWATer and Environmental Research Systems Network • Joint collaboration between the CLEANER Project Office and CUAHSI, Inc, sponsored by ENG & GEO Directorates at the National Science Foundation (NSF) • CLEANER = Collaborative Large Scale Engineering Analysis Network for Environmental Research • CUAHSI = Consortium of Universities for the Advancement of Hydrologic Science • Planning underway to build a nationwide environmental observatory network using NSF’s Major Research Equipment and Facility Construction (MREFC) funding

33. The Need…and Why Now? Nothing is more fundamental to life than water. Not only is water a basic need, but adequate safe water underpins the nation’s health, economy, security, and ecology. NRC (2004) Confronting the nation’s water problems: the role of research. ● Water use globally will triple in the next two decades, leading to increases in erosion, pollution, dewatering, and salinization. ● Only ~55% of US river and stream miles and acres of lakes and estuaries fully meet their intended uses; ~45% are polluted, mostly from diffuse-source runoff. ● Polluted runoff caused more than 16,000 beach closings and swimming advisories in the US in 2001 ● Floods cause an average of $5.2 billion/year in damage and 80 deaths/year. ● Of 45,000 U.S. wells tested for pesticides, 5,500 had harmful levels of at least one. ● Fish consumption advisories are common in 30 states because of elevated mercury levels.

34. Four critical deficiencies in current abilities: 1. Basic data about water-related processes at the needed spatial and temporal resolution. 2. The means to integrate data across scales from different media and sources (observations, experiments, simulations). 3. Sufficiently accurate modeling and decision-support tools to predict underlying processes and forecast effects of different engineering management strategies. 4. The understanding of fundamental processes needed to transfer knowledge and predictions across spatial and temporal scales—from the scale of measurements to the scale of a desired management action Courtesy of Tom Harmon

35. DRAFT Vision for the WATERS Network: WATERS is an integrated observing system that will transform our ability to predict the quality and quantity of the nation’s waters in real time.

36. CLEANER GRAND CHALLENGE • How do we detect, predict, and manage the effects of human activities and natural perturbations on the quantity, distribution and quality of water in near real time? • Develop capacity to detect effects on water of key drivers: • Population and land use shifts • Energy, water and material resource use • Climate change

37. CUAHSI GRAND CHALLENGE • Develop a predictive understanding of continental water dynamics and its interaction with climate, landscape, ecology and civilization • Three themes in CUAHSI science plan: • Patterns and variability • Climate and human development • Prediction and adaptation to change

38. The WATERS Network will: 1. Consist of: (a) a national network of interacting field sites; across a range of spatial scales, climate and land-use/cover conditions (b) teams of investigators studying human-stressed landscapes, with an emphasis on water problems; (c) specialized supportpersonnel, facilities, and technology; and (d) integrative cyberinfrastructure to provide a shared-use network as the framework for collaborative analysis 2. Transform environmental engineering and hydrologic science research and education 3. Enable more effective management of human-dominated environments based on observation, experimentation, modeling, engineering analysis, and design

39. Common Vision: WATERS Network Informatics Observatories/ Environmental Field Facilities Sensors and Measurement Facility Synthesis

40. Network Design Principles: • Enable multi-scale, dynamic predictive modeling for water, sediment, • and water quality (flux, flow paths, rates), including: • Near-real-time assimilation of data • Feedback for observatory design • Point- to national-scale prediction • Network provides data sets and framework to test: • Sufficiency of the data • Alternative model conceptualizations • Master Design Variables: • Scale • Climate (arid vs humid) • Coastal vs inland • Land use, land cover, population • density • Energy and materials/industry • Land form and geology Nested (where appropriate) Observatories over Range of Scales: Point Plot (100 m2) Subcatchment (2 km2) Catchment (10 km2) – single land use Watershed (100–10,000 km2) – mixed use Basin (10,000–100,000 km2) Continental

41. I I III II I Simplified schema of a potential national WATERS Network based on three hydrologic regions: (I) coastal, (II) humid-continental, and (III) arid-continental; large blue circles: regional, watershed-based observatories; small circles: intensively instrumented field sites

42. Humid Continental Watershed: Potential WATERS EFF site Precipitation Deposition Reservoir Water Treatment Evaporation CAFOs Agricultural Fields Sensors Retention Basin Sensor

43. Sensor Network • Distributed network of interchangeable arrays of remote and embedded stationary and mobile sensors Satellite and Remote Sensing Individual Deployed Sensors

44. WATERS Network Cyberinfrastructure Potential Observatory

45. WATERS Network CI Planning • CLEANER Cyberinfrastructure Committee is creating a CI program plan with CUAHSI • Two major projects are creating CI prototypes for WATERS (with more to come): • CUAHSI Hydrologic Information System (HIS) project (David Maidment, PI) • NCSA Environmental CI Demonstration (ECID) project (Barbara Minsker and Jim Myers co-leads) • Have jointly proposed a draft common environmental CI architecture • Are demonstrating how observatories can enable adaptive forecasting and management

46. Environmental CI Architecture: Research Services Integrated CI ECID Project Focus: Cyberenvironments Supporting Technology Data Services Workflows & Model Services Knowledge Services Meta-Workflows Collaboration Services Digital Library HIS Project Focus Analyze Data &/or Assimilate into Model(s) Link &/or Run Analyses &/or Model(s) Create Hypo-thesis Obtain Data Discuss Results Publish Research Process

47. Collaboration Services: CyberCollaboratory • A web portal to allow sharing of information and ideas across the community. • Currently being used by CLEANER Project Office team • Public CI demo space shows numerous cyberinfrastructure demonstrations http://cleaner.ncsa.uiuc.edu/cybercollab

48. CUAHSI Data Services Web application: Data Portal • Your application • Excel, ArcGIS, Matlab • Fortran, C/C++, Visual Basic • Hydrologic model • ……………. • Your operating system • Windows, Unix, Linux, Mac Internet Simple Object Access Protocol Web Services Library Source: David Maidment, Univ. of Texas

49. Path Forward: Timeline 2007 Joint workshop on integrating social sciences into EOs (January) Public comment period on draft program plan and conceptual design (science, sensor, organization, social science, education, and cyberinfrastructure-February) Completion of program plan and conceptual design (August) 2008 Consortium arrangements established Conceptual Design Review - move to MREFC Readiness Status 2009 Continue funding development of enabling technologies, test-beds, and prototypes FY 2012 Start of MREFC funding and construction phase FY 2016 Project completion; start of full-scale operations

50. Conclusions • Networked observatories will create a new paradigm for environmental research • Shared infrastructure at large scales • Interdisciplinary teams collaborating remotely to address complex environmental issues • Enabling improved understanding & management of large-scale natural environmental systems • Cyberinfrastructure will be critical to the observatory’s goals • Enabling real-time collaboration, data services, modeling, & management • Creating a national knowledge network, not a set of individual field sites and investigators