VENUS & NEPTUNE

1. VENUS & NEPTUNE DMAS Progress Report

2. What is VENUS? • Shallow-water test bed for NEPTUNE • But more than just engineering (lots of science topics) • Easy access (Saanich Inlet and Strait of Georgia) • Financed at ~10 MCAD

3. Data line to hi-speed cable Venus line landing in instrument shack Institute of Ocean Sciences - DFO VENUS Project Status • Saanich Inlet wet plant installation end of September • Shore station near IOS (in a DND building!) • About 6-7 different types of instruments initially • Test bed for HDTV camera • Data backhaul 100Mb/s to UVic (budgeted), internet; seeking DF • Strait of Georgia line in 2006

4. What is NEPTUNE? • International Project • ~250 MCAD, 30% Canada-70% US, but ∆T ≥ 3 years • Inter-operability between the two stages • NEPTUNE Canada: First of its kind RCO • ~5 nodes, 100’s instruments, 24/7 ops, 100kW power, ~8Gb/s, 2500m depth, >25 years lifetime • Themes • Science: geology (seismic, tsunamis), gas hydrates, climate, biology • Engineering: power, lifetime, uninterrupted operation, internet in the Ocean, data storage & distribution

5. NEPTUNE Project Status • Wet Plant: • Selection of main contractor for wet plant on-going (BAFO expected soon) • Decision in June, contract in July • Installation in Q2/Q3 ‘07 with a few test instruments • Shore station in Port Alberni, BC • ~40 MCAD available • Instrumentation: • Selection of science instrumentation programs on-going • Most installed in ‘08 • ~13 MCAD available

6. DMAS • DMAS has four elements: • Data acquisition & archival • User data access, selection and retrieval (implementation of observatory data access policy) • Instrument control (exploitation of various instrument modes) • Observatory monitoring, resource management (power & bandwidth) • Budget ~7 MCAD

7. DMAS status • Prototype system with data acquisition, archival, data retrieval completed in February • Prototype currently being adapted for VENUS use (DB redesigned), will evolve with time • Systems, DB Administrator and one software developer recruited • VENUS core hardware & software received VENUS DMAS schedule Data Ingest Basic Query i/f Instrument Commanding Resource Management

8. A few design constraints • DMAS shall implement the data access policy of the observatory • DMAS shall allow users to know about its capabilities (resource discovery) • DMAS shall as much as possible provide users with data in physical units. • Backhaul line may not have the same capacity or reliability as the underwater network

9. DMAS Status • Budget Issue: • Wet plant funds short due to NRE costs not shared with US partner • Savings possibly required from both science instruments and DMAS • Trying to compensate shortfall with responses to call for funding proposals • GEOIDE: ~120KCAD requested for warehouse products evaluation • CANARIE: ~2.3MCAD project on Service Oriented Architecture and workflow management (sensor and archive access through WS, workflow for event management) for DMAS. (With IBM Canada). • CANARIE: ~1.1MCAD project on sub-sea HDTV camera control and data transfer using WS and UCLP

10. DMAS Conceptual Design • System architecture to map wet plant structure: • Extend the Internet into the ocean: tree structure, DNS • Shore station ≠ Data center  backhaul  need for buffering @ shore • Initial software development limited to: • Interfaces with instruments (instrument drivers) • Data subscription (mostly for streaming instruments) • All other elements are from the public domain or off-the-shelf commercial products • Novel approaches to data access (Service-Oriented Architecture) • Use of workflow tools for preparation of event detection recipes

11. DMAS Conceptual Design Users Privil. Public Wet Plant (nodes, ext., instruments Node / extension agents Shore Station Event detect/ reaction agent Data Center Post- Proc. Instrument agents Instrument agents NOC Observatory Ops

12. DMAS Conceptual Design Streaming Instruments Real-time Subscriber Publisher Real-time Subscriber Driver/agent Pre-processing Real-time Subscriber Event detection Publisher Driver/agent Pre-processing Archive Access Metadata DB Archive Subscriber Files Archive Non-streaming Instruments Event detection Driver/agent Archive Access Warehouse DB Driver/agent Driver/agent Ins. Monitoring

13. VENUS Instruments • First Array: (Saanich inlet) • 3 types of CTDs (one with Oxygen sensor) • ADCP • ZAP (vertical and horizontal) • 8MPix moveable digital still camera • 3-way hydrophone array • HDTV camera Scalar Data in warehouse Meta-data in DB Complex Data in file storage Meta-data in DB Streaming

14. DMAS Operations • Ops & data center at UVic (not at shore station) • Shore station mostly unmanned, remotely monitored • Data on-line backup elsewhere on campus for VENUS, (in WA/OR for NEPTUNE?) • Three types of users: • Public  access to “pre-cooked” products (images, plots, …) • General scientist  access to quasi-live or historical data through thin (web browser), special-purpose middleware (e.g., MATLAB with web and DB i/f) or fat clients (data analysis packages with WS access) • PIs  direct access to own instrument for live access, through shore station (bypassing data center).

15. Expected Usage of NEPTUNE/VENUS • Some limited direct interaction to take place: • Occasional control of moveable instruments: cameras, ROVs • Restricted to instrument PI or their team members • But humans typically available only 8 hrs/day! • Expected long-term use mostly through automated event detection and scripted routine maneuvers: • Value checks against hard-coded thresholds or the instrument’s own long-term trends or correlation of multiple instruments outputs • Reactions as commands to e.g., turn on cameras and lights when “something” is detected • Reactions contingent to other phenomena: e.g., activate water column profiler winch when water current strength small enough • Automated extraction from video and audio stream, populating “content databases” that can subsequently be searched (e.g., instantaneous fish count in still images or video)

16. Network Issues • VENUS “Last Mile” • Wet plant capacity will be 1Gb/s. • NEPTUNE “Last Mile” • Wet plant capacity ~8Gb/s • Solutions? • VENUS dark fiber shore station access probably covered by a CANARIE grant (waiting for announcement) • NEPTUNE needs to apply to CANARIE for support to get dark fiber

17. Networking Issues • Access security • Protect assets  authentication, authorizations • US Navy, DND requests: remote access to shore station: re-direction of data flows at specific times

18. Conclusions • VENUS & NEPTUNE development progressing • Projects are exciting from a scientific, technological and education/outreach point of view • Need to convince science community to change its approach towards doing ocean science: exploit the power!