1 / 27

 The VENUS & NEPTUNE Canada Data Management and Archiving System

A software architecture for sensor networks.  The VENUS & NEPTUNE Canada Data Management and Archiving System. What are VENUS and NEPTUNE?. Cabled Observatories Provide real-time access to the sea floor for monitoring and controlling experiments by extending the Internet underwater

matteo
Download Presentation

 The VENUS & NEPTUNE Canada Data Management and Archiving System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A software architecture for sensor networks  The VENUS & NEPTUNE Canada Data Management and Archiving System

  2. What are VENUS and NEPTUNE? Cabled Observatories Provide real-time access to the sea floor for monitoring and controlling experiments by extending the Internet underwater VENUS: a 3 km line (Saanich Inlet -Feb '06) and a ~30 km (Strait of Georgia – Sep '07) line NEPTUNE Canada: a 800-km loop west of Vancouver Island: installation Fall 2007 & 2008 Built for 25 years of continuous operations

  3. NEPTUNE

  4. What are VENUS and NEPTUNE Canada? VENUS: not just an experiment Has science goals, built by UVic, ~10 MCAD NEPTUNE Canada is built by UVic, leading a consortium of 12 universities spanning the entire country 84 MCAD received from CFI, BCKDF, CANARIE NEPTUNE Canada is the Northern part of a two-country endeavour to build a tectonic plate-wide RCO Juan de Fuca plate is the smallest of the large Earth plates Canada’s part about 30% of total area US part funding now promised but implementation probably 4 years later than Canada's

  5. What are VENUS and NEPTUNE? Backbone Cable Undersea telecom cable (backbone)‏ Provides power (~80 KW)‏ Provides data transmission capability (20 Gbps aggregate bandwidth) to shore

  6. What are VENUS and NEPTUNE? Seafloor nodes and junction boxes The central hubs for connection of instruments Provide power ~9KW to local users 4 Gbps bandwidth available Daisy chains of junction boxes

  7. What are VENUS and NEPTUNE? Instruments VENUS will have 3 nodes, ~30 instruments, ~100 sensors NEPTUNE will have 5 nodes, ~150 instruments, ~500 sensors Instruments produce all sorts of data formats: Scalar (e.g., temperature, chemical ratio)‏ Arrays (e.g., images, velocity vectors)‏ Streams (e.g., sound, video)‏

  8. What are VENUS and NEPTUNE? Shore station Start and/or end of backbone cable Power feed equipment IP (and Sonet gear), fibre illumination equipment Data acquisition software Data buffering End of the backhaul line

  9. What are VENUS and NEPTUNE? Operation centre Hosts observatory control and monitoring Hosts main data archive At UVic Shore station will be “lights-out” management as much as possible

  10. DMAS Data Management and Archive System (DMAS)‏ Software system is the link between the underwater infrastructure and the users Architecture maps the physical layout DMAS Mandate Make data available to researchers in (near-) real time Store data for long term time-series studies

  11. DMAS Features Allow human interaction with instruments Facilitate automated, routine “survey campaigns” Facilitate automated event detection and reaction Store scalar data in data base for easy x-correlation Store complex data in fast file system User access mostly through the web (or specialized client software)‏

  12. DMAS Technologies Services Oriented Architecture Enterprise Service Bus (IBM WebSphere MQ)‏ Web Services Scientific Work Flow tools (e.g., Kepler)‏ Publish-and-subscribe approach RDBMS (Currently Sybase)‏ File Management System (HIA's AD system)‏ Java servlets, Tomcat, Hybernate, Velocity, ...

  13. DMAS Architecture

  14. DMAS Software Development Developed in-house Never been done before at this scale Requirements not easily obtained Requirements changing fast DMAS group has 3 teams Development (7 staff)‏ QA/QC (2 staff)‏ Operations & IT Support (4.5 staff – Sep '07)‏

  15. DMAS Software Development Agile Software development methodology Quick (weekly) release cycles Daily “scrum” stand-up meetings Weekly outstanding bugs review meetings

  16. DMAS Numbers are large VENUS already at over 100 millions individual scalar measurements in 12 months Continuous observation/data collection (petabyte size archive in a few years, mostly due to video material).

  17. Calendar 2006: VENUS installation, supported by iDMAS 2007: VENUS SoG installation, NEPTUNE Canada cable lay 2008: NEPTUNE Instruments installation, support by full DMAS

  18. DMAS future Present “construction” funding to stop Q4 '08 Operational phase to reduce development staff to ~3 FTEs Just bug fixes and general maintenance. To remain leading edge: need to get new resources. Plenty of ideas ... some options

  19. DMAS Value Added: “MyLab” VO: “MyLab” is a portal to Ocean science resource repositories and other relevant resources (e.g., climate, seismic, ...). Allows users to upload their own apps/scripts, in private, in “incubators” or in public spaces on any participating VO member. Collaboration environment provides transparent access to all VO resources (data, processing, visualization, publications). Processing performed by most suitable VO member. Policy: NEPTUNE, VENUS, ... users are provided with credentials to access pool of compute, storage, knowledge resources. Data policy of each data source is respected. Middleware: BPEL or other workflow tools, Web 2.0, WS, Grid access (GT4)‏

  20. DMAS Value Added: AutonomousInstrumentation VO: Underwater instruments are intelligent, make decisions and participate in data collection, analysis, interactive experiments, ... Architecture totally decoupled, computational node all the way to the edge of the network. Policy: Users define the nature of the decision making taking place Partner: Ifremer, France Middleware: Implement small footprint WS, workflow, P&S stacks over real-time applications using PTP.

  21. DMAS Value Added: Ocean “BOFs” VO: Grouping of coastal and estuary observatories who will study (dis-)similarities between their environments Policy: Share of user bases; respect data policies of each Partners: VENUS, Cawthron (NZ), SmartBay (IRL), ... Middleware: Data access WS, extension of the “MyLab” project.

  22. DMAS Value Added: Video/GIS Integration VO: Distributed geo-located multimedia content Real-time video processing. Policy: Focus on under water video Partners: ROPOS Middleware: Develop technology to link video footage with files containing position info. Develop technology to integrate in eg., GoogleMap, ... Storage/Retrieval

  23. DMAS Integrated Projects Chart

  24. Conclusions NEPTUNE Canada is going to be the first, largest ocean observatory Data Management system most modern (thanks also to CANARIE support)‏ Great potential to improve DMAS beyond end of official development phase http://www.neptunecanada.ca/ http://www.venus.uvic.ca/ http://dmas.uvic.ca/

  25. Instrumentation

More Related