1 / 25

External Observatory Integration

External Observatory Integration. Christopher Mueller Life Cycle Architecture Review December 13, 2010. Agenda. Life Cycle Architecture Review Subsystem Purpose Release 1 Product Description Use Case Overview Architectural Overview Status of Progress

honey
Download Presentation

External Observatory Integration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. External Observatory Integration • Christopher Mueller • Life Cycle Architecture Review • December 13, 2010

  2. Agenda Life Cycle Architecture Review • Subsystem Purpose • Release 1 Product Description Use Case Overview • Architectural Overview • Status of Progress • Use Cases Address, Use Cases Demonstrated • Technology Challenges and Achievements • Plan for Construction • Risk Assessment • Demonstration 2

  3. External Observatory Integration Subsystem: Purpose • Integrates external observatories (IOOS in R1) with the OOI Integrated Observatory • Facilitates inclusion of external data • Delivers data in community specific forms • Allows immediate full scale testing of the OOI-CI System • Introduces OOI to a community of early adopters 3

  4. EOI R1 Product Description Use Case Overview Responsible For Supports Uses 4

  5. Dataset Agents  Supporting UC.R1.04 • EOI Supports this use case by providing Dataset Agents that facilitate acquisition of data from external repositories (such as NDBC and AOML) • The Dataset Agents are responsible for mapping between the data format provided by the external repository and the OOI Canonical Data Model 5

  6. UC.R1.21: Derive Data Product Externally • Normalization of disparate data sources • Transformation to the desired output format • Automated delivery of data to the end user (R1 modeling groups) • Automated launching of client-side processing scripts • Resulting data may be sent back to OOI-CI 6

  7. Architectural Overview 7

  8. Architectural Overview – Client View OOI-CI Service Translation Format Conversion Notification Feeds/ Subscriptions 2010 OOI-CI – IOOS Collaboration External Observatory Integration (EOI) Espresso ROMS other models OOI-CI Service Translation Format Conversion Notification Feeds/ Subscriptions External Observatory Integration (EOI) External Observatory Integration (EOI) IOOS Catalog IOOS Catalog SOS TDS SOS SOS WaterML HTTP USGS AOML NDBC Wx NDBC HFR CO-OPS WL IOOS RAs Non-IOOS Sources IOOS Data Sources 8

  9. MARACOOS Integration Design 9

  10. Architectural Overview – OOI-CI View IOOS Modeling Groups User Interfaces Externalization and Integration Instrument Agents External Observatory Integration Application Level Services Sensing & Acquisition Data Distribution and Preservation Infrastructure DataManagement Core Infrastructure Services Common Operating Infrastructure Common Execution Infrastructure 10

  11. Architectural Overview – OOI-CI View IOOS Modeling Groups Application Services Infrastructure Services 2650-00007 OV2 CI 11

  12. Architectural Overview – OOI-CI View Publish(Dataset Agents) Subscribe(Dispatcher) http://oceanobservatories.org/spaces/display/syseng/CIAD+OV+06+External+Observatory+Integration 12

  13. Architectural Overview – Dataset Agent (3) (2) (1) (6) (5) (4) (7) http://oceanobservatories.org/spaces/display/syseng/CIAD+EOI+Dataset+Agents 13

  14. Architectural Overview – Dataset Agent http://oceanobservatories.org/spaces/display/syseng/CIAD+EOI+Dataset+Agents 14

  15. Architectural Overview – Dispatcher http://oceanobservatories.org/spaces/display/syseng/CIAD+EOI+Dispatcher 15

  16. Status of Progress • Implemented end-to-end exchange of messages on both the Publication (Dataset Agent) and Subscription (Dispatcher) sides of the system. • Completed Dataset Agent implementations for all data required by Rutgers modeling group • Completed Dispatcher implementation capable of receiving notification and reacting by launching an external workflow script 16

  17. Status of Progress –Rutgers Data Sources Currently Implemented Capabilities • Providers: • PFEG, NAVY, USGS, RADS, NCEP, AOML, NDBC • Acquisition Protocols: • OPeNDAP, SOS, WaterML, urlread • CDM Types: • Grid, Station, Station Profile, Trajectory • Data Variables: • Sea Surface Height, Currents, Winds, Salinity, Temperature, River Discharge, … • Acquisition Forms: • NetCDF, ASCII 17

  18. EOI Technology List NetCDF Java Matlab 18

  19. Plan for Construction Current Schedule: • Three 6 week Construction Iterations – 12/20/2010  5/2/2011 • IOC Review – week of 5/3/2011 Considering rescheduling to bring EOI into alignment with core subsystems: • Two 8 week Construction Iterations – 12/20/2011  4/1/2011 • EOI will be aligned for the 2nd iteration • R1 Deployment concurrent with the rest of the system http://www.oceanobservatories.org/spaces/display/CIDev/EOI+Release+1+Construction+Plan 19

  20. Plan for Construction – R1C1 Theme:Analysis of 2nd & 3rd Modeling Group & Refinement of Elaboration Implementations • Analysis of modeling group data needs • University of Hawaii • Texas A&M • Integration of mature ION capabilities • Refactoring of Dataset Agent & Dispatcher implementations • Thorough documentation http://www.oceanobservatories.org/spaces/display/CIDev/EOI+Release+1+Construction+Plan 20

  21. Plan for Construction – R1C2 Theme:Development of Additional Dataset Agents & Adoption of OOI-CI as the Data Provider for R1 Modeling Groups • Continued integration of mature ION capabilities • Development of Additional Dataset Agents • For University of Hawaii data • For Texas A&M data • Incorporation of OOI-CI as the data providerin the acquisition scripts of the 3 R1 Modeling Groups http://www.oceanobservatories.org/spaces/display/CIDev/EOI+Release+1+Construction+Plan 21

  22. Risk Assessment – EOI 22

  23. EOI Technologies at Risk for Release 1 None at this time! 23

  24. End-to-end Demonstration For Demo Only!! Translation from raw form to OOI-CI Canonical Data Model Transmission of data via ION Messages https://docs.google.com/drawings/edit?id=1HDMg7rLhic5dIeZ_iWpez8EokfTh0bnFrpNZHXteNYA&hl=en&authkey=CLG0xoMI 24

  25. Thanks ! Questions ? 25

More Related