1 / 36

MBARI’s Video Annotation and Reference System (VARS)

MBARI’s Video Annotation and Reference System (VARS). Brian Schlining and Nancy Jacobsen-Stout. MBARI. MBARI. Monterey Canyon. MBARI Video Data. More than 300 ROV dives per year More than 12,000 hours of video observations generated in 16 years More than 1.6 million individual observations.

marshall-green
Download Presentation

MBARI’s Video Annotation and Reference System (VARS)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MBARI’s Video Annotation and Reference System (VARS) Brian Schlining and Nancy Jacobsen-Stout

  2. MBARI MBARI Monterey Canyon

  3. MBARI Video Data • More than 300 ROV dives per year • More than 12,000 hours of video observations generated in 16 years • More than 1.6 million individual observations

  4. Scientific Applications Video observations and related data • Used by MBARI researchers in well over 200 professional publications and presentations, in addition to countless education and outreach projects • Span multiple sub-disciplines within biology, geology, chemistry “A picture can be worth a thousand words” (if you can find it!) Benthic Diversity Interesting Taxa (gorgonocephalus)

  5. Annotations • Trained science staff identify animal taxonomy, animal behavior, geological features, evidence of human impact, and other objects recorded in the ROV video stream • Annotations are stored in a database shared between MBARI scientists. • Database continues to evolve as our researchers learn more • Work closely with and rely on scientists to keep updated on new discoveries and publications in a timely fashion

  6. Annotation History 1st generation – Free Text • Inconsistent annotations • Difficult to search 2nd generation – VICKI/VIMS • Constrained annotations • Written in Smalltalk • Generated files 3rd generation – VARS • Constrained annotations • Written in Java • Writes information directly to database

  7. VARS - System Components Knowledge Base = hierarchical constraint lexicon & referenceMore than 3500 taxonomic, geologic, technical concepts. Reference media and descriptions. Annotation = catalog video observations, inc. visual mediaUses the KB data to create and edit structured annotations for objects in the video stream. Query = retrieve video-related dataUses the KB to structure simple to complex queries against annotations database

  8. VARS - Knowledge Base Manages constraint lexicon and references

  9. Frame capture (QuickTime for Java) VCR Control (RS-422) VARS - Annotation Generates video-annotations and frame-grabs

  10. Knowledge Base Customizable Interface VARS - Annotation Generates video-annotations and frame-grabs

  11. VARS - Query Searches for and retrieves information and frame-grabs.

  12. VARS - Query Searches for and retrieves information and frame-grabs.

  13. VARS - Query Searches for and retrieves information and frame-grabs.

  14. Vestimentiferan Tubeworms Vesicomyid Clams Example – “Cold Seeps”

  15. “Cold Seep” Map Vintage 1999

  16. “Cold Seep” Map Vintage 2004 Monterey bay is one of the most observed continental margins in the world Visited 0.9% pixels in this map

  17. Example – “Cold Seeps” • Previous “seep” models assumed seeps are associated with faults (fluid low conduits)

  18. Chemosynthetic Biological Communities occur preferentially on steep slopes? Steep slopes imply areas of recent erosion.

  19. VARS Development Goals VARS user goals include: 1) Provide responsive user interfaces to maximize user efficiency and accommodate real- and greater than real-time analyses. 2) Provide a robust system. 3) Provide clear and intuitive user interfaces. VARS developmental goals include: 1) Create maintainable (modular, understandable) software for future developers. 2) Maximize use of existing software (e.g. old VIMS query and commercially available software) 3) Provide a solution that can be exported to other institutions. 4) The database should be simple to maintain by MBARI IS administrators.

  20. VARS Requirements Vars system requirements: • Java 1.4 or greater • Java Comm (or RXTX) • Quicktime for Java (For frame-capture) • Relational Database

  21. VARS Deployment

  22. VARS Deployment Point Lobos Shore Western Flyer

  23. VARS Deployment Database replication Point Lobos Databasereplication Shore Western Flyer

  24. VARS - Annotation data

  25. VARS - Annotation data A related group of tapes. e.g. tapes from a single expedition

  26. VARS - Annotation data A video source, such as a tape or QuickTime movie

  27. VARS - Annotation data A single video frame

  28. VARS - Annotation data An object observed in a particular video-frame. The name is constrained by the knowledgebase

  29. VARS - Annotation data Descriptive information. Can also be used to link ‘concepts’. For example, ‘nanomia eating krill’

  30. VARS - Annotation data Position, physical data, expedition information, and camera data,

  31. VARS - Annotation data

  32. VARS - Knowledgebase data

  33. VARS - Knowledgebase data ~ Hierarchical, Allows representation of relations or phylogeny ~ Linked to descriptive info. (History, authors, images, video, etc. [not shown])

  34. VARS - Knowledgebase data ~ A concept may have one or more concept-names. ~ One concept-name is designated as a primary name. ~ User may annotate with any concept-name. However, only the primary name is stored in the database.

  35. VARS - Knowledgebase data

  36. VARS – Available to you VARS is available to other science institutions. See http://www.mbari.org/vars/ Fall 2005 – Release VARS as open-source December 2005 – MBARI will host a developer workshop If you are interested in using VARS contact: Judith Connor (conn@mbari.org)Director of Information & Technology Dissemination

More Related