Collaborative Design of an Image Annotation Tool for Oceanographic Imaging Systems

1. Collaborative Design of an Image Annotation Tool for Oceanographic Imaging Systems Authors: Joe Futrelle, Amber York@ Woods Hole Oceanographic Institution Overview: Analyzing underwater imagery often requires labor-intensive manual annotation of large image sets, but few general-purpose tools exist to support this work. Typically, science teams build specialized “one-off” annotation tools unsuitable for reuse in other projects. Approach: Build collaborative interdisciplinary teams with expertise in oceanographic imaging and informatics. Develop formal use cases representing consensus on needed capabilities and develop and test prototype systems. Web-based prototype annotation tool Imaging instrumentation and science teams Imaging FlowCytobot(Heidi Sosik et al) SeaBED (Hanumat Singh et al) HabCam (Scott Gallager et al) • Use case: • The ability to accurately determine the size, abundance and distribution of scallops is required for responsible management practices that regulate the scalloping industry. Optical imaging systems have proven effective in maximizing survey effort and population estimate accuracy. • The HabCam team needed the ability to identify and measure sea scallops and groundfish using a web-based annotation tool. The users also needed to specify the dominant and subdominant substrate in the image. The annotation tool also had to support to multiple users annotating images at the same time. Peter Fox, RPI Tetherless World Constellation’s iterative development process:Focus on science outcomes allows exploratory design without being constrained by technical considerations; prototyping keeps technical development focused on science outcomes Use and science outcomes: Activity diagram: Shows interaction patterns during system use • HabCam imagery from the US Continental Shelf was annotated with scallop measurements using the web-based annotation tool prototype. Results of the scallop shell measurements are shown below. • A total of 16 users made a total of 275,000 image annotatoins of scallops, groundfish, and substrate. Phytoplankton cytometry / imaging(e.g., estuarine sites) Towed benthic surveys (e.g., NA continental shelf) Autonomous benthic exploration (e.g., sub-Artic ice) Information model: In development workshops, all three teams, in collaboration with information scientists, reached consensus on an information model identifying conceptual relationships that the system needs to support. Implementation considerations are not included at this stage. Full information model contains all relevant concepts Initial prototype focuses on key subset Subsequent prototypes include more concepts as needed Add back concepts, Retain relationships Remove concepts, Retain relationships