Recent Advances in ViPER

1. Recent Advances in ViPER David Mihalcik Jonathan Shneier David Doermann

2. Work in Progress • Improved SE processes. (Web site, SourceForge, etc.). • Rewriting the GT modules using the API. • Allows for more freedom in individual modules. • Undo/redo support. • API has interface for editing viper data schema. • Already have non-editable timeline and canvas views. • Enhancing PE to support: • ROC curve drawing. • Enhancing “don’t care” functionality to support “really care” rules. Perhaps generalize to a computed “care” factor.

3. Current Hazy Plans • Enhancing API to support: • Plain text search. • Relations. • Generated attributes. • Inheritance? Possibly ontology integration: • OWL or RDFS – see paper on using description logics to enhance multimedia libraries. • Multi-level evaluation of activities. Would like to integrate: • Object-level evaluation. • Action-level evaluation. • Higher-level evaluation.

4. Process Improvement • SourceForge web site: • http://viper-toolkit.sf.net/ • Actively maintaining list of requirements and goals.

5. Timeline View • Based on Jon Heggland’s OntoLog. • Other existing tools include timeline editors in video/sound/multimedia editing tools, Plaisant’s LifeLines, etc.

6. OntoLog

7. Current Timeline

8. Lvalues - Background + Indoors + Cityscape + Outdoors

9. Objects - Objects + Faces + People + Text

10. Need for Ontology Integration • Support for relations. • This event is composed of these activities. • These people participated in these activities. • More generally, this object is related to these other objects. • Support for inheritance. • This object type represents a subset of all objects in this type, e.g. ‘arrive’ is a subclass of ‘move’.

11. Ontology Integration • Using simple description logics, probably getting a third generation, non-compatible ViPER data format in the process. • OWL – the W3C’s web ontology language, is a layer on top of RDF (what OntoLog uses) to support DL reasoning. • Using n3, would look somewhat similar to old format, if a little more difficult to understand the header or parse without an n3 parser. • Using XML, would be even more verbose than new format. • Grew from DAML, so there are already a variety of tools developed at UMCP.

12. Example of Possible Data Format owl:Ontology rdf:about <> ; owl:imports <owl> ; owl:imports <viper> . :FileInformation rdfs:subClassOf viper:FileDescriptor . :named a owl:ObjectProperty ; rdfs:subPropertyOf viper:hasAttribute ; rdfs:label "has file name of" ; rdfs:range :FileInformation ; rdfs:domain. <a.mpg> a viper:Sourcefile ; viper:hasFileDescriptor [ a :FileInformation ; :named [ a :svalue ; :hasValue “a.mpg” ] ] .

13. Activity and Event Evaluation • At the most simple, do signal detection type evaluation, allowing ROC analysis. • For near misses, can do some sort of windowed approach. • Using partial order of activities, can evaluate at different levels of meaning. • Important event -> people event -> steal book. • However, activities can be complex, and may want to evaluate them at multiple levels.

14. Activity Decomposition • Regard each event as an aggregation of smaller activities. • Check how well it segmented the activities: • For each event, compare all related objects/activities. • Get back a score for each event.