Paul Gorman, MD Lois Delcambre, PhD David Maier, PhD

1. Paul Gorman, MD Lois Delcambre, PhD David Maier, PhD DLI2 All Projects Meeting, Stratford Upon Avon 1

2. Bundles in the wild……….. Observational team:Paul GormanJoan AshMary LavelleJason Lyman …………..Bundles in captivity Computer science team:Lois DelcambreDave MaierShawn BowersLongxing DengMathew Weaver DLI2 All Projects Meeting, Stratford Upon Avon 2

3. (Wild) Bundles DLI2 All Projects Meeting, Stratford Upon Avon 3

4. (Wild) Bundles DLI2 All Projects Meeting, Stratford Upon Avon 4

5. (Wild) Bundles DLI2 All Projects Meeting, Stratford Upon Avon 5

6. (Wild) Bundles DLI2 All Projects Meeting, Stratford Upon Avon 6

7. (Wild) Bundles • manage information for diverse, complex tasks • contain selected, collected, structured, annotated • are often used in settings with: • high uncertainty • low predictability • potentially grave outcomes • time & attention are highly constrained DLI2 All Projects Meeting, Stratford Upon Avon 7

8. (Wild) Bundles • There is benefit in creating (active processing of information) • There is benefit in reusing (trigger memory) • There is benefit in sharing (establish collective, situated awareness) DLI2 All Projects Meeting, Stratford Upon Avon 8

9. Let’s take a trip to the ICU DLI2 All Projects Meeting, Stratford Upon Avon 9

10. Given…. • bundles are everywhere! • access to bundles provides access to important information • information in bundles is often copied from other information sources • we can keep copied/referenced information linked through the use of marks DLI2 All Projects Meeting, Stratford Upon Avon 10

11. The superimposed and base layers with marks Superimposed Layer marks Base Layer Information Source1 Information Source2 Information Sourcen … DLI2 All Projects Meeting, Stratford Upon Avon 11

12. (Captive) Bundles • SLIMPad - a scratchpad application to create bundles but….with referenced information connected to the underlying source data • helping us explore architectural issues for building superimposed applications • motivating definition of a metamodel to represent information with mappings to transform • inspired by the observational work (but not focused on a specific medical task) DLI2 All Projects Meeting, Stratford Upon Avon 12

13. SLIMPad demo DLI2 All Projects Meeting, Stratford Upon Avon 13

14. The SLIMPad Architecture Superimposed Information Management SLIMPad Mark Management user XML Viewer MS Power Point MS Excel IE Explorer PDF Viewer Generic Technology Excel Spreadsheets Web Pages XML Documents PPT Files PDF files DLI2 All Projects Meeting, Stratford Upon Avon 14

15. The architecture for managing SLIMPad’s superimposed information. Superimposed Information Management Java Objects (ActiveX) SLIMPad TRIM Store (Java) creates and maintains RDF Triples SLIMPad API (ActiveX) DLI2 All Projects Meeting, Stratford Upon Avon 15

16. Generic Representation of Information Basic Set of Abstractions Metamodel Model Constructs and Relationships XML Topic Map Schema-Level Data DTD Topic Map Defintions Superimposed Layer Instance-Level Data XML Document Topic Map Instances DLI2 All Projects Meeting, Stratford Upon Avon 16

17. Topic Map Example Influenced by by painter Painting Painter mentioned critiqued referenced biography influenced by by painter “Captive” “Paul Klee” critiqued biography biography “Francisco de Goya” http://... http://... http://... referenced mentioned http://... “1914” by painter http://... DLI2 All Projects Meeting, Stratford Upon Avon 17

18. Portion of Structured-Map Model as RDF (instanceOf, “TopicType”, Construct) (instanceOf, “TopicInstance”, Construct) (instanceOf, “ttypename”, Connector) (domain, “ttypename”, TopicType) (range, “ttypename”, String) (domainMult, “ttypename”, “*”) (rangeMult, “ttypename”, “1”) (instanceOf, “topic_instOf”, Conformance) (domain, “topic_instOf”, TopicInstance) (range, “topic_instOf”, TopicType) (domainMult, “topic_instOf”, “*”) (rangeMult, “topic_instOf”, “1”) TopicType ttypename : String 1 <<conformance>> topic_instOf * TopicInstance DLI2 All Projects Meeting, Stratford Upon Avon 18

19. Representing Schema (instanceOf, “painting_tt”, TopicType) (ttypename, “painting_tt”, “painting”) (instanceOf, “painter_tt”, TopicType) (ttypename, “painter_tt”, “painter”) (instanceOf, “byPainter_rt”, TopicRelType) (relType, “byPainter_rt”, “by painter”) (topicType1, “byPainter_rt”, painting_tt) (topicType2, “byPainter_rt”, painter_tt) (instanceOf, “biography_at”, AnchorType) (anchorRole, “biography_at”, “biography”) (topicType, “biography_at”, painter_tt) Topic Types (schema): painting, painter Topic Rel Types (schema): by painter Anchor Types (schema): biography by painter painting painter biography DLI2 All Projects Meeting, Stratford Upon Avon 19

20. Representing Instance (instanceOf, “painter1”, TopicInstance) (title, “painter1”, “Paul Klee”) (topicInsID, “painter1”, “5”) (topic_instOf, “painter1”, painter_tt) (instanceOf, “painting1”, TopicInstance) (title, “painting1”, “Captive”) (topicInsID, “painting1”, “19”) (topic_instOf, “painting1”, painting_tt) (instanceOf, “byPainter1”, TopicRelInst) (rel_instOf, “byPainter1”, byPainter_rt) (topicIns1, “byPainter1”, painting1) (topicIns2, “byPainter1”, painter1) (instanceOf, “biography1”, AnchorInst) (anchor_instOf, “biography1”, biography_at) (address, “biography1”, a1) (instanceOf, “a1”, Address) (markID, “a1”, “URLMarkManager@954308545”) Topic Instances (instance): Paul Klee, Captive Topic Relationship (instance): a by painter relationship Anchor (instance): a biography anchor Address (instance): mark to url DLI2 All Projects Meeting, Stratford Upon Avon 20

21. Model2 Model1 Schema1 Schema1 Instances1 Instances1 Model1 Model1 Schema2 Schema1 Instances1 Instances1 Model2 Model1 Schema2 Schema1 Instances Instances1 Three Basic Types of Mappings Source Target Mapped Inter-Model Converted Converted Inter-Schema Mapped Converted Mapped Model-to-Schema Converted Converted DLI2 All Projects Meeting, Stratford Upon Avon 21

22. (CS Team) Plans • “Schema” for bundles: through templates that predefine certain bundles and labels, and possibly predefine scraps. less restrictive than trad. schema. • Sharing: a bundle that belongs to multiple pads, a pad with multiple simultaneous users, and interchange of pads and bundles between users. • “Scoping” of base layers and marks over those layers. There are several general issues for modeling and management of superimposed information that have emerged from this work and other efforts. DLI2 All Projects Meeting, Stratford Upon Avon 22

23. (CS Team) Plans (cont.) • Distribution: more than a single mark manager. Is there a single space of marks or must applications keep track of which manager handles a particular mark? • Mark Maintenance:when base layer changes • New Kinds of Sources and Marks: beyond XML, PDF, MS Word, MS Excel, MS PowerPoint and HTML. We are also interested in marks into a “base” layer that is actually superimposed information for another application. • Application Interface: try a more query-like API. Also, consider an API generator • Superimposed Information Mapping/Conversion: applying and extending our metamodel. DLI2 All Projects Meeting, Stratford Upon Avon 23

24. www.cse.ogi.edu/footprints • demos - including the QTVR of the ICU (with toys) and SLIMPad • personnel • project description • papers • “Bundles in the Wild: Tools for Managing Information to Maintain Situation Awareness” • “Bundles in Captivity: An Application of Superimposed Information” • papers discussing superimposed information DLI2 All Projects Meeting, Stratford Upon Avon 24