Formalization of Context and Situation Awareness

1. Formalization of Context and Situation Awareness Mitch Kokar kokar@coe.neu.edu

2. Outline • What is “context”? • Formalizations of context • Situations as contexts • Situation Theory • STO – Situation Theory Ontology • Merging knowledge with context • Examples • Intelligence and context • METT-TC as context WPAFB

3. Context - dictionary Main Entry: con·textEtymology: Middle English, weaving together of words, from Latin contextus connection of words, coherence, from contexere to weave together, from com- + texere to weave 1: the parts of a discourse that surround a word or passage and can throw light on its meaning2: the interrelated conditions in which something exists or occurs WPAFB

4. Context • The same gray square looks whiter against a dark background • and blacker when placed in a bright surround • The manner in which a stimulus is perceived depends not only on its own physical characteristics but also on those of surrounding stimuli and of stimuli previously experienced by the observer. • [Encyclopedia Britannica] WPAFB

5. Another example • “I am a philosopher.” says A to B. • “She is a philosopher” says B to C. • “So you are a philosopher” says C to A • Note that “I”, “she” and “you” all refer to the same object, A. • Go figure, computer! WPAFB

6. Other context definitions • “By context, we mean information about physical characteristics (such as location and network elements), the system (such as applications running and available services), and the user (such as privacy and presence).” [Khedr, Karmouch] • “SCOPES integrates several automated and semi-automated techniques … to facilitate an incremental construction of the knowledge, referred heretofore as context, necessary to translate a query posed against a local database into an equivalent one against a remote database. [Ouksel] • “Lexical polysemy either arises due to the fact that words can be used in different part-of-speeches (syntactical polysemy) or due to words that have varying meanings in different contexts (semantical polysemy).” [Lamparter et al.] WPAFB

7. Why Use Context? • No need to explicitly encode knowledge on everything (McCarthy’s criticism of MYCIN). Use generic rules to “lift” knowledge to appropriate context. • Economy of representation (not one large database) • Inconsistent databases can co-exist, but in different contexts • Efficiency of reasoning – modularization by context • Efficiency in querying (short queries are interpretable in a context) • Lexical ambiguity (same terminology can be used in modules, context resolves ambiguity) • Flexible entailment (closed vs. open world) WPAFB

8. Formalization of “context” • McCarthy: “Whenever we write an axiom, a critic can say that the axiom is true only in a certain context. … usually can devise a more general context in which the axiom doesn’t hold.” • Guha: Formalized ist(C,p) – “proposition p is true in context C”. • Make context a first-class object – manipulate • Same context can be merged with different knowledgebases • Lifting axioms – relating truth in different contexts (axioms about context) WPAFB

9. Handling context formally • Giunchiglia: LMS – Local Model Semantics – inference rules for reasoning about context • Buvac: PLC – Propositional Logic of Context – lifting axioms WPAFB

10. Barwise & Devlin • Context is just Situation Type • If Claire rubs her eyes then Claire is sleepy. Unless there is pollen involved! WPAFB

11. Situations – First Class Citizens “One of the starting points for situation semantics was the promotion of real situations from second class citizens to first class citizens.” “By a situation, then, we mean a part of reality that can be comprehended as a whole in its own right – one that interacts with other things. By interacting with other things we mean that they have properties or relate to other things.” - Jon Barwise, The Situation in Logic, 1989 WPAFB

12. Situation Theory (Barwise/Perry/Devlin) “Information is always taken to be (…) about some situations, and is (…) built up from discrete information items known as infons.” “Infons are not things that in themselves are true or false. Rather a particular item of information may be true or false about a situation.” s ⊨ ≪R, a1, …, an, 1≫ “situation s “supports” I.e., in situation s objects a1, …, an stand in relation R s ⊨ ≪R, a1, …, an, 0≫ “situation s “supports” I.e., in situation s objects a1, …, an do not stand in relation R Problem: The symbol ⊨ means “supports” in Situation Theory but it means “satisfies” in model theory. WPAFB

13. Formalization of Situation Theory Representation (infons) Situations <<R, o1, o2, l1, t1, 1>> supports s entails supports <<type, s, Situation, l1, t1, 1>> World Agent M. M. Kokar, C. J. Matheus, and K. Baclawski. “Ontology-based situation awareness.” Information Fusion, 10:83-98, 2009. WPAFB

14. Barwise/Devlin Ontology (types) • SIT – situations • RELn – n-place relations • IND – individuals • POL – polarity (0 or 1) • LOC – locations • TIM – temporal locations • PAR – parameters • TYP – type of all types WPAFB

15. Ontologies • What is an Ontology? • Language (vocabulary) of discourse for a given domain • Definition of concepts (classes), individuals and relationships relevant to the problem domain • Captured in a formal language e.g., OWL • Computer processable semantics (so that computers can “understand”) • Data annotation • Markup of data in terms of the classes that entities represent and the values of their properties WPAFB

16. Situation Theory Ontology (STO) WPAFB

17. Situation Types WPAFB

18. Relations in STO WPAFB

19. Role of Observer • Since a situation is perceived by an observer (or starts with an utterance), therefore a situation must include an utterance (or query), as well as background knowledge of the observer. • Query then defines what should and what should not be included in the situation (and thus should, or should not, count as context). WPAFB

20. WPAFB

21. Situation Type Examples • Chasing • Running away (from a lion) • Eating (because food was brought in, Rex stops chasing Fluffy) • Still chasing (something unimportant happens, e.g., Jim’s brother comes in) • Iraq invading Kuwait WPAFB

22. Ontology Mapping & Integration • Still in the laboratory stage • IFF: Information Flow Framework -Robert Kent, Ontologos Consortium (related to the SUO effort) http://www.ontologos.org/ • IF-Map – Kalfoglou and Schorlemmer http://www.aktors.org/technologies/ifmap/ • MAFRA - A MApping FRAmework for Distributed Ontologies http://tw.rpi.edu/wiki/Main_Page • The BUSTER Project, University of Bremen – not active? • Toward Semantic Information Integration - Eduard Hovy, ISI University of Southern California, http://www.isi.edu/natural-language/people/hovy.html • Monika Crubezy - http://www-sop.inria.fr/orion/personnel/Monica.Crubezy/me.html WPAFB

23. Ontology Alignment: General Schema Using “colimit” of Category Theory • Identify what should be unified (morphisms) • “Add” two ontologies (colimit) • Insure integration is logically consistent (consistent maps, obey signatures of operations, obey axioms). • From: “IF-Map: An Ontology-Mapping Method based on Information-Flow Theory,” by Yannis Kalfoglou and Marco Schorlemmer WPAFB

24. Ontology Alignment: Example – Easy! • O1 contains “Car” • O2 contains “Automobile” • And these are synonyms • Easy! WPAFB

25. Ontology Alignment: Difficult! • River and Fleuve are close, but not the same. • Stream and Riviere are close but not the same. • Axioms would need to be preserved in order to translate correctly. • From: “IF-Map: An Ontology-Mapping Method based on Information-Flow Theory,” by Yannis Kalfoglou and Marco Schorlemmer WPAFB

26. Example: Intelligence WPAFB

27. Intelligence and Context • 80% of intelligence data today is from HUMINT • This is all about She-said, He-said … • If reifying – then all can be inferred is who said what • Need to integrate data in order to make inferences about not who said what, but what is true and what’s not • What should be integrated and what not? • Situation Theory of Barwise is all about interpretation of speech acts WPAFB

28. Example Scenario: Iran supplies weapons to Iraq Insurgents? • Military officials provides evidence Iran is supplying EFPs, mortars, other deadly weapons to Iraqi insurgents; Qods forces members in custody • The Telegraph reports a large quantity of Austrian made .50 caliber sniper rifles were seized in Baghdad • IRNA reports that Austrian-made rifles have been found by US military in Iraq. WPAFB

29. Mapping the Example to STO • Utterance situation: • U1: Iran supplies weapons to Iraqi insurgents; • Attributes of u1: time=-070211, source=United Nations • Individuals: Iran, Iranians, Rifle, Weapon, Austria, Austrians, Iraqi Insurgents • Relations (just examples): • delivers(Supplier, Receiver, Product, Year); • purchases(Customer, From, Item, Year, Price) WPAFB

30. Resource Situations • R1: U.S. Military reports a large quantity of Iranian owned rifles were seized in Baghdad • Attributes: time = 070212, source=U.S. Military • R2: Austrian sniper rifles that were exported to Iran have been discovered in the hands of Iraqi terrorists • Attributes: time=070213, source = The Telegraph • R3 (I made it up): Islamic Republic News Agency (IRNA) has reported that Austrian made weapons were found with Iraqi Insurgents • Attributes: time=010213, source=IRNA • Inferences depend on which of the situations, R1, R2 or R3, accepted as a “resource situation” WPAFB

31. Inferences • R1: U.S. Military reports a large quantity of Iranian owned rifles were seized in Baghdad • Does being in Baghdad mean same as being with Iraq Insurgents? • Does Iran make such weapons? • Could infer only what was said by the U.S. Military, but not based upon any background knowledge WPAFB

32. Inferences (2) • R2: Austrian sniper rifles that were exported to Iran have been discovered in the hands of Iraqi terrorists • If combined with R1 (and some more background knowledge) would infer (independently of what US Military has said) that the weapons found in Iraq actually came from Iran WPAFB

33. Inferences (3) • R3: Islamic Republic News Agency (IRNA) has reported that Austrian made weapons were found with Iraqi Insurgents • Could not confirm the claim that Iran delivered the weapons • It is possible that some background knowledge in the (huge) knowledge repository of the system could allow to infer that the weapons actually came directly from Austria (?) WPAFB

34. Intelligence: Another example WPAFB

35. Examples of Reports(modified a bit) • Report 1: Outside the Terminal boundary fence an employee witnessed a woman photographing terminal facility through the boundary fence from the public access area. Employee reported to security. Security approached woman who left scene in a maroon coloured car unable to ascertain registration no. Woman was 20-40 years old wearing blue coloured shorts and black short sleeve top. No other identification. • Report 2: Re: vehicle at A gate. When approached, stated only looking & left immediately. 1 female approx 25- 35 y.o shoulder length brown/blonde hair, thin & average in height, wearing dark clothes. All is in order. Vehicle Description: Maroon 4 door hatch. Unknown badge. P plates displayed. Rego XXX. • Report 3: Port Security Control Centre advised of a female of Asian appearance at the Bridge. She was looking around the area with a set of binoculars. The woman then crossed the bridge by foot. Guard pulled over and approached her to ascertain her business in the area. She informed Guard that she was a bird watcher. The woman was advised that she should not be crossing the bridge by foot. The person left without any further incident. Source: GUIDANCE PAPER FOR REPORTING OF SECURITY INCIDENTS AND EVENTS BY MARITIME INDUSTRY PARTICIPANTS Office of Transport Security, Australian Government. January 2009. WPAFB

36. What can be inferred? • Is the person the same in all reports? • Is the car the same? • Why not? Why yes? • If not, are these events linked? By what relations? • Are all events within the same time interval? • Is gate A part of Terminal? • Is 20 – 40 compatible with 25 – 35? • Is bird watching an illegal/suspicious activity? • What purpose can binoculars be used for? • Is “blue coloured shorts” and “black short sleeve top” same as “dark clothes”? • If two reports are about the same person/event, how can information be integrated (“added”)? • Many other questions … WPAFB

37. Three kinds of inference • Meta-inference • Alignment of concepts (“fence” vs. “boarder”, “approach” vs. “meet”) • Alignment of individuals (“4 door hatch” vs. “maroon car”) • Integration (of separate reports into one; could be “virtually one”) • Inferring implicit facts (materialization) • Door is part of car; color of car in Report 1 is maroon, color of car in Report 2 is maroon, … WPAFB

38. Conclusion • Tried to show that context can modify meaning • Mentioned approaches to formalize context • Tried to support use of Situation Theory to model use of context • Outlined Situation Theory and Situation Theory Ontology • Indicated issues related to merging context with a set of given facts • Suggested the merging can be done using category theory (colimit) • Showed examples of inferences that a computer system could possibly perform in order to support human analysts WPAFB

39. Backup Slides WPAFB

40. OWL Inference Engines • Pellet http://www.mindswap.org/2003/pellet/ • BaseVISor http://www.vistology.com (alpha testers welcome) • KAON2 http://kaon2.semanticweb.org/ • SNARK http://www.ai.sri.com/snark/tutorial/tutorial.html • JTP Java Theorem Prover • Racer http://www.racer-systems.com/index.phtml • CWM W3C Semantic Web's Closed World Machine • ConsVISor www.vistology.com (consistency checker) • … WPAFB

41. OWL+Rules Reasoners • JESS (Java) <- CLIPS (C++)<- OPS-5 (Lisp), rules4J, Drools, etc… • general implementations of Rete Algorithm • Jena • open source RDF/OWL parser/reasoners • OWLIM/TRREE and BaseVISor • Rete Algorithm optimized for RDF triples • Oracle OWL Prime • Forward chaining through iterative SQL queries • Supports OWL 2 RL • Ivan Herman’s OWLRL • “proof of concept implementation” of OWL 2 RL reasoner • http://www.ivan-herman.net/Misc/2008/owlrl/ WPAFB