Using Non-Taxonomic Knowledge to Improve Semantic Matching

1. Using Non-Taxonomic Knowledge to Improve Semantic Matching Peter Yeh July 22, 2003

2. Talk Outline • Introduction • Analysis of Existing Techniques • Our Approach • Initial Evaluation • Proposed Work

3. Introduction • Many AI tasks require determining whether two knowledge representations encode the same knowledge.

4. object agent Produce Car Car Acme material Car Car Gold has-part has-part Bumper Bumper material material Gold Gold Information Retrieval • Match queries with documents. Q: “A car with a bumper made of gold.” A: “Acme makes a car made of Gold.”

5. New Knowledge Destroy Destroy causes causes Create Create agent agent object object result result Microbe Microbe Pollution Pollution Food Food Conversion Existing Knowledge subevent subevent next-event Destroy Create agent object result agent Entity Entity Entity Knowledge Acquisition • Match new knowledge with existing knowledge. KB KB: Are you trying to encode a conversion?

6. IF THEN <good, Enemy-Maneuver-Engagement> agent Artillery-Unit agent agent agent agent agent Military-Unit Military-Unit agent Delay Attack causes Block causes causes causes Attack Attack Delay Delay object object object Military-Unit Military-Unit object object Armor-Unit object object agent Advance Rule-based Classification • Match rule antecedents with working memory. For example, Course of Action (COA) critiquing. Pattern COA “This COA has a rating of good for enemy maneuver engagement.”

7. The Core Problem • Solving this matching problem is hard because multiple encodings of the same knowledge rarely match exactly. • Representations don’t match exactly because: • Expressive Ontology. • Knowledge is encoded by different sources. • Knowledge being encoded is complex.

8. Types of Mismatches • Informal examination of a knowledge-base containing: • Patterns. • COAs. • Knowledge-base was built by two Subject Matter Experts (SMEs) participating in DARPA’s RKF project. • Looked for cases of mismatch.

9. Types of Mismatches (cont.) “an armored brigade engaging an armored battalion.” • Taxonomic Differences

10. Types of Mismatches (cont.) “One military unit attacking another unit.” • Taxonomic Differences • Equivalent Alternatives

11. Types of Mismatches (cont.) “Mechanized infantry brigade engaging mechanized infantry battalion.” • Taxonomic Differences • Equivalent Alternatives • Omissions

12. Types of Mismatches (cont.) “Support attack occurs before main attack.” • Taxonomic Differences • Equivalent Alternatives • Omissions • Granularity

13. Analysis of Existing Techniques • Analogy • Inexact Matching • Semantic Matching • Conceptual Indexing • Ontology Merging

14. Analogy • Analogy: mapping of knowledge from a base domain to a target domain. • Structure Mapping Engine (Forbus et. al. 89): • Maps relational knowledge (mappable systems). • Systematicity Principle used to select best analogy. • Analogy based on common generalizations (Leishman 92) • Maps both relational knowledge and object attributes. • Prefers minimal common generalization.

15. Artillery-Unit agent agent agent agent Artillery-Unit agent agent agent Artillery-Unit agent agent agent agent agent agent agent agent agent agent agent Military-Unit Military-Unit agent agent agent Delay Block causes agent agent agent causes causes causes Attack causes Block object causes causes Delay Attack causes Block causes causes causes Attack Attack causes causes object Delay Delay object object object object Armor-Unit object object object object object object object Armor-Unit object object object Military-Unit Military-Unit object object object object Armor-Unit object object object object agent agent Advance Analogy: Structure Mapping Engine

16. Inexact Matching • Inexact Matching: tries to address mismatches between representations • Graph Editing (Tsai et. al. 83, Shapiro and Haralick 81, Messmer et. al. 93, Wolverton et. al. 2003) • Uses edit distance parameters. • Similarity based on shortest sequence of edits. • Partial Matching • Does not require representations to be isomorphic. • Similarity based on amount of structural overlap. • Minimal Common Supergraph (Bunke et. al. 2000) and Maximal Common Subgraph (Bunke and Shearer 98).

17. Artillery-Unit agent agent Artillery-Unit agent Artillery-Unit agent agent agent agent agent agent Military-Unit Military-Unit agent Delay Block causes agent Attack causes Block object Delay Attack causes Block causes causes causes Attack Attack Delay Delay object object Armor-Unit object Armor-Unit object object object Military-Unit Military-Unit object object Armor-Unit object object agent Advance Inexact Matching: MCS

18. Semantic Matching • Semantic Matching: uses knowledge to match representations. • Projection: • Uses taxonomic knowledge. • Ontoseek (Guarino et. al. 99) and ELEN (Huibers et. al. 96). • Projection+: Projection alone is too restrictive • -projection (Genest and Chein 97). • Common generalization, graph splitting, regular expressions (Fargues 92, Buche et. al. 2000, Martin et. al. 2001). • Semantic Overlap • Maximal Joins and Generalizations (Myaeng 92, Poole et. al. 95). • Shared Semantic Structures (Zhong et. al. 2002).

19. agent Artillery-Unit agent Delay Attack Armor-Unit object object agent Artillery-Unit agent agent Artillery-Unit agent agent Artillery-Unit agent agent agent Military-Unit Delay Attack agent agent agent Military-Unit agent Delay Attack causes Block causes causes Attack Delay Delay Attack causes Block causes causes Attack Delay object Armor-Unit object object object object Military-Unit object Armor-Unit object object object Military-Unit object Armor-Unit object object agent Advance agent Advance Semantic Matching: Semantic Overlap

20. Conceptual Indexing • Conceptual indexing: how to organize and index knowledge. • Requires so form matching. • Generalization hierarchy (Bournard et. al. 95, Ellis 92, Levinson 82, Woods 97). • Knowledge indexed by common generalizations. • Generalizations organized hierarchically by subsumption relationships. • Retrieve Most Specific Subsumer (MSS) of a query. • Match procedure is similar to Projection - suffers the same problems.

21. Ontology Merging and Translation • Ontology Merging: merge multiple ontologies built by different sources • Chimaera (McGuinness et. al. 2000) • SMART (Noy and Musen 99). • Ontology Translation: translates a representation from one language to another • Ontomorph (Chalupsky 2000). • Goals are different but share some of the same problems.

22. Our Approach • The goal of this research is to solve the matching problem. • We believe existing semantic approaches can be extended with additional knowledge to significantly improve matching. • What kinds of additional knowledge? • Transformations • Handle mismatches. • Improve matching. • Not taxonomic knowledge.

23. Our Approach (cont.) • Generality and domain-independence. • Want additional knowledge (e.g. Transformations) to be useful across domains. • We believe domain-independence is possible given a reusable domain-neutral upper ontology. • Contains a small set of general concepts. • SMEs use this upper ontology to build KBs on specialized topics (e.g. chemistry, biology, battle space planning). • No training in logic or knowledge representation.

24. Transformations KE SME/KE KB Illustration of Our Framework Ontology Domain-independent KB for the task of matching. KB can be viewed as a domain-specific matcher (e.g. match symptoms to diseases).

25. Our Prototype • Extend semantic matchers with transformations. • Apply transformations in a forward-chaining manner. • Use existing techniques for reasoning with Conceptual Graphs (Corbett et. al. 99, Salvat et. al. 96, Willems 95): • Projection. • Unification. • Graph rules. • Two caveats because existing techniques lead to promiscuous matches.

26. agent agent Person: X Person: X Buy Buy object object Car Car Person: Y Person: Y origin origin Driving-License possesses Driving-License possesses agent Person: Bob Car object Like agent Person: Bob Person recipient Buy object Car Car object Like Person: X agent Buy object Car Car object Sell origin Person: John agent origin Person: John Person agent Transformations that Retains Semantics Projection

27. agent agent Person: X Person: X Buy Buy object object Car Car Car object Sell Person: Y Person: Y origin origin Person: Y agent Driving-License Driving-License possesses possesses object Sell agent agent Car Person: Bob Person: Bob Person recipient Person: John agent Buy Buy object object Car Car Car object object Car Sell Sell object Sell origin origin Person: John Person: John Person Person: Y agent agent agent Transformations that Retains Semantics

28. agent agent Person: X Person: X Buy Buy object object Car Car Car object Sell Person: Y Person: Y origin origin Person: Y agent agent Person Driving-License Driving-License possesses possesses Buy object Car agent agent Person: Bob Person: Bob origin Person Buy Buy object object Car Car object Sell Driving-License possesses origin origin Person: John Person: John agent Rule Applicability

29. agent agent Person: X Person: X Buy Buy object object Car Car Car Car object object Sell Sell Person: Y Person: Y origin origin Person: Y Person: Y agent agent Driving-License Driving-License possesses possesses agent agent Person: Bob Person: Bob Person recipient Buy Buy object object Car Car Car object object Sell Sell origin origin Person: John Person: John Person agent agent Rule Applicability

30. Enumerating Transformations • Transformations derived from our domain-neutral upper ontology. • Enumerated all ways that a relation can be legally used to encode information in a conceptual graph. • Considered whether the same information can be expressed differently. • Enumeration was possible because: • Small upper ontology. • Each concept had well-defined semantics.

31. Transformations Enumerated • We were able to enumerate about 300 transformations. • Resulting transformations fall into three general categories: • Transitivity • Part Ascension • Transfers Through

32. Transformations Enumerated (cont.)

33. object A: Attack Armor-Unit causes B: Attack Block agent C: object Attack Artillery-Unit 1: Attack Military-Unit agent causes agent Artillery-Unit agent 2: D: Delay Attack Delay Artillery-Unit agent agent agent Military-Unit object E: 3: agent Attack Delay Military-Unit Armor-Unit agent agent Delay 4: F: Attack causes Block causes Delay Block Artillery-Unit Military-Unit causes Attack Delay object object G: 5: object Block Delay Military-Unit Armor-Unit object Military-Unit object Armor-Unit H: object causes object Block Delay I: agent Advance Armor-Unit agent Advance Example: Our Approach l1 = l1 = {(1,A)} M = { } {(1,A)}

34. M = { {(1, A)}, {(3,C)}, {(4,D)}, {(5,E)} } C C C C D D D D agent agent agent agent Artillery-Unit Artillery-Unit Artillery-Unit Artillery-Unit agent agent agent agent C D agent agent agent agent agent agent agent agent agent 3 3 3 3 Artillery-Unit agent Military-Unit Military-Unit Military-Unit Military-Unit F F F F agent agent agent agent 4 4 4 4 B B B B H H H H agent agent 3 Military-Unit F agent causes causes causes causes Delay Delay Delay Delay Attack Attack Attack Attack causes causes causes causes Block Block Block Block 2 2 2 2 4 causes causes causes causes Attack Attack Attack Attack B H Delay Delay Delay Delay G G G G object object object object causes Delay Attack causes Block 2 causes Attack E E E E 5 5 5 5 Delay G object 1 1 1 1 Armor-Unit Armor-Unit Armor-Unit Armor-Unit object object object object Military-Unit Military-Unit Military-Unit Military-Unit object object object object object object object object object object object object E 5 A A A A 1 Armor-Unit object Military-Unit object object object I I I I agent agent agent agent Advance Advance Advance Advance A I agent Advance Example: Our Approach

35. causes Action Action causes Action Transformations causes Action Action agent Artillery-Unit agent agent agent Military-Unit agent causes Delay Attack causes Block causes Attack Delay object Armor-Unit object Military-Unit object object object agent Advance causes Action Action causes Action causes Action Action Example: Our Approach

36. agent Artillery-Unit agent agent agent Military-Unit agent causes Delay Attack causes Block causes Attack causes Action Action causes Action Delay object Transformations causes Armor-Unit object Military-Unit object object object causes Action Action agent Advance agent Artillery-Unit agent agent agent Military-Unit agent causes Delay Attack causes Block causes Attack Delay object causes Armor-Unit object Military-Unit object object object agent Advance Example: Our Approach

37. Initial Evaluation • Used our matcher in an application in the domain of battle space planning (DARPA's RKF Project). • The task is to analyze COAs. • Battle space ontology built by extending our upper ontology. • Two military analysts used this ontology to build KBs containing: • Patterns. • COAs. • Our matcher matched the patterns to COAs.

38. Example Output

39. Experiment 1 • Evaluates our first hypothesis. • How significant is the improvement? • Compared our matcher to: • Maximal Common Subgraph (MCS). • Semantic Search Lite (SSL). • Methodology: • 300 domain-neutral transformations; 80 domain-specific transformations. • Matched the patterns to the COAs. • A pattern matches a COA if the match score meets or exceeds a pre-specified threshold. • Used metrics of precision and recall.

40. Experiment 1: Precision

41. Experiment 1: Recall

42. Experiment 2 • Initial evaluation of our second hypothesis. • Assesses the domain independence of using transformations. • Limited - conducted in only one domain, but can still offer some insight. • Methodology: • Divided transformations into 2 groups (domain-neutral vs. domain-specific). • Used domain-neutral transformations to construct DN • Used domain-specific transformations to construct DS • Everything else is the same as Experiment 1.

43. Experiment 2: Precision

44. Experiment 2: Recall

45. Proposed Work • More Comprehensive Evaluation. • Use background knowledge. • Incorporate indexing to make matching more efficient.

46. Comprehensive Evaluation • Evaluate our approach in several applications in four domains. • Four data sets: • Chemistry (Halo). • Biology (RKF). • Battle Space Planning (RKF). • Office Procedures (EPCA). • Three Applications: • Elaboration: Chemistry and Office Procedures. • Question Answering: Biology and Battle Space. • Plan Evaluation: Battle Space and Office Procedures.

47. Ontology Block prevents Move object object Military-Unit Block Background Knowledge • Background Knowledge. • Can be used to normalize new knowledge at acquisition time via a join (Mineau et. al. 93). • Idea can be applied to matching. • Increase similarity. • Two problems: • When should a join be performed? • How to better control the join?

48. Attack causes Block object object Military-Unit Move object Military-Unit Block Block prevents prevents Move Move Attack prevents Move Move object object Attack object object Attack object object Military-Unit Military-Unit object object Military-Unit Military-Unit Military-Unit object Military-Unit Background Knowledge • Background Knowledge. • Can be used to normalize new knowledge at acquisition time via a join (Mineau et. al. 93). • Idea can be applied to matching. • Increase similarity. • Two problems: • When should a join be performed? • How to better control the join?

49. Indexing • Need indexing to make matching more efficient. • A common technique is a generalization hierarchy • Overhead for storage can be expensive. • Finding the MSS can also be expensive. • We intend to study: • How to index knowledge by content? • Other index structures that are more parsimonious.