A posteriori evaluation of Ontology Mapping results Graph-based methods for Ontology Matching

1. A posteriori evaluation of Ontology Mapping resultsGraph-based methodsfor Ontology Matching Ondřej Šváb KIZI for KEG seminar

2. Agenda • Conference track within OAEI-2006 • Initial manual empirical evaluation • Empirical Evaluation via Logical Reasoning • Mapping debugging based on Drago system • Experiments with OntoFarm collection • Consensus Building Workshop • Mining over the mappings with meta-data for KEG seminar

3. Agenda • Conference track within OAEI-2006 • Initial manual empirical evaluation • Empirical Evaluation via Logical Reasoning • Mapping debugging based on Drago system • Experiments with OntoFarm collection • Consensus Building Workshop • Mining over the mappings with meta-data for KEG seminar

4. Conference track - Features • Broadly understandable domain Conference organisation • Free exploration by participants within 10 ontologies • No a priori reference alignment • Participants: 6 research groups for KEG seminar

5. Conference track - Dataset OntoFarm collection http://nb.vse.cz/~svabo/oaei2006/index2.html for KEG seminar

6. Conference track - Participants • 6 participants • Automs • Coma++ • OWL-CtxMatch • Falcon • HMatch • RiMOM for KEG seminar

7. Conference track - Goals • Focus on interesting mappings and unclear mappings • Why should they be mapped? • Arguments: against and for • Which systems did discover them? • Differences in similarity measures • Underlying techniques? for KEG seminar

8. Evaluation • Processing all mappings by hand • Assessment based on personal judgement of organisers (consistency problem) • Tags: TP, FP, interesting, ?, heterogenous mapping • Types of errors and phenomena: • subsumption, inverseproperty, siblings, lexical confusion for KEG seminar

9. Evaluation… • Subsumption mistaken for equivalence • Author,Paper_Author • Conference_Trip, Conference_part • Inverseproperty • has_author,authorOf • Siblings mistaken for equivalence • ProgramCommittee,Technical_commitee • Lexical confusion error • program,Program_chair • Relation – Class mapping • has_abstract,Abstract • Topic,coversTopic; read_paper,Paper for KEG seminar

10. Evaluation… • Some statistics as a side-effect of processing for KEG seminar

11. Evaluation… for KEG seminar

12. Agenda • Conference track within OAEI-2006 • Initial manual empirical evaluation • Empirical Evaluation via Logical Reasoning • Mapping debugging based on Drago system • Experiments with OntoFarm collection • Consensus Building Workshop • Mining over the mappings with meta-data for KEG seminar

13. Mapping debugging • Goal: to improve the quality of automatically generated mapping sets using logical reasoning about mappings • Prototype of the debugger/minimezer implemented on top of the DRAGO DDL reasoner • Semi-automatic process for KEG seminar

14. Drago – Distributed Reasoning Architecture for Galaxy of Ontologies • Tool for distributed reasoning • Based on DDL (Distributed Description Logics) • Services • check ontology consistency, • build classification, • verify concepts satisfiability, • check entailment • Resource: [http://drago.itc.it/] for KEG seminar

15. DDL • Representation framework for semantically connected ontologies • Extension of Description Logics (local interpretation, distributed ,…) • Distributed T-box • Semantic relations represented via directed bridge-rules: • bridge rules: • From the point of view of ontology j for KEG seminar

16. DDL inference mechanism • Extension of tableau algorithm • Inference of „new“ subsumption via ‘subsumption propagation mechanism’ • And its generalized form with disjunctions,… for KEG seminar

17. Drago - architecture DRP=Drago Reasoning Peer peer-peer network of DRPs for KEG seminar

18. Drago - implementation • Ontological language OWL • Mapping between ontologies represented in C-OWL • Distributed Reasoner – extension of OWL reasoner Pellet (http://www.mindswap.org/2003/pellet/) • Communication amongst DRP via HTTP for KEG seminar

19. Mapping debugging • 1st step: diagnosis - detect unsatisfiable concepts (inconsistent ontology) • Assumption: semantically connected ontologies are consistent (without unsatisfiable concepts) • Therefore, unsatisfiable concepts in target ontology are caused by some mappings for KEG seminar

20. Mapping debugging • 2nd step: discovering minimal conflict set • Two conditions: • Set of mappings causing inconsistency and • By removing a mapping, concept is satisfiable • 3rd step: debugging • User feedback • Removing mapping with the lowest degree of confidence • Compute semantic distance of the concept names using WordNet synsets for KEG seminar

21. Mapping debugging • 4th step: minimization • Removing redundant mappings • It leads to minimal mappings set with all the semantics (logically-equivalent minimal version) for KEG seminar

22. Experiments with OntoFarm • Mapping between class names • Six ontologies involved, • Results from four matching systems were analysed • Results of reasoning-based analysis: for KEG seminar

23. Experiments with OntoFarm Interpretation: 1. the lower number of inconsistent alignments, the better quality of mappings 2. this analysis reveal non-obvious errors in mappings obivously incorrect mappings non-obivous errors in mappings for KEG seminar

24. Agenda • Conference track within OAEI-2006 • Initial manual empirical evaluation • Empirical Evaluation via Logical Reasoning • Mapping debugging based on Drago system • Experiments with OntoFarm collection • Consensus Building Workshop • Mining over the mappings with meta-data for KEG seminar

25. Consensus Building Workshop • Discussion about interesting mappings discovered during manual and automatic evaluation • Reaching agreement • Why should they be mapped? • Arguments: against and for • During discussion the following order of arguments were taken into account: • lexical reasons • context of elements (subclasses superclasses, subproperties, superproperties), consider extensions of classes (set interpretation) • Properties related to classes • Axioms (more complex restrictions) for KEG seminar

26. Ilustrative examples • Person vs. Human Against: different sets of subconcepts For: the same domain Result: YES for KEG seminar

27. Ilustrative examples • PC_Member vs. Member_PC Who is the member of Program Committee? Ontologies have different interpretation. Either PC_Chair=Chair_PC or PC_Member=Member_PC result: PC_Chair=Chair_PC Therefore: PC_Member!=Member_PC for KEG seminar

28. Ilustrative examples • Rejection vs. Reject • Both are related to the outcome of the review of a submitted paper • Their position in taxonomy reveal differences in meaning Reccommendation is input Decision is output of the process of revieving for KEG seminar

29. Ilustrative examples • Location vs. Place • Location relates to the country and city where conference is held • Place relates to parts of building where particular events take place • It is need to look at the range and domain restrictions of related properties: Location is domain of properties: locationOf Location is range of properties: heldIn iasted:Place is domain of properties: is_equipped_by sigkdd:Place is range of properties: can_stay_in for KEG seminar

30. Lessons learned • Relevance of context • Lexical matching not enough • Local structure not enough? • Advice: employ semantics, background knowledge (eg. Recommendation and Decision case) • Semantic relations • Equivalent mappings quite often lead to inconsistencies • Many concepts are closely related but not exactly the same • Advice: discover not only equivalent mappings for KEG seminar

31. Lessons learned • Alternative Interpretations (intended meaning) • incomplete specification in ontologies lead to diverse interpretations (PC_Member case), • Advice: check consistency of proposed mappings for KEG seminar

32. Agenda • Conference track within OAEI-2006 • Initial manual empirical evaluation • Empirical Evaluation via Logical Reasoning • Mapping debugging based on Drago system • Experiments with OntoFarm collection • Consensus Building Workshop • Mining over the mappings with meta-data for KEG seminar

33. Mining over the mappings with meta-data • Introduction to Mapping Patterns • Mining • 4ft-Miner • Mining over Mapping Results for KEG seminar

34. Mapping patterns • Deal with (at least) two ontologies • Reflect the structure of ontologies and include mappings between element of ontologies • Mapping pattern is a graph structure • nodes are concepts, relations or instances • Edges are mappings or relation between (domain, range) elements or structural relations between classes (subclasses, siblings) for KEG seminar

35. Mapping patterns - examples • The simplest one • Parent-child triangle for KEG seminar

36. Mapping patterns - examples • Mapping along taxonomy • Sibling-sibling triangle for KEG seminar

37. Mapping patterns - usage • Mining knowledge about habits? • Enhance Ontology Mapping? for KEG seminar

38. 4ft-Miner • Procedure from the LISp-Miner data mining system • This procedure mines for association rules • , where , is antecedent is succedent are condition is 4ft-quantifier – statistical or heuristic test over the four-fold contingency table of and . for KEG seminar

39. Mining over Mapping Results - data • Data matrix Name of mapping system Name of elements in mapping Types of elements (‘c’, ’dp’, ‘op’) Validity of the correspondence Ontologies where elements belong to Types of ontologies (‘tool’, ‘insider’, ‘web’) Manual label – ‘correctness’ (‘+’, ‘-’, ‘?’) Information about patterns in which this mapping plays role • Measure and result of the other mapping from pattern for KEG seminar

40. Mining over Mapping Results – analytic questions • 1. Which systems give higher/lower validity than others to the mappings that are deemed ‘in/correct’? • 2. Which systems produce certain mapping patterns more often than others? • 3. Which systems are more succesful on certain types of ontologies? for KEG seminar

41. Mining over Mapping Results • Output: • Ad 1) • Falcon system: twice more often ‘incorrect’ mappings with medium validity than all systems (on average) • RiMOM and HMatch systems: more ‘correct’ mappings with high validity than all system (on average) • Ad 2) • HMatch: its mappings with medium validity more likely instantiate Pattern 1 than with all validity values of such correspondences • RiMOM: its mappings with high validity more likely instantiate Pattern 2 than with all validity values of such correspondences • Ad 3) • Automs: has more correct mappings between ontologies which are developed according to web-pages, than all systems (on average) • OWL-CtxMatch: has more correct mappings between ontologies which are developed by insiders, than all systems (on average) • ‘on average’ relates to average difference: a(a+b+c+d)/((a+b)(a+c))- 1 for KEG seminar

42. Graph-based methodsfor Ontology Matching (first experience) Ondřej Šváb for KEG seminar

43. Agenda • Graph in Ontology Mapping • Graph Matching Problem • Similarity Flooding • Structural Method for KEG seminar

44. Basic notation and terminology • Graph • V is the set of vertices (nodes) • E is the set of edges (arcs) • Types of graphs • Directed, undirected • Acc. To information connected with nodes and edges • Labelled graph • Attributed graph • Tree is connected graph without circle • Rooted tree, … for KEG seminar

45. Ontology Mapping - formal definition • Ontology contains entities={concepts, relations and instances} • Ontologies O1, O2 consider as directed cyclic graphs with labelled edges, labelled nodes Alignment A is the set of mapped pairs (a,b), where a N1 and b N2. for KEG seminar

46. Simplifacation – example1 Consider just subclass/superclass relation, without multiple inheritance Ontologies as rooted trees for KEG seminar

47. Labels of concepts – example1 for KEG seminar

48. Suggested structure-based technique • Onto2Tree • ExactMatch -> initial mapping (s:Thing=t:Thing) • PropagateInitMappings – using structures of trees and initial mappings to deduce new subsumption relations • … for KEG seminar

49. New subsumptions – example1 for KEG seminar

50. Graph Matching Problem • Graph are used in many fields (effective way of representing objects) • Exact graph matching (isomorphism) • Inexact graph matching (homomorphism) • One-to-one • Many-to-many matching (even more difficult to solve, preferable more concrete results) • complexity problem! – combinatorial nature of graph matching problem for KEG seminar