1 / 38

Controlled Ontology Evolution Through Semiotic-based Ontology Evaluation

Controlled Ontology Evolution Through Semiotic-based Ontology Evaluation. International Workshop on Ontology Dynamics IWOD 2008 Renata Dividino dividino@uni-koblenz.de Daniel Sonntag sontag@dfki.de. Overview. Motivation Semiotic-based Ontology Evaluation

osanna
Download Presentation

Controlled Ontology Evolution Through Semiotic-based Ontology Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Controlled Ontology Evolution Through Semiotic-based Ontology Evaluation International Workshop on Ontology Dynamics IWOD 2008 Renata Dividino dividino@uni-koblenz.de Daniel Sonntag sontag@dfki.de

  2. Overview Motivation Semiotic-based Ontology Evaluation Controlled Ontology Evolution through Evaluation Semiotic-based Ontology Evaluation Tool Evaluation Results Conclusion

  3. Motivation = Reason for changes! not good for my intend of use? Apply changes! not good for my application? Apply changes! Inconsistent? Apply changes! not aligned to dependent ontologies? Apply changes!

  4. Motivation = Evaluation criterion! Is the ontology still good (or better) for my intend of use? Which version is the best one for my app? Is the ontology consistent? Can I apply these changes without affecting the dependent ontologies?

  5. Overview Motivation Semiotic-based Ontology Evaluation Semiotics Semiotics & Ontology Semiotics & Ontology Evaluation Controlled Ontology Evolution through Evaluation Semiotic-based Ontology Evaluation Tool Evaluation Results Conclusion

  6. The Meaning Triangle: (Ogden and Richards 1923) Semiotics is composed of three fundamental components: (Moris, 1938) Concept Syntax Semantics Pragmatics Yojo Symbol Object

  7. Semiotic Object Language Ontology Syntax Syntax Semantics Semantics Pragmatics Pragmatics (Niles & Pease, 2001)

  8. Concept Communication Context Rep Concept The ontology’s representation is interpretable by some agent graph-like structures containing terms and their inter-relationships represent an intended conceptualization. Symbol Object Object Intended Conceptualization Symbol Ontology Graph (Gangemi at al, 2006)

  9. Structural Measures • Annotations/Documentation: • Structural-Related: Depth, Breath, Modularity. • Functional-Related: Expert‘s Judgments, Data Set. • User-Oriented: Deployment, Commercial , History/Review, Version. Ontology profile (Usability Dimension) Functional Measures • Depth • Breath • Modularity • … • Consistency • Complexity • Concept Satisfiability • … Syntax (Topological Dimension) Formal Semantics (Logical Dimension) • Expert‘s judgments • User satisfaction • Agreement satisfaction • Data Set • Task Assessment • Modularity Assessment • … Precision-Recall Based Measures(Functional Dimension) Pragmatics Measures Semiotic Measures Assesing the ontology syntax and formal semantics Assesing the ontology cognitive semantics Assesing the ontology pragmatics

  10. Overview Motivation Semiotic-based Ontology Evaluation Controlled Ontology Evolution through Evaluation Semiotic-based Ontology Evaluation Tool Evaluation Results Conclusion

  11. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation (Stojanovic, 2004) Semiotic Ontology Evaluation Process

  12. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Explicit Requirements Reasons for changes = Evaluation Criteria Semiotic Ontology Evaluation Process

  13. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Structural Evaluation

  14. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Making the ontology changes visible in a form of an adequate representation Ontology changes need to be managed such that the ontology remains consistent Structural Evaluation

  15. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Structural Evaluation Functional Evaluation

  16. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Verify consistency of dependent ontologies Ontology changes need to be managed such that the ontology remains consistent Structural Evaluation Functional Evaluation

  17. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Structural Evaluation Functional Evaluation Pragmatics Evaluation

  18. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation User is able to approve the changes applied or to reverse them Structural Evaluation Functional Evaluation Pragmatics Evaluation

  19. Controlled Ontology Evolution through Evaluation Controlled Ontology Evolution through Evaluation Capturing Validation Representation Semantics of change Propagation Implementation Structural Evaluation Functional Evaluation Pragmatics Evaluation

  20. Overview Motivation Semiotic-based Ontology Evaluation Controlled Ontology Evolution through Evaluation Semiotic-based Ontology Evaluation Tool Implementation Evaluation Results Conclusion

  21. Semiotic-based Ontology Evaluation Tool Structural Evaluation Functional Evaluation Pragmatics Evaluation

  22. Semiotic-based Ontology Evaluation Tool Validation Representation Semantics of change Propagation Implementation

  23. Semiotic-based Ontology Evaluation Tool Semiotic Measures Implementation: • Consistency Checking: onto changes remains consistent Representation Semantics of change • Task-based Approach: onto. changes max. performance Semantics of change Propagation • Annotation Analysis: changes reported for versioning Validation

  24. Use of the reasoners RACER System and Pellet. Consistency Checking „…a logical theory is consistent if it does not contain a contradiction, or, more precisely, for no proposition φ are both φ and ¬φ provable.“ Person Seal Shark (primitive class) Animal and eats some (Person and Seal) Disjoint (Person, Seal) Inconsistent

  25. Task-based Approach How effective a given ontology is in the light of a well-defined task (Porzel, 2004; Maedche & Staab,2002) Task Application Ontology Performance Results Compare with Gold Standard Answers Improvements ?

  26. Task-based Approach Evaluate different ontology versions! Is my evolved ontology still good (or better) for my intend of use? Ontology V0.1 changes Ontology V0.2 max. performance for a specific task!

  27. Task-based Approach How efficient is the system to answers questions using just ontologies Question-Answering SWIntO V0.2 SmartWeb Performance Results V0.1 Compare to Gold Standard Answers Improvements

  28. Task-based Approach Evolved SmartWeb Performance Results Compare with Gold Standard Answers Improvements 1. Plug the Evolved Ontology 2. Query the system 3. Check Time Performance • Lexicon • Taxonomy • Semantic Relations 4. Compare with GS Answers 5. Make Report 6. Apply changes!

  29. Usability-Related Evaluation Annotation Analysis: Quantitative analysis of the amount of metadata linked to the tag ”rdf:comments” <owl:AnnotationProperty rdf:ID=“structural“/> <owl:AnnotationProperty rdf:ID=“functional“/> <owl:AnnotationProperty rdf:ID=“user-oriented“/> <owl:AnnotationProperty rdf:ID=“consistency“> <rdfs:subPropertyOf rdf:resource=“#structural“/> </owl:AnnotationProperty> <owl:AnnotationProperty rdf:ID=“task-assessment“> <rdfs:subPropertyOf rdf:resource=“#functional“/> </owl:AnnotationProperty> … <owl:Class rdf:ID="Teacher"> <rdfs:comment>Teacher Class </rdfs:comment> <rdfs:subClassOf rdf:resource="#Person"/> </owl:Class>

  30. Overview Motivation Semiotic-based Ontology Evaluation Controlled Ontology Evolution through Evaluation Semiotic-based Ontology Evaluation Tool Evaluation Results Conclusion

  31. Evaluation Results SWIntO Ontology (SmartWeb Project*) Foundational (DOLCE) and general (SUMO) knowledge Domain- and task-specific knowledge Football (soccer) entities and events Navigation Linguistic information Discourse Multimedia * http://www.smartweb-project.org/ SmartDOLCE:Entity … … SmartSUMO:Attribute SmartSUMO:SocialRole … SportEvent:FootballPlayer SportEvent:FootballOrganizationPerson … … … …

  32. Consistency Checking

  33. Functional Evaluation I

  34. Functional Evaluation II SWIntO V.0.3.2 SWIntO V.0.3.3 Q1:Which matches took place in the semifinals in 1954? Time-performance:31,10 ms GS-performance:26,23 ms Vocabulary Overlap = 100% Hierarchy Overlap = 87% Relation Overlap = 45% Q1:Which matches took place in the semifinals in 1954? Evaluated Relation: GS Relation: Time-performance:31,10 ms GS-performance:26,23 ms Vocabulary Overlap = 100% Hierarchy Overlap = 87% Relation Overlap = 45% List of Overlap Descriptions: Evaluated Relation: GS Relation: Q2:Who was the world champion in 1990? Q2:Who was the world champion in 1990?

  35. Annotation Analysis

  36. Conclusion Evaluation framework to support and control ontology evolution Apply changes to an ontology keeping its quality with respect to the purpose of the ontology (or the purpose of the ontology changes) Controlled evolution by assessing the quality of the ontology with respect to all semiotic dimensions -> Ontology changes captured by ontology evaluation process Implementation = choose three measures which are essential in any ontology evolution/evaluation process Structural Dimension: Consistency Checking Functional Dimension: Task-based Evaluation Usability Dimension: Annotation Analysis Future Work level of granularity & integration

  37. Thank you for your attention!

  38. RACER System, Renamed abox and concept expression reasoner . http://www.racer-system.com R. Porzel and R. Malaka, A task-based approach for ontology evaluation, 2004. M. Ciaramita J. Lehmann A. Gangemi, C. Catenacci, Modelling ontology evaluation and validation. V. Sugumaran A. Burton-Jones, V. C. Storey and P. Ahluwalia, A semiotic metrics suite for assessing the quality of ontologies, 2004. Janez Brank, Marko Grobelnik and Dunja Mladenic, A Survey of Ontology Evaluation Techniques. W. Wahlster, Smartweb: Mobile applications of the semantic web, In Proceedings of Informatik. 2004s. P. Buitelaar and A. Frank,Ontology-driven Predicate-Argument Structure Analysis for Event Annotation John F. Sowa, Ontology, Metadata, and Semiotics, 2000. P. Cimiano,Text Analysis and Ontologies, 2006. C.W. Morris, Foundations of a theory of signs. In: International Encyclopedia of Unified Science (O. Neurath, R. Carnap & C. Morris, eds), Chicago University Press, Chicago, pp. 77-138. 1938. I. Niles and A. Pease .Towards a Standard Upper Ontology, 2001. Y. Sure and R. Studer. On-to-knowledgemethodology - final version. Technical Report Deliverable 18, Institute AIFB, University of Karlsruhe, 2002. A. Gangemi, C. Catenacci, M. Ciaramita, and J. Lehmann. Qood grid: A metaontology-based framework for ontology evaluation and selection. In Proceedings of EON2006, 2006. N. Guarino.. Towards a Formal Evaluation of Ontology Quality, pages 1541–1672. IEEE Intelligent Systems, 2004. D.Orbele et al. Dolce ergo sumo: On foundational and domain models in swinto (smartweb integrated ontology), 2006.. N. Noy. Evaluation by ontology consumers. pages 1541–1672. IEEE Intelligent Systems, 2004. K. Dellschaft and S. Staab. On How to Perform a Gold Standard Based Evaluation of Ontology Learning, 2006 A. Maedche and S. Staab.Measuring Similarity between Ontologies, 2002. L. Stojanovic. Methods and tools for ontology evolution. PhD thesis, University of Karlsruhe (TH), 2004. References

More Related