H.Hlomani , M.G.Gillespie , D.Kotowski , D. A. Stacey School of Computer Science

1. Utilizing a Compositional System Knowledge Framework for Ontology Evaluation:A Case Study on BioSTORM H.Hlomani, M.G.Gillespie, D.Kotowski, D. A. Stacey School of Computer Science University of Guelph Guelph, Ontario, Canada

2. Who Are We? • Guelph Ontology Team (GOT) • School of Computer Science, University of Guelph • Website: http://ontology.socs.uoguelph.ca • Research Foci: • Compositional Systems • Workflow Planning • Ontology Discovery and Reuse Knowledge Engineering and Ontology Development 2011

3. Goal of this Presentation • This paper is a case study based upon a framework established in the paper: “A Knowledge Identification Framework for the Engineering of Ontologies in System Composition Processes” (IRI 2011) • To introduce aspects of an ODCS that needs to be considered when designing ontologies • Explain the checklist developed from KIFEO • Application of checklist to BioSTORM Knowledge Engineering and Ontology Development 2011

4. Ontology Driven Compositional System (ODCS) • An Ontology Driven Compositional System is reasons with ontological representations to construct a resultant Source Giliepse et. al. (2011) Knowledge Engineering and Ontology Development 2011

5. ODCS Examples:Semantic Web Services • Automatic Composition of Web Services • Arpinaret al. (2005) • WebService.owl • Process.owl • Domain.owl Knowledge Engineering and Ontology Development 2011

6. ODCS Examples:BioSTORM Agent Composition • Automatic composition of syndromic surveillance software agents • DataSource.owl • SurveillanceMethods.owl • SurveillanceEvaluation.owl Knowledge Engineering and Ontology Development 2011

7. ODCS Examples:Algorithm Composition • Semi-automatic composition of Algorithms • Hlomani & Stacey (2009) • Algorithm.owl - Timeline.owl • Gillespie et al. (2011) • StatisticalModelling.owl • PopulationModelling.owl Knowledge Engineering and Ontology Development 2011

8. Let’s Not Reinvent the Wheel • Each system defines there own way to share knowledge. • Often this method is unique to each system. • However all these systems are trying to accomplish the same thing (even though they may be named different things) • Define Data Architecture • Describe Compositional Units • Define a Workflow Knowledge Engineering and Ontology Development 2011

9. Wouldn’t it be Nice • Method for understanding what knowledge to capture. • To have a basis for evaluating our knowledge bases. • Identifying elements not captured but which may be important as the system evolves. Knowledge Engineering and Ontology Development 2011

10. Knowledge Identification Framework Purpose • Generalize knowledge entities within any type of ODCS • Propose collaborative vocabulary • Assist with Merging and Mapping between ODCS ontologies • Enhance adaptability of future ontologies for ODCS’s

11. Knowledge Identification Framework Five Categories ofKnowledge • Compositional Units • Workflow • Data Architecture • Human Actors • Physical Resources

12. Knowledge Identification Framework Internal vs. External • Compositional Units • Workflow • Data Architecture • Human Actors • Physical Resources

13. Knowledge Identification Framework Internal vs. External • Compositional Units • Work-flow • Data Architecture • Human Actors • Physical Resources

14. Knowledge Identification Framework Syntactic vsSemanticKnowledge Entities • Syntactic entities represent actual objects • Semantic entities represent the realization of those actual objects • Like “Information Realization” ontology design pattern (Gangemi & Prescutti, 2009)

15. Knowledge Identification Framework Semantic KnowledgeEntity Sub-Types • Function • Data • Execution • Quality • Trust

16. Knowledge Identification Framework Relationships between Knowledge Categories • Syntactic Relationships • Semantic Relationships

17. Relationships between Knowledge Categories Syntactic Relationship Example ---- ---- Human Actor Data Architecture Data Architecture Compositional Unit Compositional Unit Data Source requires Input Specification Input Specification has_input Algorithm contains Datum sameAs can_use contains Data Source Person owns

18. Relationships between Knowledge Categories Semantic Relationship Example (Function & Trust) ---- Human Actor Compositional Unit Input Specification SpaceTime Dimension has_feature Algorithm recommends Person trusts_ using works_in OrganizationalRole trusts Person

19. Let’s Apply This Framework • Develop an evaluation tool: • Identifies areas of knowledge or relations which may be missing in key systems ontologies. • In the paper we focused on using the framework to identify whether or not the ontologies designed are adaptable. • We apply our framework to an existing ODCS system: • BioSTORMontologies Knowledge Engineering and Ontology Development 2011

20. What is being Evaluated • There are different aspects of ontology evaluation (Brank 2005, Vrandecic 2009) • Context – considering the aspects of the ontology in relation to other variables in its environment • How well a given aspect satisfies certain criteria • Adaptability – extent to which ontology can be extended without breaking axioms Knowledge Engineering and Ontology Development 2011

21. The Tool • Evaluation Checklist • A tool often used within software quality assurance. • Enables the quantitative analysis of ontologies as well as allows for repeated use. • Used by many industries to ensure the highest quality of there products. Knowledge Engineering and Ontology Development 2011

22. The Tool • The sections of our checklist • Part A: ODCS & Ontology Overview • Part B: ODCS & Categories of Knowledge (Syntax and Semantics) • Part C: Internal Relation Ship • Part D: Human Actor Relationships • Part E: Physical Resource Relationships • Part F: Overall Assessment • Part G: Extra Space for Comments Knowledge Engineering and Ontology Development 2011

23. Methodology: Steps • Review ontologies and supporting documentation (and publications if they exist). • Understand system-specific domain and the domain-specific application. • Run a preliminary overview (Part A of the checklist: ODCS & Ontology Overview). Knowledge Engineering and Ontology Development 2011

24. Methodology: Steps cont’d • For each category of knowledge that exists with the ontologies, document it (Part B). • Consider all possible relationships that could exist between the categories of knowledge (Part C,D, and E). • Provide an overall assessment (Part F) utilizing the evaluation within the checklist. • Additional comments and important points discovered during the review (Part G). Knowledge Engineering and Ontology Development 2011

25. Knowledge Engineering and Ontology Development 2011

26. DISCLAMER! • It is important to note that the ontologies used in BioSTORMwork as intended within that system • This evaluation is not meant to comment about the adaptability of the ontologies within the bioSTORM project but the adaptability of the ontologies if they were to be used within another generic ODCS. Knowledge Engineering and Ontology Development 2011

27. Results • No knowledge of chronological ordering captured within workflow ontologies. • The JADE-CLASS is difficult to adapt, as most ODCS would not use this multi-agent system, thus contextually this CU syntax is difficult to adopt. • The data source ontology have non- domain-specific descriptions and thus this ontology is highly adaptable and can be used for any type of system. Knowledge Engineering and Ontology Development 2011

28. Summary • Knowledge Identification Framework assists in: • Detailing relationships between the categories of knowledge • Both syntactic and semantic • Merging and mapping between ODCS ontologies • Can be used to develop tools to evaluate ontologies. • From our case study we were able to evaluate the ontologies of an existing ODCS and gauge adaptability. Knowledge Engineering and Ontology Development 2011

29. Thank you http://ontology.socs.uoguelph.ca Knowledge Engineering and Ontology Development 2011

30. Examples of Knowledge Entities Compositional Unit Examples Syntactic: • Algorithm, Web Service, System Library Function, Input/Output Specification Semantic: subType::Function(i.e. Domain-specific actions) Data aggregation/conversion/plotting/analysis, Statistical model, Aberrancy detection, etc. subType::ExecutionsubType::Quality Operating systemAverage Runtime

31. Examples of Knowledge Entities Data Architecture Examples Syntactic: • Single Datum, Structured Data, Data Source, Data Set Semantic: • subType:Data • Data Context, Data Context Component • DataSource Structure, DataSourceFileFormat • Data Structure (i.e., Matrix, Vector, Variable) • Data Type • Units of Measure

32. Examples of Knowledge Entities Human Actor Examples Syntactic: • Person, Organization, Recommendation Semantic: subType: Trust • Role (i.e., software developer, domain-expert, novice-user) • Recommendation Context • Organization Type • Organization Governance