1 / 13

QA of Ontologies

QA of Ontologies. OWL Tutorial December 6 th 2005. The GIGO Problem. Logic may be necessary But it is not sufficient Can not validate all possible inferences in advance Instead must prove: Reasoner is sound Axioms are correct Induction does the rest. Barriers to Ontology QA.

brianna-hassett
Download Presentation

QA of Ontologies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. QA of Ontologies OWL Tutorial December 6th 2005

  2. The GIGO Problem Logic may be necessary But it is not sufficient Can not validate all possibleinferencesin advance Instead must prove:Reasoner is sound Axioms are correct Induction does the rest

  3. Barriers to Ontology QA • Absolute Measures • No ‘Gold Standard’ • Mutual cross validation only as good as the parts • Manual checking error-prone • and can’t measure HOW error prone because of (1) • Comparitive Measures • Better than worse does not imply good • Relative Measures • Provides unequivocal evidence of improvement • But not of proximity to goal • Falling error detection rate does not imply none exist

  4. Types of Axiom Quality Philosophical Rigour Ontological commitment Content correctness Fitness for purpose

  5. Philosophical Rigour • 2500 years of research • Theories of time, mereonomy, containment • Often FOL, so not computable • Similar upper level ontologies • DOLCE, BFO • But not 100% agreement: Realist vs Cognitivist

  6. Ontological commitment • Formally specified semantic equivalence • Logical transformation to canonical form • Semantically equivalent but no logical transform • ‘Fixation of femur by means of inserting pins’ • ‘Insertion of pins to fixate the femur’ • Metamodel rules/commitments • Arbitrary choice of preferred form • Conventions to be applied throughout ontology • And by all applications that use it

  7. Content correctness • Metadata Provenance, lexical annotations etc • Truth • ‘Structure of labial vein’ is-a ‘Superficial vein of face’ • Completeness – ambiguity and omission • Thymus secretes Thymosin; Thymosin is-a Hormone…but omits Thymus is-a endocrine gland • Conciseness • Redundant inclusion of inferrable axioms • Consistency – contradictory, duplicated, circular • endocrine surgery vs endocrine surgeons • Traumatic unilateral amputationUnilateral traumatic amputation

  8. Fitness for Purpose • Best theories no guarantee of usability or utility • Lab experiences no predictor of field behaviour • All for nothing, if user can’t use it • Interrater variability

  9. X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X Fitness for Purpose:Inter-rater variability Headcloth Cloth Scarf Model Person Woman Adults Standing Background Brown Blue Chemise Dress Tunics Clothes Suitcase Luggage Attache case Brass Instrument French Horn Horn Tuba

  10. Fitness for Purpose:Inter-rater variability • Miscoding • Code meaning is inappropriate to thing being described • Instrument definitely not a french horn • Missed coding • Not coding something that could be coded • No code for the table/platform • Overcoding • Code meaning is more detailed than justified • Can the gender really be determined? • Undercoding • Code meaning is less detailed than justified • Brass Instument vs Tuba

  11. Fitness for PurposeWhat ontological properties.. • Increase usablility and utility? • Are a prerequisite for them? • Decrease usability or utility?

  12. Ontology QAHow much quality do we need? Perfection is unattainable Trade-offs between quality and… Performance Cost Maintainability Usability Acceptance Utility

  13. REVIEWExploreconsequences DEVISEQIAlgorithm APPLYQIAlgorithm Need for CQI, not final QAAssure the process not the product ASSESSIdentifyproblems Use/Test cases & exemplars

More Related