1 / 16

Ontology Evaluation, Metrics, and Metadata in NCBO BioPortal

Ontology Evaluation, Metrics, and Metadata in NCBO BioPortal. Natasha Noy Stanford University. What Are We Trying to Do?. You’ve built an ontology, how do you let the world know? You need an ontology, where do you go to get it? How do you know whether an ontology is any good?.

oma
Download Presentation

Ontology Evaluation, Metrics, and Metadata in NCBO BioPortal

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ontology Evaluation, Metrics, and Metadata in NCBO BioPortal Natasha Noy Stanford University

  2. What Are We Trying to Do? • You’ve built an ontology, how do you let the world know? • You need an ontology, where do you go to get it? • How do you know whether an ontology is any good?

  3. BioPortal: A Community-Based Ontology Repository http://bioportal.bioontology.org

  4. BioPortal Today • 195 ontologies in biomedical domain (as of 4/28/2010) • Ontologies in six different formats • ~ 1.7 million classes • ~ 2 million mappings

  5. Evaluating Ontologies in BioPortal • Ontology metrics • Computed automatically • User reviews • Submitted by users: star ratings and free-text reviews along different dimensions • Other metadata • “Stamp of approval” from a specific community

  6. Ontology Metrics • You can find the details here:http://www.bioontology.org/wiki/index.php/Ontology_Metrics • Statistics: • Number of classes • Number of instances • Number of properties • Maximum depth of the class hierarchy • Maximum number of siblings • Average number of siblings • Metrics assessing conformance to best practices: • Classes with a single subclass • Classes with a large number of subclasses (more than 20) • Classes with no definition • Auditing information: • Contributors and authors of individual classes

  7. The Metrics Page in BioPortal You can view metrics for any ontology

  8. Peer Review of Ontologies Which ontology is appropriate for my task? • The people who know the answer to these question are • (maybe) ontology authors • other users of the ontology • Our solution: Allow users to provide ratings for ontologies

  9. Reviewers Provide • General review and rating • Usage information • Which applications have successfully used the ontology? • What problems were encountered? • Coverage • Does it cover the domain properly? • Are there major gaps? • Are some parts developed better than others?

  10. Projects in BioPortal • Users can describe their ontology-based projects in BioPortal • Projects are linked to ontologies they are using • Ontology reviews can be done in the context of projects

  11. Dimensions of a Review • Degree of formality • Documentation and support • Usability • Domain coverage • Correctness • Quality of content

  12. Organizational Stamps of Approval • BioPortal ontologies belong to different groups: • OBO Foundry members and candidates • WHO Family of International Classifications • UMLS • caBIG • We can add other groups…. • Each ontology can be in 0, 1, or more groups • We should (but currently don’t) link to the criteria for inclusion in each group

  13. Maintaining Metadata throughOntology Versioning • We attach reviews to a specific version of an ontology • We expose the reviews for all subsequent versions • We plan to add an indicator “This review was submitted for an earlier version of this ontology” Potentially, any part of the description can change

  14. Representing Metrics and Reviews • Ontology Metadata Vocabulary (OMV): • OMV is a metadata schema that captures salient information about an ontology • OMV is implemented as an OWL ontology • BioPortal Metadata ontology extends OMV to add NCBO-specific metadata and evaluation metrics

  15. BioPortal Metadata Ontology

  16. Discussion • We can extend the BioPortal Metadata ontology to represent • Ontology profiles • Additional metrics • Sets of metrics and criteria for a specific organization • All information in BioPortal is accessible through REST services • Other applications can use the metadata • For example, there can be an external application that compares different profiles • All metadata is specific to an ontology version, but it is accessible through a version-independent ontology id

More Related