1 / 37

Using Resource Description Framework (RDF) to carry metadata for datasets

This resource explains the importance of using RDF to carry metadata for datasets in OpenDAP, including solving structural issues and supporting different languages. It also discusses the benefits of RDF in mapping different data models and enhancing the metadata mapping process.

frankied
Download Presentation

Using Resource Description Framework (RDF) to carry metadata for datasets

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Resource Description Framework (RDF) to carry metadata for datasets M.Benno Blumenthal and John del Corral International Research Institute for Climate and SocietyOpenDAP 2007 http://iridl.ldeo.columbia.edu/ontologies/

  2. RDF is important for OpenDAP because • By embedding OpenDAP in an RDF document, metadata (a.k.a. attributes) not understood by OpenDAP code are easily carried in a semantically-valid way • Explicit relationships between OpenDAP variables can cleanly solve netcdf common name vs OpenDAP GRID/MAP structures, while avoiding retransmission of common independent variables • Explicit mapping between the different data models of the different OpenDAP APIs

  3. RDF is important for OpenDAP because • Support for different languages can be built on top of RDF object support, e.g. Ruby ActiveRDF

  4. Why RDF? Web-based system for interoperating semantics A key part of the Semantic Web RDF/OWL is an interesting technology, but it is even more interesting when it is clear that it can help solve our problems

  5. Standard Metadata Standard Metadata Schema/Data Services Datasets Tools Users

  6. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users Many Data Communities

  7. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users Super Schema Standard metadata schema

  8. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users Super Schema: direct Standard metadata schema/data service

  9. Flaws • A lot of work • Super Schema/Service is the Lowest-Common-Denominator • Science keeps evolving, so that standards either fall behind or constantly change

  10. StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users RDF Standard Data Model Exchange Standard metadata schema RDF RDF RDF RDF RDF RDF

  11. RDF RDF RDF RDF RDF StandardMetadataSchema StandardMetadataSchema StandardMetadataSchema StandardMetadataSchem StandardMetadataSchema RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF Datasets Datasets Datasets Datasets Datasets Tools Tools Tools Tools Tools Users Users Users Users Users RDF Data Model Exchange Standard metadata schema RDF

  12. queries queries queries RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF Architecture Virtual (derived) RDF

  13. Why is this better? • Maps the original dataset metadata into a standard format that can be transported and manipulated • Still the same impedance mismatch when mapped to the least-common-denominator standard metadata, but • When a better standard comes along, the original complete-but-nonstandard metadata is already there to be remapped, and “late semantic binding” means everyone can use the new semantic mapping • Can uses enhanced mappings between models that have common concepts beyond the least-common-denominator • EASIER – tools to enhance the mapping process, mappings build on other mappings

  14. CF attributes NC basic attributes IRIDL attributes SWEET Ontologies CF Standard Names CF Standard Names As Terms IRIDL Terms SWEET as Terms Search Terms Gazetteer Terms

  15. Sample Tool: Faceted Search http://iridl.ldeo.columbia.edu/ontologies/query2.pl?...

  16. Distinctive Features of the search • Search terms are interrelated • terms that describe the set of returns are displayed (spanning and not) • Returned items also have structure (sub-items and superseded items are not shown)

  17. Architectural Features of the search • Multiple search structures possible • Multiple languages possible • Search structure is kept in the database, not in the code http://iridl.ldeo.columbia.edu/ontologies/query2.pl

  18. RDF: framework for writing connections Triplets of • Subject • Property (or Predicate) • Object URI’s identify things, i.e. most of the above Namespaces are used as a convenient shorthand for the URI’s

  19. Datatype Properties {WOA} dc:title “NOAA NODC WOA01” {WOA} dc:description “NOAA NODC WOA01: World Ocean Atlas 2001, an atlas of objectively analyzed fields of major ocean parameters at monthly, seasonal, and annual time scales. Resolution: 1x1; Longitude: global; Latitude: global; Depth: [0 m,5500 m]; Time: [Jan,Dec]; monthly”

  20. Object Properties {WOA} iridl:isContainerOf {Grid-1x1}, {Grid-1x1} iridl:isContainerOf {Monthly}

  21. WOA01 diagram

  22. Standard Properties {WOA} dcterm:hasPart {Grid-1x1}, {Grid-1x1} dcterm:hasPart {MONTHLY} Alternatively {WOA} iridl:isContainerOf {Grid-1x1}, {iridl:isContainerOf} rdfs:subPropertyOf {dcterm:hasPart}

  23. netcdf/CF in RDF Object properties provide a framework for explicitly writing down relationships between data objects/components, e.g. vague meaning of nesting is made explicit Properties also can be related, since they are objects too {SST} rdf:type {cfatt:non_coordinate_variable}, {SST} cfatt:standard_name {cf:sea_surface_temperature}, {SST} netcdf:hasDimension {longitude}

  24. RDF Tools • Transport/Exchange (RDF/XML) • Storage • RDF APIs (Redland,Jena,Sesame) • Query (SPARQL,SeRQL, …) • Basic Semantics

  25. Search Interface Term • http://iri.columbia.edu/~benno/sampleterm.pdf

  26. Ontologies Use Conventions to connect concepts to established sets of concepts Generate additional “virtual” triples from the original set and semantics RDFS – some property/class semantics OWL – additional property/class semantics: more sophisticated (ontological) relationships

  27. OWL Language for expressing ontologies, i.e. the semantics are very important. However, even without a reasoner to generate the implied RDF statements, OWL classes and properties represent a sophistication of the RDF Schema However, there is a serious split in world view from what we have been talking about: concepts as classes vs concepts as individuals

  28. Faceted Search Explicated

  29. Search Interface • Items (datasets/maps) • Terms • Facets • Taxa

  30. Search Interface Semantic API {item} dc:title dc:description rss:link iridl:icon dcterm:isPartOf {item2} dcterm:isReplacedBy {item2} {item} trm:isDescribedBy {term} {term} a {facet} of {taxa} of {trm:Term}, {facet} a {trm:Facet}, {taxa} a {trm:Taxa}, {term} trm:directlyImplies {term2}

  31. Faceted Search w/Queries http://iridl.ldeo.columbia.edu/ontologies/query2.pl?...

  32. queries queries queries RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF RDF Architecture Virtual (derived) RDF

  33. IRI RDF Architecture Data Servers MMI Ontologies JPL bibliography Start Point Standards Organizations RDF Crawler Location Canonicalizer RDFS Semantics Owl Semantics SWRL Rules SeRQL CONSTRUCT Time Canonicalizer Sesame Search Queries Search Interface

  34. CF attributes NC basic attributes IRIDL attributes SWEET Ontologies CF Standard Names CF Standard Names As Terms IRIDL Terms SWEET as Terms Search Terms Gazetteer Terms

  35. RDF is important for OpenDAP because • By embedding OpenDAP in an RDF document, metadata (a.k.a. attributes) not understood by OpenDAP code are easily carried in a semantically-valid way • Explicit relationships between OpenDAP variables can cleanly solve netcdf common name vs OpenDAP GRID/MAP structures, while avoiding retransmission of common independent variables • Explicit mapping between the different data models of the different OpenDAP APIs • Build on language support of RDF objects

  36. Embedded OpenDAP Ontology

  37. Topics/Issues • OpenDAP and RDF: can we transport data semantics without fixing the entire schema? • netcdf/HDF and RDF: do we need non-contextual modeling in our metadata transport/storage? • Concepts as classes vs concepts as individuals • Sub-classes vs sub-categories

More Related