1 / 35

NDG Status

NDG Status. Bryan Lawrence (on behalf of a big team). +. +. ]=. +[. +. +. BADC, BODC, CCLRC, PML and SOC. Motivation Standards Feature Types NDG Products Overall Architecture (Taxonomy, Deployment) Discovery Portal Data Extractor MOLES (Security, NumSim) CSML CSML Description

adele
Download Presentation

NDG Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NDG Status Bryan Lawrence (on behalf of a big team) + + ]= +[ + + BADC, BODC, CCLRC, PML and SOC

  2. Motivation Standards Feature Types NDG Products Overall Architecture (Taxonomy, Deployment) Discovery Portal Data Extractor MOLES (Security, NumSim) CSML CSML Description Prototyping in MarineXML Round-Tripping NDG Timeline Outline

  3. NCAR Complexity + Volume + Remote Access = Grid Challenge British Atmospheric Data Centre http://ndg.nerc.ac.uk British Oceanographic Data Centre

  4. Want interdisciplinary semantic access to information, not abstract data getData(potential temperature from ERA-40 dataset in North Atlantic from 1990 to 2000) not: getData(“era40.nc”, ‘PTMP’, 20:50, 300:340, 190:200) or even worse: for j=1990:2000 getData(“era40_”+j+“.nc”, ‘PTMP’, 20:50, 300:340) Lossy is OK! Care less about completeness of representation than semantic unification Integration – semantics

  5. Want decentralised access to resources, implies: Data lies in Multiple Locations Protected by different “user databases” (/etc/passwd, whatever) Differing standards for access control (from none, to restricted to license holders) No overall agreement on access control “roles”! No central user database Needs to be scalable. When NDG started there was NO grid based paradigm that met these requirements. Now Shibboleth and/or Permis meet some of these requirements, but we have designed and prototyped a unique implementation of a new security infrastructure. Integration – Access Control

  6. ISO 19101: Geographic information – Reference model …in a defined logical structure… …delivered through services… …and described by metadata. A geospatial dataset… …consists of features and related objects… Standards

  7. Standards • Geographic ‘features’ • “abstraction of real world phenomena” [ISO 19101] • Type or instance • Encapsulate important semantics in universe of discourse • “Something you can name” • Application schema • Defines semantic content and logical structure • ISO standards provide toolkit: • spatial/temporal referencing • geometry (1-, 2-, 3-D) • topology • dictionaries (phenomena, units, etc.) • GML – canonical encoding [from ISO 19109 “Geographic information – Rules for Application Schema”]

  8. Feature types Application schema ISO 19109 ISO 19103 ISO 19110 Universe of discourse Standards ISO standards • TC211 – around 40 standards for geographic information • Cover activity spectrum: discovery  access  use • Provide a framework for data integration

  9. CSML NCML+CF MOLES THREDDS CLADDIER DIF -> ISO19115 Architecture: NDG Metadata Taxonomy … not one schema, not one solution!

  10. Architecture: Deployment

  11. Discovery Service NDG Products: Discovery Portal http://ndg.nerc.ac.uk/discovery NB: Web Service Interface (you can do the search from your own site and format and present the results there!

  12. Can order responses by Title, Data Centre or Temporal coverage (default random) Choose to return either data or “B-”Metadata Look at DIFs in either HTML or XML

  13. NDG Products: MOLES Ugly as sin! A hint of things to come:

  14. Core linking concept is the deployment MOLES: implementation of a Data Production Tool at an Observation Station on behalf of an Activity that produces a Data Entity Activity DataProductionTool ObservationStation Links the metadata records into a structure that can be turned into a navigable structure Deployment Each of the main metadata objects has security data attached to it. This means that this can be applied to queries on the metadata Data Entity

  15. Simulators as data production tools: NumSim NDG Products: NumSim

  16. NDG Security Assumption – user is known to the AttributeAuthority so no role mapping is necessary NDG Products: NDG Security Proxy Cert. • UserSession • CredWallet SessionManager WS User ID AttributeCertificate User Database AttributeAuthority WS AuthorisationRequest User Roles • Role Map CredWallet Stores any Attribute Certificate acquired Attribute Certificate Attribute Certificate + data URI Access Decision Client Application Gatekeeper WS

  17. NDG Products: DataExtractor

  18. Background activity being parallelised with GODIVA/CCLRC e-science collaboration (spectral -> gridpoint + CDMS + visualisation tools) Download either plot or the data that went into the plot. NDG Products: GEOSPLAT

  19. ERA40: • All driven from one CDML file, 9 TB online spherical harmonics, looking like 40 TB “virtual” gridded!

  20. Aims: provide semantic integration mechanism for NDG data explore new standards-based interoperability framework emphasise content, not container Design principles: offload semantics onto parameter type (‘phenomenon’, observable, measurand) e.g. wind-profiler, balloon temperature sounding offload semantics onto CRS e.g. scanning radar, sounding radar ‘sensible plotting’ as discriminant ‘in-principle’ unsupervised portrayal explicitly aim for small number of weakly-typed features (in accordance with governance principle and NDG remit) NDG-A: Climate Science Modelling Language

  21. CSML feature types defined on basis of geometric and topologic structure Climate Science Modelling Language

  22. CSML feature types examples... ProfileSeriesFeature ProfileFeature GridFeature Climate Science Modelling Language

  23. MarineXML Testbed For each XSD (for the source data) there is an XSLT to translate the data to the Feature Types (FT) defined by CSML. The FT’s and XSLT are maintained in a ‘MarineXML registry’ Phenomena in the XSD must have an associated portrayal Data from different parts of the marine community conforming to a variety of schema (XSD) The FTs can then be translated to equivalent FTs for display in the ECDIS system XSD XML Biological Species S52 Portrayal Library XSD XML Chl-a from Satellite XML Parser MarineGML(NDG) Feature Types XSLT XML XSLT XSLT SENC SeeMyDENC XSD MeasuredHydrodynamics XML XSLT XML XSLT XSLT ECDIS acts as an example client for the data. XSD Data Dictionary XML ModelledHydrodynamics The result of the translation is an encoding that contains the marine data in weakly typed (i.e. generic) Features Features in the source XSD must be present in the data dictionary. XSD Feature described using S-57v3.1Application Schema can be imported and are equivalent to the same features in CSML’ XML S-57v3 GML Slide adapted from Kieran Millard (AUKEGGS, 2005)

  24. MarineXML Testbed Biological sampling station with attributes for the species sampled at each Grid of Chl-a from the MERIS instrument on ENVISAT Predicted and measured wave climate timeseries (height, direction and period) Vectors of currents from instruments Slide adapted from Kieran Millard (AUKEGGS, 2005)

  25. All this requires agreement on standards The Concept of re-using Features Here structured XML is converted to plain ascii text in the form required for a numerical model HTML warning service pages are generated ‘on the fly’ Here the same XML is converted to the SENC format used in a proprietary tool for viewing electronic navigation charts. XML can also be converted to SVG to display data graphically Slide adapted from Kieran Millard (AUKEGGS, 2005)

  26. conceptual model New Dataset Conforms to 101010 UGAS produces <gml:featureMember> <NDGPointFeature gml:id="ICES_100"> <NDGPointDomain> <domainReference> <NDGPosition srsName="urn:EPSG:geographicCRS:4979" axisLabels="Lat Long" uomLabels="degree degree"> <location>55.25 6.5</location> </NDGPosition> </domainReference> </NDGPointDomain> <gml:rangeSet> <gml:DataBlock> <gml:rangeParameters> <gml:CompositeValue> <gml:valueComponents> <gml:measure uom="#tn"/> <gml:measure uom="#amount"/> <gml:measure uom="#gsm"/> </gml:valueComponents> </gml:CompositeValue> </gml:rangeParameters> <gml:tupleList> XML V1.0 will be in NDG Alpha GML app schema GML dataset Application instance parser CSML Round Tripping - 1 Managing semantics

  27. V1.0 in NDG Alpha CF Dataset scanner 101010 CF produces <gml:featureMember> <NDGPointFeature gml:id="ICES_100"> <NDGPointDomain> <domainReference> <NDGPosition srsName="urn:EPSG:geographicCRS:4979" axisLabels="Lat Long" uomLabels="degree degree"> <location>55.25 6.5</location> </NDGPosition> </domainReference> </NDGPointDomain> <gml:rangeSet> <gml:DataBlock> <gml:rangeParameters> <gml:CompositeValue> <gml:valueComponents> <gml:measure uom="#tn"/> <gml:measure uom="#amount"/> <gml:measure uom="#gsm"/> </gml:valueComponents> </gml:CompositeValue> </gml:rangeParameters> <gml:tupleList> XML V1.0 in NDG Alpha GML app schema GML dataset Application instance parser CSML Round Tripping - 2 Managing data - 1

  28. CF Dataset CF Dataset 101010 101010 Define Dataset DECISION PROCESSES <gml:featureMember> <NDGPointFeature gml:id="ICES_100"> <NDGPointDomain> <domainReference> <NDGPosition srsName="urn:EPSG:geographicCRS:4979" axisLabels="Lat Long" uomLabels="degree degree"> <location>55.25 6.5</location> </NDGPosition> </domainReference> </NDGPointDomain> <gml:rangeSet> <gml:DataBlock> <gml:rangeParameters> <gml:CompositeValue> <gml:valueComponents> <gml:measure uom="#tn"/> <gml:measure uom="#amount"/> <gml:measure uom="#gsm"/> </gml:valueComponents> </gml:CompositeValue> </gml:rangeParameters> <gml:tupleList> XML Add Information GML dataset Managing Data 2 scanner XSLT PUBLISH ISO19115

  29. NDG2 runs until September 2007: NDG-Alpha (June 2006) Not all components in place (particularly delivery broker) Not many (maybe only DX) products will be deployable by non-NDG participants (too much hard work installing things that haven’t been optimised for installation) Discovery portal will be (is now) usable, linking to NCAR data etc, but isn’t very user friendly (options not obvious etc). NDG-Beta (Feb 2007) Most components should work, but deployment of software may still be difficult by non-participants NDG-Prod (Jun 2007) Should be deployable and far more user friendly (spending from Feb-June working on deployment and friendliness, no new functionality) Last few months working on sustainability etc NDG Timeline http://proj.badc.rl.ac.uk/trac/roadmap

More Related