1 / 21

CSML – Stocktake and Forward Look

CSML – Stocktake and Forward Look. Andrew Woolf ( A.Woolf@rl.ac.uk ), Dominic Lowe CCLRC Rutherford Appleton Laboratory with contributions from NDG Team (Bryan Lawrence, Roy Lowry, Kerstin Kleese van Dam, Ray Cramer, Siva Kondapalli, Sue Latham, Kevin O’Neill, Ag Stephens). Outline.

amish
Download Presentation

CSML – Stocktake and Forward Look

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSML – Stocktake and Forward Look Andrew Woolf (A.Woolf@rl.ac.uk), Dominic Lowe CCLRC Rutherford Appleton Laboratory with contributions from NDG Team (Bryan Lawrence, Roy Lowry, Kerstin Kleese van Dam, Ray Cramer, Siva Kondapalli, Sue Latham, Kevin O’Neill, Ag Stephens)

  2. Outline • CSML: History • CSML: Present • CSML: Future

  3. CSML: History • NERC DataGrid: the integration problem • multiple organisations, formats, storage mechanisms (file, relational) • only commonality is data semantics

  4. <gml:definitionMember> <om:Phenomenon gml:id="taxon"> <gml:description>The taxon name</gml:description> <gml:name codeSpace="http://www.vliz.be">taxon</gml:name> </om:Phenomenon> </gml:definitionMember> </NDGPhenomenonDefinitions> <!--===================================================================--> <gml:FeatureCollection> <!-- ============================================================== --> <gml:featureMember> <NDGPointFeature gml:id="ICES_100"> <NDGPointDomain> <domainReference> <NDGPosition srsName="urn:EPSG:geographicCRS:4979" axisLabels="Lat Long" uomLabels="degree degree"> <location>55.25 6.5</location> </NDGPosition> </domainReference> </NDGPointDomain> <gml:rangeSet> <gml:DataBlock> <gml:rangeParameters> <gml:CompositeValue> <gml:valueComponents> <gml:measure uom="#tn"/> <gml:measure uom="#amount"/> <gml:measure uom="#gsm"/> </gml:valueComponents> </gml:CompositeValue> </gml:rangeParameters> <gml:tupleList> 'ANTHOZOA',63.1,missing 'Scoloplos armiger',66.1,missing 'Spio filicornis',10,missing 'Spiophanes bombyx',60.3,missing 'Capitellidae',131.8,missing 'Pholoe',10,missing 'Owenia fusiformis',23.4,missing 'Hypereteone lactea',6.8,missing 'Anaitides groenlandica',13.2,missing 'Anaitides mucosa',6.8,missing Feature types Application schema ISO 19109 ISO 19103 ISO 19118 ISO 19110 Universe of discourse CSML: History • The standards-based approach • “Conceptual modelling is the process of creating an abstract description of some portion of the real world and/or a set of related concepts.” (ISO 19101)

  5. CSML: History • Is this standards stuff relevant?! One example: • EU INSPIRE Directive • Establishes European Spatial Data Infrastructure – mandating interoperability of environmental data across EEA • “[ISO/OGC] standards and specifications ... will be considered as reference material for the Consolidation Team and relevant Drafting Teams.”(INSPIRE Work Programme Preparatory Phase 2005 – 2006) • Annex III Data Themes, incl.: • Atmospheric Conditions • Meteorological geographical features • Oceanographic geographical features • Sea regions

  6. CSML: History grep ‘ISO|OGC|WFS|WCS|WMS|GML’ {AGU,AMS,GO-ESSP}/* | wc ?

  7. <?xml version="1.0" encoding="UTF-8"?> <schema targetNamespace="http://ndg.nerc.ac.uk/csml" xmlns="http://www.w3.org/2001/XMLSchema" xmlns:csml="http://ndg.nerc.ac.uk/csml" xmlns:om="http://www.opengis.net/om" xmlns:gml="http://www.opengis.net/gml" elementFormDefault="qualified" attributeFormDefault="unqualified" version="0.1"> <annotation> <documentation>CSML application schema</documentation> </annotation> <!--====================================================================== --> <import namespace="http://www.opengis.net/gml" schemaLocation="GML-3.1.0/base/gml.xsd"/> <import namespace="http://www.opengis.net/om" schemaLocation="phenomenon.xsd"/> <!--====================================================================== --> <!--===== Root element for CSML dataset =====--> <!--====================================================================== --> <complexType name="DatasetType"> <complexContent> <extension base="gml:AbstractGMLType"> <sequence> <element ref="csml:UnitDefinitions" minOccurs="0" maxOccurs="unbounded"/> <element ref="csml:ReferenceSystemDefinitions" minOccurs="0" maxOccurs="unbounded"/> <element ref="csml:PhenomenonDefinitions" minOccurs="0"/> <element ref="csml:_ArrayDescriptor" minOccurs="0" maxOccurs="unbounded"/> <element ref="gml:FeatureCollection" minOccurs="0" maxOccurs="unbounded"/> </sequence> </extension> </complexContent> </complexType> <element name="Dataset" type="csml:DatasetType"/> <!--====================================================================== --> <!--===== Dictionary/definition elements =====--> <!--====================================================================== --> <complexType name="ReferenceSystemDefinitionsType"> <complexContent> <extension base="gml:DictionaryType"/> </complexContent> </complexType> <element name="ReferenceSystemDefinitions" type="csml:ReferenceSystemDefinitionsType"/> <complexType name="ReferenceSystemDefinitionsPropertyType"> <sequence> <element ref="csml:ReferenceSystemDefinitions" minOccurs="0"/> </sequence> <attributeGroup ref="gml:AssociationAttributeGroup"/> </complexType> British Oceanographic Data Centre British Atmospheric Data Centre ‘Governance Principle’ ISO standards Tooling Conceptual model Schemas CSML: History Climate Science Modelling Language

  8. CSML feature type Description Examples ProfileSeriesFeature TrajectoryFeature Discrete path in time and space of a platform or instrument. ship’s cruise track, aircraft’s flight path PointFeature Single point measurement. raingauge measurement ProfileFeature ProfileFeature Single ‘profile’ of some parameter along a directed line in space. wind sounding, XBT, CTD, radiosonde GridFeature Single time-snapshot of a gridded field. gridded analysis field PointSeriesFeature Series of single datum measurements. tidegauge, rainfall timeseries ProfileSeriesFeature Series of profile-type measurements. vertical or scanning radar, shipborne ADCP, thermistor chain timeseries GridSeriesFeature Timeseries of gridded parameter fields. numerical weather prediction model, ocean general circulation model GridFeature Pure geometry feature type ‘Coverage’ feature types CSML: Present Conceptual model

  9. CSML Dataset csml:UnitDefinitions csml:ReferenceSystemDefinitions csml:PhenomenonDefinitions csml:NetCDFExtract, csml:AggregatedArray, csml:GRIBExtract, csml:NASAAmesExtract, etc Storage descriptors gml:FeatureCollection CSML: Present • GML Application schema • all sections optional (cf. UDUNITS.xml) GML dictionaries (cf. CFStandardNames.xml)

  10. CDAT/cdunif + csmlscan CSMLFile.xml CSML: Present • Tooling • Scanner • uses ‘feature file map’ to characterise relationship between feature type and directory/files being scanned • individual Python classes for GML classes, feature types, dataset, etc  all have toXML() method for generating GML/CSML • instances populated by Scanner $ python csmlscan.py -d /badc/ecmwf-e40/data/gp/gp/ap-o era40gridseries.xml -c GridSeriesFeature -t 'time'

  11. CSML: Present tree = ElementTree(file=CSMLFile) dataset = Dataset() dataset.fromXML(tree.getroot()) ... feature = dataset.featureCollection.members[0] variablePhenomenon = feature.parameter units = feature.rangeSet.aggregatedArray.uom ... csml = dataset.toXML() • Tooling • Parser • all GML/CSML Python classes have fromXML() method • called recursively by Parser on ElementTree representation ElementTree + parser CSMLFile.xml

  12. CSML: Present • Tooling • API • experimental at present • defined against CSML data model • e.g.: Dataset.getFeatureList(), GridSeriesFeature.getDomain(), GridSeriesFeature.subsetToGridSeries() • Different feature types may support different operations (part of UML model!) • ‘Subsetting’ regarded as ‘extracting a feature’

  13. CSML: Present • Higher-level services • BADC DataExtractor • OGC Web Coverage Service • extension of GALEON server http://glue.badc.rl.ac.uk/cgi-bin/TPAC/WCS-CSML?SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&COVERAGE=solar_3&CRS=EPSG:4326&BBOX=0,-90,356,90&TIME=2790-01-01/2790-05-01&FORMAT=CF-netCDF&RESX=2&RESY=2

  14. CSML: PresentCSML (and friends): software stack

  15. CSML Present Other User Example: Norwegian Met Office

  16. CSML: Future • Immediate issues (v2) • Feature types: • Swath • ProfileSeries{Radar, Section, ProfileSeries}? • Lose ‘composite domain pattern’? • GML 3.2 ( ISO 19136) • remove ‘storage descriptors’ from core CSML schema • deployment, testing

  17. CSML: Future • Longer-term issues • GML irregular grids – CSML extends gml:Grid (evolving standards is slooow!) • ISO 19131 ‘Data Product Specifications’ (e.g. CRS should be defined in DPS, not AS) • GML & file-based data (will be discussed next week at OGC TC in Edinburgh)

  18. <gml:definitionMember> <om:Phenomenon gml:id="taxon"> <gml:description>The taxon name</gml:description> <gml:name codeSpace="http://www.vliz.be">taxon</gml:name> </om:Phenomenon> </gml:definitionMember> </NDGPhenomenonDefinitions> <!--===================================================================--> <gml:FeatureCollection> <!-- ============================================================== --> <gml:featureMember> <NDGPointFeature gml:id="ICES_100"> <NDGPointDomain> <domainReference> <NDGPosition srsName="urn:EPSG:geographicCRS:4979" axisLabels="Lat Long" uomLabels="degree degree"> <location>55.25 6.5</location> </NDGPosition> 'Anaitides mucosa',6.8,missing Feature types Application schema ISO 19103 ISO 19109 ISO 19118 ISO 19110 Universe of discourse CSML: Future Other activities: EU project ‘RISE’ RISE and MOTIIVE are both pilots for INSPIRE EU project ‘MOTIIVE’ MetOffice GALEON operational meteorology netCDF data standards

  19. CSML: Future • The CSML ‘niche’ • set of base feature types for specialising or using as-is

  20. CSML: Future • CSML ‘out-of-the-box’ • conceptual model • GML application schema • ‘geometric affordances’ (i.e. operations for generic Grid, Profile, Point coverages + series)

  21. Links • CSML: http://ndg.nerc.ac.uk/csml • CSML validator: http://glue.badc.rl.ac.uk/cgi-bin/csml/parseTest.py • MetOffice ‘Exeter Communiqué: http://www.metoffice.gov.uk/informatics • NDG CSML subversion: http://glue.badc.rl.ac.uk/viewsub/viewvc.cgi/ndg/TI02-CSML/

More Related