220 likes | 356 Views
The NERC DataGrid. Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC etc) Representing the NDG and a dose of the “Portals Project”. +. +. =. +. +. NOCS, PML, STFC, BODC, BADC. Simulations. Assimilation.
E N D
The NERC DataGrid Bryan Lawrence Director of the STFC Centre for Environmental Data Archival (BADC, NEODC, IPCC-DDC etc) Representing the NDG and a dose of the “Portals Project” + + = + + NOCS, PML, STFC, BODC, BADC
Simulations Assimilation Complexity + Volume + Remote Access = Grid Challenge British Atmospheric Data Centre British Oceanographic Data Centre http://ndg.nerc.ac.uk
Ground based observation networks Met Office surface stations Model output NWP, ECMWF reanalyses & Climate models Satellite data TOMS, Envisat & MSG NERC programmes data UTLS, CWVC & URGENT Data Sets “A collection of files with a common theme and administration” Datasets statistics • 147 datasets • ~130TB (unique) • ~80 Million files
Find data Find out about data Subset and difference data Visualise data All using SECURE interdisciplinary technology. Sound easy doesn’t it? Want interdisciplinary semantic access to information, not abstract data getData(potential temperature from ERA-40 dataset in North Atlantic from 1990 to 2000) not: getData(“era40.nc”, ‘PTMP’, 20:50, 300:340, 190:200) or even worse: for j=1990:2000 getData(“era40_”+j+“.nc”, ‘PTMP’, 20:50, 300:340) The NDG Use Cases
The big: Climate in 2010 – A graphic Illustration Figures from Gary Strand, NCAR, ESG website
NDG Deployment Vocab Server: 120,000+ concepts 100 lists 75,000 Relationships (SKOS)
MOLES: Metadata Objects for Linking Environmental Science A data production tool deployed at an observation station on behalf of an activity produces a dataset. Datasets have provenance
A-Metdata We choose to deploy an “Application Schema” of the “Geographic Markup Language”. This allows us to define our data to support RE-USE by other communities – interdisciplinarity, and to be INSPIRE compliant! Getting down to the data itself
CSML Dataset • Dictionaries: • coordinate reference systems, • phenomena (parameters) • units of measure • Feature Collections • CSML Features are SPECIALISATIONS of GML Features! • Storage Descriptor • Re-usable description of storage (NB: CSML Software Stack, WMS, WCS etc)
CSML Feature Types Abstract Specific: point data, time-series of point data, trajectory data, point collections, profiles, profile series, Ragged profile series, section, ragged section, scanning radar, grid, gridseries and swath
Beyond measuring things at points CF concepts: Cell-Methods (statistical e.g means etc) Cell-Bounds (intensive quantities, Extensive quantities, Relationship to coverage domain) New work: Exploiting Sensor Web Enablement
April 2007, at The Cosenors House, Abingdon Brought NDG, Unidata, OGC, Italian NRC together to get a shared vision for scientific data types. Roadmap for bringing our activities together over next few years. Harmonising: NDG CSML OGC Observations and Measurements (with CSML and MOLES) UNIDATA Community Data Model INRC NCML-G Data Modelling Workshop
Data Model Relationships Route to harmonising GeoSciML and CSML
What we’ve done (are doing) Schema: MOLES CSML DIF + Lots of Population Issues Pylons Interface & portals Protocols: OGC +OAI Pylons based portal Python WCS/WMS Clients CSML Toolbox Vocab Server
Now and the Future More protocol improvements (= wider community of providers and users) More deployment. More population. Evolution WITH the web.