The IRI Climate Data Library: translating between data cultures

1. Benno Blumenthal International Research Institute for Climate Prediction Columbia University The IRI Climate Data Library: translating between data cultures

2. Work presented has been done by • John del Corral • Michael Bell • Emily Grover-Kopec

3. Outline • Data Cultures • Needed Functionality for usefully merging • Data model changes • Functionality changes • Example • Mapping SimpleFeatures to OpenDAP • Lessons Learned

4. Economics Public Health “geolocated by entity” Ocean/Atm “geolocated by lat/lon” multidimensional GIS “geolocation by vector object or projection metadata” spectral harmonics equal-area grids GRIB grid codes climate divisions IRI Data Collection Dataset • Dataset • Dataset • Variable • ivar • ivar multidimensional

5. GRIB netCDF images binary spreadsheets shapefiles Database Tables queries Servers OpenDAP THREDDS images w/proj IRI Data Collection Dataset • Dataset • Dataset • Variable • ivar • ivar

6. GRIB netCDF images binary spreadsheets shapefiles Database Tables queries Servers OpenDAP THREDDS images w/proj IRI Data Collection Dataset • Dataset • Dataset • Variable • ivar • ivar User Interface viewer manipulations calculations Calculations “virtual variables” Clients OpenDAP THREDDS Data Files netcdf binary images Tables OpenGIS WMS v1.3

7. Functionality for new Users • want to use our current data holdings • aggregation (particularly of images) • time analysis of GIS data • translation to “entity” basis

8. Data Model Changes • projection attributes • SpatialReferenceSystemWKT (An OpenGIS standard) • SpatialReferenceSystemDims • geometry data object • OpenGIS Simple Feature

9. Functionality Changes • geometry display: fill, fillby, stroke, mark, ... • projection functionality: objects with differing projection attributes can be made to match • rasterization: geometry object converted to raster • computes the fraction of each gridbox that is covered by the given geometry object

10. Example • NDVI[x y time] Albers projection and location [d_name] Lon/lat geometry produces NDVIg[d_name time]

11. weighted-average

12. Simple Features and OpenDAP v2.0 structure

13. Simple Features and OpenDAP v2.0 attributes • SpatialReferenceSystemWKT Projection information • OpenGISSimpleFeature Point, LineString, Polygon, MultiPoint, MultiLineString,MultiPolygon,GeometryCollection Or • Dimensionality • 0 (point) 1 (line) 2 (polygon)

14. Sample Attributes NDVI [ x y time] SpatialReferenceSystemWKT PROJCS["Albers_Equal_Area_Conic",GEOGCS["GCS_North_American_1927",DATUM["D_North_American_1927",SPHEROID["Clarke_1866",6378206.4,294.9786982]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Albers"],PARAMETER["False_Easting",0],PARAMETER["False_Northing",0],PARAMETER["Central_Meridian",20],PARAMETER["Standard_Parallel_1",21],PARAMETER["Standard_Parallel_2",-19],PARAMETER["Latitude_Of_Origin",1],UNIT["Meter",1]] SpatialReferenceSystemDims x y

15. Lessons Learned • nice to have a location object • Place to put attributes For example, if I have an image ndvi[x y time] with an Albers projection, clearly the projection is an attribute of x y. Seeing as x y are contained in ndvi, putting the SpatialReferenceSystemWKT attribute almost makes sense, but then I have to add the other attribute SpatialReferenceSystemDims to clarify. But now I have attributes in a container which sometimes apply up, and sometimes down.

16. Lessons Learned • Our structural representations (OpenDAP, netCDF) are missing recursion • Must be able to specify different standards for different parts of a dataset – country codes are an iso standard, for example, but are unlikely to ever be part of CF or WCS • Must be able to transmit everything, not just anointed datasets • Bringing together different concepts in different data cultures (multidimensionality, location as object) improves both sides of the divide.