Multidimensional Data and GIS

1. Multidimensional Data and GIS Steve Kopp Nawajish Noman ESRI

2. Overview • Introduction to Multidimensional Data • What is netCDF? • Some netCDF data sources • Working with netCDF in ArcGIS

3. Multidimensional Data Data cube (3D) or hypercube (4D,5D…) • Temperature varying with time • Temperature varying with time and altitude T Y X

4. Gridded Data Raster Point Features

5. Multidimensional Data Time = 3 Time = 2 Time = 1

6. Time = 3 Time = 2 Time = 1 Multidimensional Data

7. Multidimensional Data Time = 3 Time = 2 Time = 1

8. Multidimensional Data Time = 1 Time = 2 Data Cube Time = 3 Time Slices

9. Multidimensional Data

10. Multidimensional Data Formats • NetCDF (network Common Data Form) • HDF (4.x and previous releases), HDF-EOS, HDF5 (Hierarchical Data Format) • GRIB, GRIB II (GRIdded Binary)

11. What is NetCDF? NetCDF (network Common Data Form) A platform independent format for representing multi-dimensional array-orientated scientific data. Self Describing - a netCDF file includes information about the data it contains. Direct Access - a small subset of a large dataset may be accessed efficiently, without first reading through all the preceding data. Sharable - one writer and multiple readers may simultaneously access the same netCDF file. NetCDF is new to the GIS community but widely used by scientific communities for around many years

12. Why NetCDF? Most commonly used format in the oceanographic and atmospheric science for observational data and numerical modeling • The National Center for Atmospheric Research (NCAR) • University Corporation for Atmospheric Research (UCAR) NOAA's Climate Diagnostics Center (CDC) • Los Alamos National Laboratory (LANL) • The National Center for Supercomputing Applications • US Air Force and Navy • Atmospheric Research in Australia • Australia Defense • UK Hydrographic Office • NATO • ...

13. What is a NetCDF file? NetCDF is a binary file A NetCDF file consists of: Global Attributes: Describe the contents of the file Dimensions: Define the structure of the data (e.g Time, Depth, Latitude, Longitude) Variables: Holds the data in arrays shaped by Dimensions Variable Attributes: Describes the contents of each variable CDL (network Common Data form Language) description takes the following form netCDF name { dimensions: ... variables: ... data: ... }

14. Storing Data in a netCDF File

15. Storing Data in a netCDF File Time = 1 Y = 1 X = 1 to 4

16. Storing Data in a netCDF File Time = 1 Y = 1 to 4 X = 1 to 4

17. Storing Data in a netCDF File Time = 1 to 3 Y = 1 to 4 X = 1 to 4

18. NetCDF Conventions • CF Conventions (Recommended, if applicable) • COARDS Conventions (1995 standard that CF Conventions extends and generalizes) • GDT Conventions (1999 standard that CF Conventions extends and generalizes) • CDC Conventions (for gridded data, compatible with but more restrictive than COARDS) • NCAR-RAF Conventions for Aircraft Data • AMBER Trajectory Conventions for molecular dynamics simulations • NUWG Conventions (1992-1995 effort to create some observational data conventions) • PMEL-EPIC Conventions • GDV Conventions (deprecated)

19. CF Convention Climate and Forecast (CF) Convention http://www.unidata.ucar.edu/software/netcdf/docs/conventions.html Initially developed for • Climate and forecast data • Atmosphere, surface and ocean model-generated data • Comparable observational datasets • The CF conventions generalize and extend the COARDS (Cooperative Ocean/Atmosphere Research Data Service) convention. • CF is now the most widely used conventions for geospatial netCDF data. It has the best coordinate system handling.

20. Current and Future Standards Fill in details… Galeon WCS OGC testbed CF v2

21. NetCDF Data Sources • Community Climate Systems Model (CCSM)  http://www.ccsm.ucar.edu, https://www.earthsystemgrid.org/ • The CCSM is fully-coupled, global climate model that provides state-of-the-art computer simulations of the Earth's past, present, and future climate states. • 100 yrs of climate change forecast data (2000-2099) • Control runs (1870-1999) and scenario runs • Temperature, precipitation flux, surface snow thickness, snowfall flux, cloud water content, etc. • Program for Climate Model Diagnosis and Intercomparison (PCMDI) http://www-pcmdi.llnl.gov/

22. NetCDF Data Sources • Vegetation and Ecosystem Modeling and Analysis Project (VEMAP) http://dataportal.ucar.edu/vemap/main.html • VEMAP was a large, collaborative, multi-agency program to simulate and understand ecosystem dynamics for the continental United States. • The VEMAP Data Portal is a central collection of files maintained and serviced by the NCAR Data Group • Climate data interval: Annual, monthly, and daily. • Data type: Historical and model results • Data: Temperature, irradiance, precipitation, humidity, incident solar radiation, vapor pressure, elevation, land area, vegetation, water holding capacity of soil, etc.

23. NetCDF Data Sources • British Atmospheric Data Center (BADC) http://badc.nerc.ac.uk/data/ • The role of the BADC is to assist UK atmospheric researchers to locate, access and interpret atmospheric data. • Many datasets are publicly available but datasets marked with key symbol have restricted access. • Datasets are organized by projects or organizations. • Climatology Interdisciplinary Data Collection (CIDC) has monthly means of over 70 Climate Parameters. • Met Office - Historical Central England Temperature Data has the monthly series, which begins in 1659, is the longest available instrumental record of temperature in the world. The daily series begins in 1772.

24. NetCDF Data Sources • National Oceanic & Atmospheric Administration (NOAA) • National Digital Forecast Database (NDFD) http://www.nws.noaa.gov/ndfd/ • Radar Integrated Display with Geospatial Element (RIDGE) http://www.srh.weather.gov/ridge/ • Precipitation Analysis http://www.srh.noaa.gov/rfcshare/precip_download.php • Climate Diagnostics Center http://www.cdc.noaa.gov/

25. NetCDF in ArcGIS • NetCDF data is accessed as • Raster • Feature • Table • Direct read (no scratch file) • Exports GIS data to netCDF

26. NetCDF Tools Toolbox: Multidimension Tools • Make NetCDF Raster Layer • Make NetCDF Feature Layer • Make NetCDF Table View • Raster to NetCDF • Feature to NetCDF • Table to NetCDF • Select by Dimension

27. Using NetCDF Data Behaves the same as any layer or table Display • Same display tools for raster and feature layers will work on netCDF raster and netCDF feature layers. • Graphing • Driven by the table just like any other chart. • Animation • Multidimensional data can be animated through a dimension (e.g. time, pressure, elevation) • Analysis Tools • A netCDF layer or table will work just like any other raster layer, feature layer, or table. (e.g. create buffers around netCDF points, reproject rasters, query tables, etc.)

28. NetCDF Layer/Table Properties Raster Feature Table

29. Making a NetCDF Raster Layer • Drag and drop • Use the tool Usage: MakeNetCDFRasterLayer <in_netCDF_file> <variable> <x_dimension> <y_dimension> <out_raster_layer> {band_dimension} {Dimension {Value};Dimension {Value}...} {BY_VALUE | BY_INDEX}

30. Time = 1 Changing Time Slice (Selecting a different “dimension value”)

31. NetCDF and Spatial Reference • Geographic Coordinate Systems (GCS) • X dimension units: degrees_east • Y dimension units: degrees_north • Projected Coordinate Systems (PCS) • X dimension standard_name: projection_x_coordinate • Y dimension standard_name: projection_y_coordinate • Variable has a grid_mapping attribute. • CF conventions currently supports only eight predefined coordinate systems • Undefined • If not GCS or PCS • ArcGIS writes (and recognizes) PE String as a variable attribute.