The OPeNDAP/OGC Gateway

1. The OPeNDAP/OGC Gateway A NASA ACCESS Project integrated from two proposals: -- The Development and Deployment of a CEOP Satellite Data Server (Ken McDonald, GSFC) -- Gateway for Interoperability of Atmosphere, Land, Ocean, and Modeling Science Data (Liping Di, GMU) Wenli Yang Center for Spatial Information Science and Systems (CSISS) School of Science George Mason University http://csiss.gmu.edu OPeNDAP Developer’s Meeting, Feb. 21-23, 2007, Boulder, Colorado

2. Project Team PI and Co-Is: Ken McDonald (PI, NASA/GSFC), Yonsook Enloe (SGT Inc.), Liping Di and Wenli Yang (GMU), Dan Holloway (OPeNDAP), Ben Domenico (Unidata), Glenn Rutledge (NOAA/NCDC). Other members: Chengfang Hu and Min Min (GMU), Sr. S/W Engr.(OPeNDAP) Science advisors and user feedback: Professor Toshio Koike (University of Tokyo), Dr. Mike Bosilovich (NASA GSFC)

3. Project Goal To address the interoperability of two data system infrastructures widely used by different segments of the Earth science research and applications community, namely the Earth science community which uses OPeNDAP and THREDDS protocols and the geospatial community which uses OGC protocols.

4. Specific Objectives • To allow a user of a DAP client to discover and access data provided through OGC servers. • To leverage WCS rectification/reprojection and interpolation operations with DAP access to satellite products for the CEOP science community. • To allow a user of a OGC client to discover data available through THREDDS servers.

5. CEOP Satellite Data Server OPeNDAP/WCS Gateway Components

6. CEOP Satellite Data Server OPeNDAP/WCS Gateway Components • The original design was to develop the gateway components. The gateway can then be installed with the OPeNDAP server to link the server to a WCS server. • With the development of server4, many of the components are already included in the server. Thus, an independent gateway is not needed. • The CF-1.0 compliant netCDF format handler is embedded into the WCS server. Server4 can always expect a valid CF-netCDF from the WCS server. • THREDDS catalog generator will be developed as a THREDDS server at the front end and as a WCS 1.1 client, which sends describeCoverage requests, at the back end. • Currently, the catalog generator is implemented by making use of an XML configuration file (the WCS server’s capabilities XML file) without issuing requests to the WCS server.

7. OPeNDAP Server Implementation Approach WCS DAP OLFS BES Local cache THREDDS • OLFS interacts with local catalog to identify data source as WCS. • OLFS instructs BES to set container type WCS; passes name, target, type to BES. • BES sets container to WCS, uses name, target, type to interact with remote WCS. • BES interns WCS response to local cache. • BES uses handler (NetCDF, HDF, <type>) to process cached file to satisfy DAP request. • Subsequent DAP requests operate against local cache until cache refresh signaled.

8. The Test Implementation http://test.opendap.org:8080/opendap/data/wcs/Georectified-Grid/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.MODIS_Grid_8Day_1km_LST.nc.html Corresponding WCS call http://data.laits.gmu.edu/cgi-bin/ACCESS/wcs300?service=WCS&version=1.0.0&request=getCoverage&coverage= /home/mmin/grid1/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.hdf:Grid:MODIS_Grid_8Day_1km_LST:LST_Day_1km& crs=EPSG:4326&bbox=-100.8,38,-92.1,39.9&format=netCDF&width=300&height=200

9. The Test Implementation http://test.opendap.org:8080/opendap/data/wcs/Georectified-Grid/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.MODIS_Grid_8Day_1km_LST.nc.html

10. DAS response http://test.opendap.org:8080/opendap/data/wcs/contents.html

11. DDS response

12. WCS Coverage f(x,y,z,t) = {T, RH, P,…} Domain => Range

13. WCS Domain

14. WCS Range

15. DescribeCoverage Request

16. DescribeCoverage Response

17. GetCoverage Request Example

18. GetCoverage Request Example

19. GetCoverage Response Example <?xml version="1.0" encoding="UTF-8"?> <OperationResponse xmlns="http://www.opengis.net/ows/1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 ../owsInputOutputData.xsd"> <ReferenceGroup> <Abstract>Coverage created from GetCoverage operation request to a WCS</Abstract> <Reference xlink:href="coverage/image.tiff" xlink:role="urn:ogc:def:role:WCS:1.1:coverage"/> <Reference xlink:href="coverage/metadata.xml" xlink:role="urn:ogc:def:role:WCS:1.1:metadata"/> </ReferenceGroup> </OperationResponse>

20. y Last grid point (x0,y0) v (x2,y2) b (u2,v2) (a2,b2) point Pin P1=(deltU,0) P2=(0,deltV) (x1,y1) (u1,v1) (a1,b1) a point Pout p2 First grid point x p1 (u0,v0) Figure 1 boundingBox specified in CRS (a,b). u Figure 2 Source coverage and transformed boundingBox in source coverage’s baseCRS (x,y). Blue area shows boundingBox being extended to the nearest grid posts. Figure 3 Output coverage and transformed boundingBox and extended boundingBox in output coverage’s baseCRS (u,v). Blue area shows the minimum subset in source coverage. The following steps describe one of (many?) approaches, called approach two, of a getCoverage request/response for a 2D grid, assuming that the boundingBox CRS, source grid baseCRS, and output grid baseCRS are different: • A client specifies a boundingBox in CRS (a,b) and specifies the output grid’s origin (u0,v0) and offset p1 and p2 in the baseCRS of the output grid (u,v) (this baseCRS is specified in wcs:GridCRS). • The boundingBox is transformed to the wcs:GridCRS of the source gird (green area in fig. 2) and the extent is extended to a nearest minimum area (blue area in fig.2) that covers the boundingBox. The grid point values of the are subsetted. • The boundingBox is transformed to the wcs:GridCRS (u,v) of the output gird, whose grid point locations are determined by the origin (u0,v0) and offset values p1 and p2 defined in the output part of the getCoverage request. The transformed boundingBox (green area in fig. 3) is extended to a nearest grid points that completely include the transformed boundingBox. This output grid is shown in the grid points constructed by the black lines. Note that this output grid may not necessary cover all areas of the minimum subset in the source grid (the blue area in figure 2) as shown in figure 3, dependent on such factors as baseCRSs and offset values of source and output grid. • The minimum subset from the source grid (blue area in figure 2) is transformed to the output grid points. • In this method, some values in the minimum subset may not be used (e.g., point Pin in figure 3) while some output grid points may not be available (e.g., point Pout in figure 3). Such issues can be avoid in approach one discussed in the previous chart. • In the output grid, the positions are defined at grid points, not grid cells. If the grid is interpreted as composed of grid cells, the grid cells look like something as shown by the grid constructed by the dashed dark red lines. The position of the center of each such cell is defined (or, cell position is defined at the cell center).

21. Last grid point (x0,y0) y v b (a2,b2) P1=(deltU,0) P2=(0,deltV) (a1,b1) a p2 First grid point p1 (u0,v0) x Figure 1 boundingBox specified in CRS (a,b). u Figure 3 sourceCoverage, its baseCRS (x,y) and origin (x0,y0). The The green and blue areas are correspondent to the boundingBox and output grid. Figure 2 Output coverage extent (blue) and transformed boundingBox (green) in output coverage’s baseCRS (u,v). The output grid origin is at (u0,v0). The following steps describe one of (many?) approaches, called approach one, of a getCoverage request/response for a 2D grid, assuming that the boundingBox CRS, source grid baseCRS, and output grid baseCRS are different: • A client specifies a boundingBox in CRS (a,b) and specifies the output grid’s origin (u0,v0) and offset p1 and p2 in the baseCRS of the output grid (u,v) (this baseCRS is specified in wcs:GridCRS). • The boundingBox is transformed to the wcs:GridCRS of the output gird and the extent of the output grid is determined based on the transformed boundingBox and the origin (u0,v0) and offsets p1 and p2, by extending the transformed boundingBox to the closest grid points to completely include the boundingBox. The origin (u0,v0) and offsets p1 and p2 are defined in the output coverage’s basedCRS, which is specified/included in the wcs:GridCRS. The transformed boundingBox is shown in green and the output grid extent is shown in the blue area (grid points constructed by the black lines) in figure 2. • Values for the grid points in the output grid are derived by determining their positions in the source grid. In the source grid, the green and blue areas show the areas of the boundingBox and the output grid if they would be transformed to the baseCRS of the source coverage. These areas, however, usually need not to be transformed to the source coverage’s baseCRS. A server may chose to transform the blue area so that only a minimum subset of the source grid needs to be read (note that the extent of the minimum subset in the source grid is also dependent on interpolation method.). • In the output grid, the positions are defined at grid points, not grid cells. If the grid is interpreted as composed of grid cells, the grid cells look like something as shown by the grid constructed by the dashed dark red lines. The position of the center of each such cell is defined (or, cell position is defined at the cell center).

22. MODIS Data in Swath and Lat/Lon Coordinates

23. ASTERData in Swath and Lat/Lon Coordinates

24. OGC Geoscience Gateway

25. WCS-geoscience Gateway Prototype in THREDDS

26. OGC CSW The OGC Catalog Service for Web specifies the interfaces, bindings, and a framework for defining application profiles required to publish and access digital catalogues for geospatial data and services.

27. OGC Catalog UML Model

28. Common Queryable Elements

29. OGC CSW Application Profiles The ISO19115/19119 profile explains how catalogue services based on the profile are organized and implemented for the discovery and management of geospatial data and service metadata which are compliant with the ISO19115 and 19119 standards. The ebRIM profile explains how services based on the more general OASIS ebXML Registry Information Model are organized and implemented.

30. Connecting THREDDS to CSW • The first of the following two approaches are adopted: • Mapping THREDDS metadata to ISO metadata and implementing a CSW server based on the ISO profile. • Implementing a CSW server for THREDDS metadata based on ebRIM profile.

31. ISO19115 Metadata Information Identification Information

32. THREDDS Catalog Information Model

33. THREDDS/CSW Mapping The first step is to mapping the semantically equivalent metadata items between the ISO19115 and the THREDDS information models.

34. THREDDS Catalog Ingestor The ingestor is a THREDDS catalog server client at the front end. It obtains information of data sets in the THREDDS catalog maps the information to ISO metadata. At the back end, it writes to the CSW database through JDBC. This will require that the ingestor have write permission to the CSW database. It is also planned, if resources are available, to implement the ingestor as a CSW client at the back end. The client can register the THREDDS metadata to any compliant CSW server through CSW protocol.

35. THREDDS Dataset Inventory Catalog

36. THREDDS Catalog Ingestor Tool Design

37. GMU CSW Search Interface http://geobrain.laits.gmu.edu/csw/discovery/

38. GetRecord Request