1 / 10

GHRSST Aggregations using NcML

GHRSST Aggregations using NcML. Upendra Dadi. GHRSST Overview. Aggregation Process. http://data.nodc.noaa.gov/opendap/. http://dods.jpl.nasa.gov/opendap/. ghrsst/. ghrsst/. L4/. L4/. /L2P_Gridded. /L2P_Gridded. L2/. L2/. ghrsst_combined.xml. L4/. /L2P_Gridded. L2/.

faunus
Download Presentation

GHRSST Aggregations using NcML

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GHRSST Aggregations using NcML Upendra Dadi

  2. GHRSST Overview

  3. Aggregation Process http://data.nodc.noaa.gov/opendap/ http://dods.jpl.nasa.gov/opendap/ ghrsst/ ghrsst/ L4/ L4/ /L2P_Gridded /L2P_Gridded L2/ L2/ ghrsst_combined.xml L4/ /L2P_Gridded L2/ (L3 will be addedin GDS v2) Time

  4. Time granularity of originator data is not necessarily same as the granularity required for a data analysis task. NcML aggregations could help here. Ideal for climate related studies. Lessons Learned (not to any scale) hourly daily weekly monthly seasonal annual decadal centurial mellinial

  5. Anyone with access to web could create the aggregations, one doesn't have to be inside NODC. Aggregations created by one user could be used by others. Having a shared repository of NcML files could be useful.

  6. Performance is the biggest short coming. Large amount of time spent on decompressing the data. NetCDF-4 could help. Tools like nccopy are useful to the end user. Having tools to update the local physical version of the dataset when the NcML changes would be useful. Running time for retrieving time series at a point for a two month period for an L4 product is 90 sec Repeating the same query for another point but for the same time period took 2 sec

  7. Issues with caching. It would be useful to have elements in the NcML to update the individual NcMLs in the cache periodically instead of entire cache.

  8. Several interesting possibilities. Allows integration of data from heterogeneous sources over web to create virtual datasets. Datasets from different disciplines could be integrated. Ability to represent vector data using netCDF would make such integration more attractive to mainstream GIS users.

  9. NODC has lot of in-situ(observational) data. Ability to aggregate not just 2d arrays but also individual profiles & trajectories into multi-profiles and multi-trajectories would be very useful. time time

  10. Similar to ETL tools used in Data Warehousing. Equivalents in Relational World, but the data is more complex than most relational databases can handle.

More Related