1 / 13

Tony Rees and Glenelg Smith Divisional Data Centre + Remote Sensing Facility

Application of c-squares spatial indexing to an archive of remotely sensed data. Tony Rees and Glenelg Smith Divisional Data Centre + Remote Sensing Facility CSIRO Marine Research, Australia (Tony.Rees@csiro.au). The starting point ….

Download Presentation

Tony Rees and Glenelg Smith Divisional Data Centre + Remote Sensing Facility

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Application of c-squares spatial indexing to an archive of remotely sensed data Tony Rees and Glenelg Smith Divisional Data Centre + Remote Sensing Facility CSIRO Marine Research, Australia (Tony.Rees@csiro.au)

  2. The starting point … • Archive of c.60,000 satellite AVHRR scenes, 1986-current, accumulating at another 15+ per day • Region covered varies for each scene • Metadata (satellite, orbit, time parameters) and computed polygon (e.g. 7 points per side) available for each scene • Determining which scenes include any given point, on-the-fly (to answer user’s request) is a standard requirement, but not trivial – available methods may be slow, or inaccurate, or both...

  3. example in more detail ... x • Example – may need to query which out of these four scenes includes point “x” (in reality, stop/start points of scenes also vary, also other scenes cross at different angle/s)

  4. Options for spatial indexing Bounding polygons • Require a GIS system (or similar) for spatial queries (polygon overlays); can be slow to test many thousands of scenes • Can be problems with footprints which cross the date line, or include a pole Bounding rectangles • Computationally cheap, rapid to query, no GIS required • Typically a poor fit to actual data footprints (typically these are not square, or even nearly so) – many “false positives” • Can be problems with footprints which cross the date line, or include a pole Gridded (tile based) representation • Potentially simple to query, no GIS required • May be poor fit to actual data footprints if tile size is large (smaller tiles best, however index gets larger – tradeoff required here) • Compromise approach – slightly inexact compared with polygon overlays, but should be better performance for real-time queries (most computation has been done in advance) • Problems with footprints which cross the date line, or include a pole, can be pre-handled

  5. C-squares basics • Based on a tiled (gridded) representation of the earth’s surface, at choice of 10 x 10, 5 x 5, 1 x 1, 0.5 x 0.5 degree tiles, etc. (0.5 x 0.5 degree tiles are used in this example) • Every tile has a unique, hierarchical alphanumeric ID (incorporates the ID of all parent, grandparent, etc. in every child tile ID) • Dataset (=scene) extents are represented by a list of all the tiles included in, or intersected by the footprint • Spatial search comprises looking for one or more tile IDs in the set associated with any dataset (= simple text search). (more details – seewww.marine.csiro.au/csquares/ )

  6. C-squares nomenclature – ten degree squares (follows WMO 10 x 10º square nomenclature)

  7. 10 degree square: 3213:*** 5 degree square: 3214:1** 1 degree square: 3315:100 Adaptive square size – 10 x 10, 5 x 5, 1 x 1, 0.5 x 0.5 degrees (etc.) – produces quadtree-like efficiencies (“compaction”) (this example shows detail to 1-degree square level) - can reduce storage requirement by up to 90%, cf. constant-resolution encoding

  8. 10 x 10 degree squares (28) (base level of hierarchy, cannot compact) 5 x 5 degree squares (99) = 36 after “compaction” 1 x 1 degree squares (1,982) = 515 after comp. 0.5 x 0.5 degree squares (7,691) = 704 after comp. 0.5 x 0.5 degree squares - detail A real-world exampleNOAA-12 .. 21 Jun 2003 06:23

  9. Scene <> c-square pairs (600-1200 rows per scene) Metadata table concept – 2 “search” tables only Scene details (1 row per scene)

  10. Clickable map interface

  11. Example search result

  12. SQL for spatial search(example for 0.5 degree search square) select distinct A.scene_id, B.satellite, B.scene_date_time, B.image_location from satdata_csq_all A, scene_info B where ( (sqrsize = 0.5 and (A.csquare = search_csq -- e.g. 3414:100:1 (0.5 degree square) or A.csquare = substr(search_csq,1,8)|| ':*' -- 1 level of compression or A.csquare = substr(search_csq,1,6)|| '**:*' -- 2 levels of compression or A.csquare = substr(search_csq,1,4)||':***:*') -- 3 levels of compression ) -- (other supported search square size options inserted here) ) and (startdate is null or B.scene_date_time >= startdate) and (enddate is null or B.scene_date_time <= enddate) and (sat = 'any' or B.satellite = sat) and A.scene_id = B.scene_id order by B.scene_date_time, B.satellite;

  13. Summary of steps involved • (1) Encoding – using custom polygon-to-c-squares algorithm (first attempt: 3+ mins per scene; second attempt: 30 seconds per scene to encode) • (2) Storage – first attempt: all scene/c-square pairs in one large table, second attempt: partitioned by 10 x 10º “bins” (100 smaller tables, for faster searching) • (3) Querying – using standard SQL and text matches • (4) (optional) Visualising – generating footprints, using web call to multi-purpose c-squares mapper More information: c-squares website - http://www.marine.csiro.au/csquares/ CMR Satellite Data spatial search: http://www.marine.csiro.au/remotesensing/csq-chooser.htm

More Related