Find Satellite Imagery in a Database Quickly and Accurately

1. Search Accuracy Forming Query Database Search Numbers per Data Set Fields per Granule Lat/Lon Poor Fast Fast None 4 Orbit Predict Excellent Depends on size of date range Slow 6 per day 2 NOSE Good Very Slow Very Fast 100,000 typical 1 80W 64W 88E 106E BOSA Excellent Fast Fast 4 3 Orbit track Eighth orbit granule Coverage area as lat/lon extremes Why Use BOSA? With the dramatic increase in satellite-born sensor resolution, traditional methods of spatially searching for orbital data have become inadequate. As datavolumes increase end-users of the data have become increasingly intolerant of false positives. And, as computing power rapidly increases end-users expect correspondingly rapid search speeds. Meanwhile, data archives have an interest in delivering the minimum amount of data that meets users’ needs. This keeps their costs down and allows them to serve more users in a timelier manner. Many methods of spatial search for orbital data have been tried in the past and found wanting. The ever-popular lat/lon bounding box on a flat Earth is highly inaccurate. Spatial search based on nominal “orbits” is somewhat more accurate at much higher implementation cost and slower performance. Spatial search of orbital data based on predict orbit models are very accurate at a much higher maintenance cost and slower performance. This poster describes the Backtrack Orbit Search Algorithm—an alternative spatial search method for orbital data. Backtrack has a degree of accuracy that rivals predict methods while being faster, less costly to implement, and less costly to maintain than other methods. Does it Work? It's a heuristic so shortcuts are taken to speed up processing. For example, the orbit propagator uses a simplified model that assumes a circular orbit with constant declination and period (valid for almost all Earth observing satellites). Nevertheless we have found in practice that the algorithm is very accurate. It is also tunable. By varying the parameters that describe the sensor’s coverage we can be more or less strict about how much overlap constitutes a “hit.” We first developed the algorithm for the DMSP Digital Archive. BOSA drove an interface to search the inventory of OLS imagery at the National Geophysical Data Center (NGDC). Even though that interface never saw much use, it proved the concept worked. After further refinements the algorithm was also used in the Polar Spatial Query (PSQ) data server to search for AVHRR and SSM/I data at NSIDC. PSQ has been online and delivering data to users for over two years now. Currently BOSA is being modified to search the ECHO database. Users area of interest End Circular Lat = 49N Start Circular Lat = 39N It's orbital data, so we should do orbit searching. If we have an orbit model and the correct orbital parameters we can find out when the satellite was over the area of interest and search by time of acquisition. The problem is that orbit models are computationally intensive and orbital parameters require constant maintenance. Also, the search criterion is the union of many small time intervals. Database management systems are not efficient at such searches. An alternative scheme proposes “nominal” orbits. The Nominal Orbit Spatial Extent (NOSE) method attempts to predefine every possible orbit and give each an ID. System limitations require that those nominal orbits must be further broken down into smaller spherical polygons. A typical implementation might define 360 “orbits” comprising 36 polygons each. All this is pre-loaded into the database (360x36x8 = 103,680 numbers) and during a search the user’s area of interest is compared to each polygon. If a match is found data with the corresponding orbit ID is returned. This method has a large per sensor database footprint, a large per sensor cost associated with pre-defining the “orbits,” a large performance hit associated with comparing all those polygons, and can still generate up to 10% false positives, or more in some cases. It is a database solution to a mathematical problem. The Backtrack Orbit Search Algorithm fixes all these problems. It requires only a few additional fields be stored in the database and no changes to the database management software. The algorithm is easy to implement and maintain, runs efficiently for large databases and large numbers of queries, and returns accurate results. The Lat/Lon Bounding Box Problem: 85% False Positives! Ascending Passes The Algorithm in a Nutshell The basic idea is that, geometrically, all orbits look the same except for where the Earth happens to be as it spins under the satellite. This particular attribute of the orbit is called the “crossing longitude.” So, to search a database of scenes from orbiting satellites all you have to do is find the intervals of crossing longitudes for which it was possible for the satellite-born sensor to have seen the search area at some point in the orbit. Then, select only those scenes that are in the right place in the orbit—what we call “circular latitude.” Once the database is populated with the crossing longitude and start and end circular latitude for each scene the search is executed with a single Boolean statement. Of course it takes a bit of processing to generate that statement but it only needs to be done once for each search area. Start Circular Lat = 131 Stop Start End Circular Lat = 141 80W 64W Descending Passes Just What I Asked For Ideally, when searching a database for satellite scenes that cover a particular area of interest, a spatial query will return all scenes intersecting the search area and no scenes that do not intersect the search area. The problem then is how to do this efficiently and accurately. For most data sets a granule-by-granule comparison of the exact outline of each scene with the search area is impractical. An obvious shortcut is to describe scenes by their latitude, longitude bounding boxes. A search then consists of the conjunction of latitude and longitude intervals. This approach works fairly well for scenes and search areas near the equator but problems quickly arise as you move north orsouth. For example, a lot of environmental data these days comes from polar orbiting satellites. The problem when applying the lat/lon bounding box approach to polar orbiting satellite data is that even though a single orbit covers a small fraction of the Earth, its lat/lon bounds are the full -90 to 90 latitude and -180 to 180 longitude. This is because it orbits over both poles. Half orbits have the same problem and any smaller fraction of an orbit will always have longitude bounds wider than the scene. The problem is small at the equator but gets worse as the scene edge approaches the pole. At 60 degrees the box is twice as big as it should be. Given the huge volumes of today's satellite data, false positives are a big problem for both scientists and data providers. Stop Start Start Stop 80W 64W 88E 106E Search Intervals BOSA is part of the Spheres Java package and is available for download from: http://chukchi.colorado.edu/SPHERES Find Satellite Imagery in a Database Quickly and Accurately Backtrack Orbit Search Algorithm Ken Knowles and Ross Swick http://nsidc.org/data/tools/ SQL> select distinct file_ID, start_date, data_set, from inventory where data_set like "AVHRR_LAC" and ( ( ( (start_clat between 39.0 and 49.0) or (end_clat between 39.0 and 49.0) or (start_clat < 39.0 and end_clat > 49.0) ) and ( ascending_crossing between -80.0 and -64.0) ) or ( (start_clat between 131.0 and 141.0) or (end_clat between 131.0 and 141.0) or (start_clat < 131.0 and end_clat > 141.0) ) and (ascending_crossing between 88.0 and 106.0) ) ) )