Smugglers’ Path Planning on Compressed Terrain UniGIS, May 2006

1. Smugglers’ Path Planning on Compressed TerrainUniGIS, May 2006 W Randolph Franklin, Rensselaer Polytechnic Institute mail@wrfranklin.org, 703-447-7808 http://wrfranklin.org 5/2/06 9pm

2. Goals of this project • Alternate terrain representations • Compact; lossy - size / quality tradeoffs. • Bias representations towards legal terrain • Process datasets up to 10000x10000 • Uncompression speed is more important than compression speed. • Evaluate on visibility, mobility metrics. UniGIS, May 2006

3. Team • Prof Randolph Franklin – helping everyone • Prof Caroline Westort – strategizing about scooping • Prof Frank Luk – numerical analysis advice • Prof Barbara Cutler – computer graphics advice • Metin Inanc – scooping etc • Zhongyi Xie – ODETLAP • Dan Tracy – SVD approximations, multiobserver siting • Duane Niemeyer and crew, ESRI – implementing siting toolkit UniGIS, May 2006

4. First: Study Terrain Properties • Many local max, few local min • Long range order - rivers • Elevation and slope discontinuities are common, and are very important for mobility and visibility UniGIS, May 2006

5. Which is land, which water? You can answer this => there is unexploited structure. UniGIS, May 2006

6. Where is This? UniGIS, May 2006

7. Answer UniGIS, May 2006

8. Examples of Other Errors UniGIS, May 2006

9. Fourier Series • Widely used • Excellent for representing many physical phenomena, like vibrations. • Quite unsuitable for terrain. • They assume C∞ continuity • The truncated series is too smooth • …and has many local minima UniGIS, May 2006

10. Fractals ?! • I cannot understand why anyone thinks that they look realistic. • Enough degrees of freedom can make (almost) any representation work. • Ptolemaic epicycles were as accurate as Keplerian ellipses. • However... what's natural? UniGIS, May 2006

11. Sample Results • TIN: represented a 10800x10800 array to 3% max elevation error with 157,735 triangles. • Scooping: represented w111n31 with 7x7 linear scoops with average error 0.1% and max error 2%. • Using 7x7 scoops on one 3592x3592 dataset, multiobserver siting had only 6.5% error. • ODETLAPping 400x400 piece of Lake Champlain W with 1/9 the points: error was 0.9m (0.1%). • Combining TIN with ODETLAP: captures essence of surface with very few points. • ODETLAP: Can fill radius 40 circles of missing data. UniGIS, May 2006

12. Test Data Used • USGS 1° DEM: Lake Champlain W, a nice mix of mountains, lowlands, and lake. • DTED level 2 data from March 2005 kickoff meeting. • SRTM DTED level 2 data from Jan 2006 CDs. • We’re standardizing on these nicely mountainous cells: • w119n36.dt2, w120n38.dt2, w121n45.dt2, w112n41.dt2, w112n40.dt2 • Per NGA request in Feb, we added some less mountainous cells: • w112n32.dt2, w113n32.dt2, w113n33.dt2, w113n40.dt2, w113n41.dt2, w114n32.dt2, w114n33.dt2, w115n33.dt2, w119n33.dt2 UniGIS, May 2006

13. w111n31 w111n32 w111n33 w112n32 w112n40 w112n41 w119n36 w120n38 w124n38 w121n45 UniGIS, May 2006

14. Test Data Complexities Varying Resolution Bunched Elevations UniGIS, May 2006

15. Testing Protocol • Compute some property on the original terrain representation (large matrix of elevation posts), and again on the alternative representation. • Measure the difference. • Sample various test datasets. • Tradeoff size vs quality. UniGIS, May 2006

16. Protocol 1: Elevation Testing • Measure both average absolute error and RMS error • Some representations do better at one than other. • Examine terrain to see if features captured. • Important but hard to quantify. No toy datasets UniGIS, May 2006

17. Protocols 2-4: Visibility Testing Various levels of complexity: • Evaluate differences in observer viewsheds • Evaluate visibility index of every observer. • Evaluate multiobserver siting quality. UniGIS, May 2006

18. Protocol 2: Viewshed Testing • Select parameters: • Radius of interest (R) • Observer, target heights (H) • Compute viewshed bitmaps of many observers on both the original and the alternative representations. (we can do this fast for large R). • For some observer O, let Vo be its viewshed on the original, and Va be its viewshed computed on the alternative representation. Vo and Va are bitmaps (i.e., sets of target points visible by O). Report |Vo-Va| + |Va-Vo|. UniGIS, May 2006

19. Interpolating LOS between posts • Possible research • Motivation: in one test, we tried various interpolation methods (min, max, linear) • ½ of all the targets changed visibility UniGIS, May 2006

20. The Known Unknowns of Viewsheds • Small changes in LOS interpolation cause large changes in visibility. • One half of this cell has uncertain visibility. UniGIS, May 2006

21. Protocol 3: Visibility Index Testing • Consider each post in term as an observer. • Compute its visibility index. • Monte Carlo sampling: pick T random targets, compute their visibility, and report the fraction visible. • Produce a map of all the visibility indexes. • Compare the visibility index map of the original terrain representation to the map of the alternative representation. UniGIS, May 2006

22. Protocol 4: Multiple Observer Siting Testing • Site a set of observers, So, on the original terrain rep. • Site a set of observers, Sa, on the alternative terrain rep. • Transfer Sa to the original rep. • Compare quality of Sa to So. UniGIS, May 2006

23. Multiobserver Siting Status • Software written by WRF, and extended Christian Vogt as a masters thesis. • It works, but is complicated and messy. • ESRI is productizing it with ArcGIS. • Dan Tracy is using it internally to evaluate our compression. UniGIS, May 2006

24. Multiobserver Siting Steps • Find approximate visibility index of every point in cell, using Monte Carlo sampling. • Partition the cell into blocks and pick the best potential observers in each block. • Using a greedy algorithm, select the best of the best observers. • We have considerably studied the tradeoffs here. UniGIS, May 2006

25. Enforcing Intervisibility • After the first best-of-the-best, add only new observers that are inside the joint viewshed of the previous best-of-the-best. No intervis  Intervis  UniGIS, May 2006

26. Effect of Intervisibility • This reduces the joint viewshed considerably. UniGIS, May 2006

27. Reduced Resolution Effect on Siting • Lowering horizontal resolution lowers observer siting quality. • Lowering vertical resolution does not as much. • Visibility, computed on lower resolution, is too high. UniGIS, May 2006

28. Experiments Reducing horizontal resolution Reducing vertical resolution UniGIS, May 2006

29. With or w/o intervisibility Intervisibility enforced No intervisibility required Color -> elevation; Black -> hidden. UniGIS, May 2006

30. Alternative Representations • TIN • Scooping • ODETLAP • Combinations of the above, e.g., ODETLAP uses TIN points. UniGIS, May 2006

31. Note: Good compression techniques are multistep JPEG: • Rotate RGB -> YCrCb • Discrete cosine transform • Low-pass filter • Arithmetic encode Text compression: • Run length encoding • Burrows-Wheeler transformation • Move to front • Another run length encoding • Arithmetic encode UniGIS, May 2006

32. TIN Status • We can process 10800x10800 arrays of posts in ½ hr on PC • No external storage is used. • Dataset formed by catenating nine 3601x3601 cells from data from the Savannah March kickoff meeting. • Elevation range: 3600. UniGIS, May 2006

33. TIN Features • Progressive resolution since points are inserted greedily. • “Feature” points on peaks and ridgelines, and edges joining them, may be more important. • Our TIN program selects them automatically; no need for manual identification and constrained triangulation. • The points selected for the triangulation are assumed to be important, and can be fed into other methods, like ODETLAP. • TIN is a piecewise linear triangular spline. Preliminary experiments with a higher degree spline showed no consistent improvement, and so were suspended. UniGIS, May 2006

34. Lossless TIN • TIN is inherently lossy. • Make it lossless: • Use DETLAP to fit a surface to the TIN points. • Compute the errors. • Compress (x,y), z, errors each with PAQ3N. • Test on w111n3111, 400x400. UniGIS, May 2006

35. Coding the TIN Representation • Q: How to code the {(x,y,z)} TIN points? • Note: order of the set elements is immaterial. • A1: Use bitmap coding techniques for {(x,y)}. • There is an info-theoretic limit here. • Order of z is now determined. • A2: Compress z with a sequence of Burrow Wheeler, Run Length and Arithmetic Coding. UniGIS, May 2006

36. Alternative Representations • TIN • Scooping • ODETLAP • Combinations of the above, e.g., ODETLAP uses TIN points. UniGIS, May 2006

37. Scooping Representations • This is longterm research. • The goal is to smash through the information theoretic barrier to terrain compression by utilizing geologic information. • We are pursuing several representations in parallel. UniGIS, May 2006

38. Scooping Status Several subprojects: • 3-axis milling machine experiments with set of simple drills. • Complete cover test with parameterized sloped drills. • Theoretical thinking about how scooping is different from, e.g, wavelets. UniGIS, May 2006

39. Dynamic Strategy • Hypothesize. • Experiment. • Observe. • Modify hypothesis. • Repeat. UniGIS, May 2006

40. Drilling Test on Lk Champlain W - Ideas • Purpose: to learn what types of drills are more useful. • Matlab program. • Greedy optimization: at each step, select the drill instance: (drill type, x, y, z) that removes the most material. • Drill instances will generally overlap. • If two drills drill the same post, then the min elevation is used. • This nonlinearity is radically different from wavelets. UniGIS, May 2006

41. Subtleties of the Greedy Optimization • The drill that removes the most material at the start may be rendered completely redundant by later drills. • Reminiscent of stepwise multiple linear regression – an initially important variable may ultimately be unimportant. • Future possibility: removing as well as adding drills. UniGIS, May 2006

42. Encoding the Drills (Future) • Note that the order of drill application is immaterial. • Therefore, when coding, group all the instances of the same drill together, to save space. • Then sort the instances by location and delta encode the locations, since successive instances are relatively close. UniGIS, May 2006

43. Drilling Test on Lk Champlain W - Observations • 12012 posts. • Initial volume of extra material: 1.9·109 • Remaining after 10,000 steps: 4.2·107 • Mean error: 30m (2%) • Frequency of use of each drill type → • Surprising observation: flat bottom drills beat curved bottoms. UniGIS, May 2006

44. More General (Sloped) Drills • Tradeoff powerful, large to encode, basis elements, vs small simple elements, of which we need more. • Sweet point: basis elements resemble object being approximated. • Purpose: to better understand scooping, while initiating experiments in slope-preservation during lossy compression. • Underlying assumption: little long range correlation of elevation or slope. UniGIS, May 2006

45. Regular Scoop Details • 7x7 Scoop size will represents 49 elevations using only 3 coefficients • 7 is not a magic number but good enough for Level 2 DTED cells • Large Errors are rare and mean error is very low, less than 2m • Each scoop is a tilted plane which minimizes the error • Regularity brings simplicity to the representation UniGIS, May 2006

46. Regular 7x7 Tile Scoop Representation W111°N31° Reconstructed (Left), Error (Right) Factor of 49 reduction in number of points UniGIS, May 2006

47. Percent of Elevation Errors onW 111° N 31° UniGIS, May 2006

48. 7x7 Regular Sloped Scoop VIX Evaluation • Comparing Postings with Visibility Index Larger Than 80% • Original (Above), Reconstructed (Below) • Yellow: High VIX • Green: Low VIX • Difference is not easy to discern UniGIS, May 2006

49. Viewshed Evalution of Regular Scooping • Dataset: 3595×3595 • Number of observers: 81 • Elevation range: 809 to 2882. • Observer/target height is 10. • Radius of interest: 300. UniGIS, May 2006

50. Multiobserver Viewshed Comparison Original Terrain Rep Tile 7 Alternate Terrain Rep UniGIS, May 2006