Accelerated Image Orthorectification in NASA WorldWind

1. Accelerated Image Orthorectification in NASA WorldWind By Craig Collins

2. Topics covered… • A low cost method of georeferencing data • Projectively texturing on 3D terrain • Shadow mapping • Lens Distortion correction

3. Photogrammetry with a UAV • Camera is a ‘frame sensor’ • Various rigorous sensor models exist including pushbroom, scanline, and synthetic aperture radar • DEM – digital elevation model (essential for orthorectification) • Orthorectification – a type of 2D map projection where there is no perspective distortion

4. OSSIM ossimImageMosaic • Performs histogram matching, tonal balancing, and layering of data • Supports Applanix POS/AV DG sensor model – frame sensor (a calibrated camera) • Blends photos using layering, closest-to-center, feather blending, etc, etc… • Requires a pretty good DEM

5. The old method of georeferencing • Space resection using ground control, tie points, and camera model geometry • The camera model data is reconciled by ground control, however no biases are seen in the data

6. The Kalman filter • The Kalman filter is an efficient recursive filter which estimates the state of a dynamic system from a series of incomplete and noisymeasurements. • An example of an application would be to provide accurate continuously-updated information about the position and velocity of an object given only a sequence of observations about its position, each of which includes some error. It is used in a wide range of engineering applications from radar to computer vision.

7. The new method of direct georeferencing “The combination of these … GPS and IMU, with a Kalman filter allows the accurate determination of both attitude (roll, pitch, heading) and position (x, y, z) of the camera at the time of exposure. This results in the same information that is normally obtained in photogrammetric block or bundle adjustments used in film collections. The primary operational difference is that the Emerge approach does not require ground control to obtain the solution of the attitude and position data. This saves large amounts of labor and schedule time, resulting in a low cost, georeferenced image product. “

9. Accuracy of direct georeferencing of RMK images (scale 1:13000, hg=2000m)[1]

10. UAV demo for SGI workstation “UAV demo shows an unusual use of projective texturing and shadow testing for accelerated image orthorectification. “

11. SGI’s UAV demo technology • Written in OpenGL for the SGI workstation • Undos perspective distortion of an image on a per pixel basis through perspective divide • Uses shadow mapping techniques to occlude regions of terrain not covered by image

13. Shadow Mapping with HLSL • Draws an image of the scene from the light’s perspective • Depth buffer stores information of distance to closest surface for each pixel represented in the scene • The depth buffer is projectively textured to the scene • The scene is drawn from the camera’s perspective. Parts of scene where depth value is less than true value of depth to camera are in shadow

16. Pixel Shader 2.0 – Released 2004 • Small programs are written in HLSL (High Level Shader Language), a language resembling C++ • The program is ran on the GPU rather than the CPU • Allows DirectX developer to use more complex rendering techniques (shadow mapping, projective texturing, etc, etc…)

17. Lens distortion terminology • Tangential lens distortion – "decentering", or imperfect centering of the lens components and other manufacturing defects in a compound lens • Radial lens distortion – Distortion present in fisheye lenses • Principal point – The center point of an aerial photograph

18. "Plumb Bob" model of lens distortion • Contains 5 distortion coefficients (K) • Other inputs – Focal length, principal point, and skew coefficient compose the camera matrix • Not advised to use 5th coefficient, 3rd and 4th are generally not used either • It is claimed that the error in the principal point of today’s cameras is negligible

19. Matlab camera toolbox • r is radius from principal point to pixel

20. UAV Plugin 1.1 uses K1 and K2 • Using only the first two is much quicker and results are acceptable • Bilinear interpolation eliminates aliasing effects that cause image to look rough and blocky • All 5 coefficients can be found using the Matlab camera toolbox • OpenCV supports camera calibration

21. UAV for World Wind - Additional features • Code exists as a plugin (no more compilation required by the developer) • Real-time modification of individual image parameters • Coexists with WW images • Support for pre-processing of images on pixel level (for histograms)

23. Sources • GPS/inertial data in aerial photogrammetry - http://www.ifp.uni-stuttgart.de/forschung/photo/georef-Dateien/georef.en.html • SGI UAV demo http://www.sgi.com/products/software/performer/brew/uav.html • American Society for Photogrammetry and Remote Sensing http://www.asprs.org/publications/pers/2002journal/may/georef.html • OSSIM http://www.ossim.org/ • Real-time Lens distortion correction http://ieee.stanford.edu/ecj/docs/ECJ_demo.pdf • Matlab Camera Calibration script http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/parameters.html • Kalman filtering • http://www.cs.unc.edu/~welch/kalman/