Georeferencing of Images by Exploiting Geometric Distortions in Stereo Images of UK DMC

1. Georeferencing of Images by Exploiting Geometric Distortions in Stereo Images of UK DMC Final Defense Mirza Muhammad Waqar

2. Advisor • Dr. RafiaMumtaz (SEECS, NUST) • Guidance & Examination Committee • Dr. EjazHussain (IGIS, NUST) • Dr. Rizwan Bulbul (IGIS, NUST) • Muhammad Hussan (IGIS, NUST)

3. Contents • Overview • Novel Contribution • Objectives & Scope • Research Novelty • Literature Review • Thermo elastic Model • Model Inversion • Results • Conclusions

4. Overview • Georeferencing • It is the process of assigning geographic coordinates to a digital image. • It is a process for correcting spatial location and orientation of a satellite image • Types of Georeferencing • Direct Georeferencing • Indirect Georeferencing • Required for • Geospatial Analysis • Change Detection • Urban Planning • Map update

5. Overview • Traditional Method of Georeferencing • Require Ground Control Points (GCPs) • GCPs are acquired manually and hence an expensive task • Regions like deserts which lack salient features and have homogeneous texture, selection of GCP is difficult • A large number are required for complex terrain with varied surface elevation. • Accuracy of exterior orientation depends on the accuracy of GCP • Not suitable for push broom imagery as every scanned line possesses a different set of exterior orientation parameters • Proposed method is based on Direct Georeferencing • Does not require Ground Control Points (GCPs) • Utilizes • Satellite position/velocity data • Attitude data • Sensor Configuration to determine the pixel’s geo location.

6. Objectives • The primary goal of this research is to provide accurate georeferenced imagery using no GCPs by mitigating thermo elastic effect. • This will be accomplished by the following objectives • Modeling thermo elastic effect as a transformation matrix • Model inversion in order to extract the thermal deformation from the image offsets • Find geodetic coordinates by correcting pointing error • Validate the complete model by testing it on UK-DMC imagery

7. Novel Contributions • Modeled the thermo elastic effect as transformation matrix by exploiting the inter image offsets present between the pair of images. • Inverted the model to extract the thermal deformation knowledge to remove the pointing error. • Developed a new direct georeferencing method capable of modeling the pointing errors.

8. UK DMC – Sensor Geometry Launch by Surrey Space Center, UK in 2003 operated by DMC international imaging • Sensor: Push broom • Made up of 6 CCD Channels • Making 2 banks of 3 (Port and Star-Board Array) • Separate by an angle β across track • Separate by an angle α along track • Both Port & Star-board Array have10000 linear CCD detectors • 19500 pixel image width with 500 pixels overlap • GSD = 32 m • Spectral Bands: Green, Red, NIR • Ground Swath = 640 km Star-Board Array Port Array DMC Multi Spectral Imager (MSI)

9. Direct Georeferencing • Direct Georeferencing • Transforms the image coordinate in camera frame to geodetic frame. • Satellite onboard attitude data • Satellite position and velocity data • One crucial point • Accuracy of estimated geo-locations is directly dependent on the accuracy of onboard sensor measurements • Main Advantage • Requires no GCPs • Becoming more robust and accurate every year with the ongoing development in GPS and inertial equipment. • Can be applied with aerial camera, hyperspectral scanner, Synthetic Aperture Radar, LIDAR

10. Thermo-elastic Effect • Due to thermo-elasticity • Satellite cools and heat up periodically which causes it to contract and expand. • This introduces changes in the relative orientations of the attitude sensors (e.g. star tracker) and the imager. (Attitude of Imager) (From Attitude Sensor)

11. Literature Review on Thermo elastic Effect • In 2003, work on “Active pixel array devices in space missions“ has been done and it has been found that • The X-ray Telescope for NASA’s Swift mission incorporates a Telescope Alignment Monitor (TAM) to measure thermo-elastic misalignments between the telescope and the spacecraft star tracker. • In 2005, research on “Pleiades-HR Image System Products, Quality And Geometric Accuracy” has been done. • It has been found that attitude sensors are mechanically fixed on the telescope support to minimized the thermo elastic distortions.

12. Literature Review on Thermo elastic Effect • Radhadevi et al in “In-flight geometric calibration of fore and aft cameras of CARTOSAT-1”devised a method for in-flight geometric calibration for Cartosat-1 • The objective of this study is to ensure the best absolute 3D pointing accuracy and relative location accuracy of the cameras. • It is concluded that accuracy of direct orientation observation could be brought down to better than 100 m with the inclusion of in-flight calibrated parameters in to the adjustment model.

13. Literature Review on Thermo elastic Effect • In 2008, a research is conducted on, “Attitude Performance Requirements and Budgeting for RASAT Satellite” • In this study various sources of errors are identified for RASAT satellite. These includes • Star Tracker Errors • Controller Errors • Thermo elastic Error • Finally these errors are combined together in an error budgeting tool for analysis.

14. Research Novelty • In direct geo-referencing the major source of error • Thermo-elastic effect • Creates misalignment between the imaging sensor and the attitude sensor onboard. • Measured and realized orientations will be different. • Mechanical design changes • to reduce the distortion by mounting the imager/star-tracker assembly together on compliant mounts • But this solution will be costly and require changes in the physical mounting of the sensors. • Towards such ends we proposed to model the thermo elastic effect as a transformation matrix • by using the inter-image offsets present between the pair of images. (Attitude of Imager) (From Attitude Sensor)

15. Modeling Thermo-elastic Effect • The thermal deformation could be a rotation, scaling or some sort shearing in the pixels. T = Trotation + Tscale + Tshear • Rotation • It has the major effect on the accuracy of geo-locations. • Small deviations from the true orientation cause a large displacement on the ground. • Scaling • Due temperature changes the focal length of the imager contracts and expands. • This effects the scaling in the height (Z-direction) • Shearing • Non parallel projection of imager results in shearing of pixels.

16. Modeling Thermo-Elastic Effect as a Rotation • This can be determined by • Rotation of Imager • Determined by inter-image offsets present between stereo pair. • Mathematical model has been developed • The inter image offsets equations are the function of senor configuration angles and attitude components. (Attitude of Imager) (From Attitude Sensor)

17. Modeling Thermo-Elastic Effect as a Rotation • Let the port and starboard pixel in body frame be and • Let T be the matrix that represents the thermo elastic deformation. • Let A be the attitude matrix • Next step is to find the equations for the inter-image offsets.

18. Modeling Thermo-Elastic Effect as a Rotation • Column Shift • Determine the corresponding Pixels • Measure the distance of port and starboard pixel • End of their respective array • Difference in the distances gives the column shift Where Where • Row Shift • Time separation • Delay for the corresponding point to appear in the second image

19. Modeling Thermo-Elastic Effect as a Rotation Overlapping region Star Board Array Direction in which satellite is moving Δr Min Row Shift Min Column Shift Max Row Shift Min Column Shift Δc Δr be the row shift Δc be the column shift Port Array

20. Inter-Image Offsets T = At Nominal Attitude and nominal thermo-elastic effect Row Shift Column Shift

21. Offset Modeling as Parabola & Straight Line Attitude matrix • Offset Modeling • Column shift represents parabola • Row shift represents straight line • Column shift as Parabola • Express parabola parameters in terms of attitude components. H (peak value of parabola) a (shape of parabola) Parabola Thermo elastic matrix Where xo (x-intercept for peak value of parabola)

22. Offset Modeling as Parabola & Straight Line • Row shift as straight line • Express slope m and intercept c in terms of attitude parameters Straight Line

23. Effect of Attitude on the Offset Parameters

24. Model Inversion - Modeling Thermo elastic Effect as Rotation • Attitude is determined • Estimating the unknown components of attitude matrix by • solving the offsets equations linearly. • Use properties of attitude matrix • Mapping the estimated components to general attitude matrix • Determine the attitude by using the standard equations Attitude matrix Properties of Attitude Matrix

25. Modeling Thermo-elastic Effect as Scaling • Temperature changes effects the focal length of the imager • Introduces height changes along the Zaxis • Change in scale (along z-axis) effects ground separation of the port and starboard imaging planes. • This will effect the row shift magnitude. • Hence row shift constant equation will be used to extract

26. Modeling Thermo-elastic Effect as Scaling • The cofactor of the Tscaling matrix can be written as, • Where • The cofactor terms appear in expression of row and column offset parameters.

27. Modeling Thermo-elastic Effect as Scaling • Column offset Equations • Where

28. Modeling Thermo-elastic Effect as Scaling • Row Offset Equations • Similarly using cofactors, the expression for b1, b2, b3, b4, b5, b6, b7 and b8 can be reduced to • Using above values, the row shift parameters can be written as

29. Simulations – Sensitivity of Scaling w.r.t offset parameters

30. Scale Extraction from row offset • With scaling matrix the row offset equation will take the form • From the above equation can be found as

31. Modeling Thermo-elastic Effect as Shearing • Shearing slides one edge of an image along the X or Y axis, creating a parallelogram. • The amount of the shear is controlled by a shear angle

32. Modeling Thermo-elastic Effect as Shearing • The cofactors of shear matrix can be written as • The cofactor terms appear in the expression of row and column offset parameters. • Hence

33. Modeling Thermo-elastic Effect as Shearing • Column Offsets Equations • Row Offset Equations Where

34. Simulations – Sensitivity of Shearing w.r.t Offset Parameters

35. Shear Extraction from column offset • Shear pushes the pixels across the track thus effecting the column offset. Therefore column offset will be used to find the shear factor where

36. Thermo-elastic Matrix • Thermo-elastic matrix will be the sum of rotation, scale and shear extracted from the image offsets • Now inserting this matrix between the body and orbital frame will mitigate the misalignment between the imager and the attitude sensor.

37. Direct Georeferencing using onboard Attitude Sensor • Study Area: UK • Date: May 23, 2004 • Rows: 16250 • Columns: 10000 • GPS Positions • X = 4500717.0 m • Y = -55020.32 m • Z = 5445201.0 m • Attitude Values • Roll = 32 centidegree • Pitch = -25 centidegree • Yaw = 47 centidegree

38. Attitude & GPS data Provided by SSTL GPS Data Attitude Data

39. Direct Georeferencing using exterior orientation data • By using the onboard exterior orientation data, the accuracy of 10-15Km has been achieved.

40. Offset estimation using Window based Scheme Column Shift Row Shift

41. Measured Image offsets over the entire Scene length

42. Stereo Angle Estimation • Prior to model inversion, the sensor configuration angles of UK DMC must be determined which are α and β. • The β is being determined by SpaceMetric • β = 12.6448o. • However the magnitude of α can be found from • Rearranging equation of row offset’s constant • For UK DMC band 3 image pair, the value of α was found to be 0.04560.

43. Direct Georeferencing using Imager Attitude • By using imager attitude, the accuracy of 7-10Km has been achieved.

44. Thermo elastic Rotation

45. Direct Georeferencing using thermo elastic Rotation • By applying the thermo elastic • Rotation • The accuracy of 1-5Km has been achieved.

46. Thermo elastic Scale

47. Direct Georeferencing using thermo elastic Rotation and Scale • By applying the thermo elastic • Rotation • Scale • The accuracy of 1-3Km has been achieved.

48. Thermo elastic Shear

49. Georeferencing using thermo elastic Rotation, Scale and Shear • By applying the thermo elastic • Rotation • Scale • Shear • The accuracy of 1-1.5Km has been achieved.

50. Potential Benefits • No GCPs required: • The major benefit of this approach is that it does not based on GCPs. • Collection of GCPs is time consuming and expensive • Difficult to collect GCPs having homogeneous texture • Large number of GCPs are required for complex terrain • Unsuitable for pushbroom imagery • Thermo elastic correction • To date, modeling of thermo elastic effect as a transformation matrix has not been explored. • The thermo elastic effect is deemed as major source of error in this work • Not only provide cost effective solution but also mitigate pointing error • No additional hardware • Low cost and low mass system for obtaining geolocation • Can work with conventional EO cameras with no additional hardware • Frequent Attitude Observations • Due to small baseline or angular separation between the sensors, the registration time for the corresponding pixels is very small which results in rapid attitude observations.