ELE 488 Fall 2006 Image Processing and Transmission (10-17-06)

1. 10/17/06 ELE 488 Fall 2006Image Processing and Transmission (10-17-06) • Geometric Transformation • translation • rotation • scaling • Affine • Image Registration • Goodness of fit

2. Mosaicking and Registration Reference Image Floating Image

3. Identify Features and Matchto Determine Geometric Transformation Image I (reference) h(x,y) Image II (floating) h–1(x’,y’) Two cartesian coordinate systems: (x,y) and (x’,y’) Forward: (x’,y’) = h(x,y) Reverse: (x,y) = h-1(x’,y’)

4. y y’ (1, 1) (2, 1) (3, 1) x’ x (1, 0) Four Basic Transformations translation rotation scaling shearing

5. Combining Transformations • Rotation and translation • Successive application of rotation, translation and scaling. • Involves matrix multiplications and additions • Homogeneous Coordinates Matrix multiplication, no addition

6. Translation – Rotation – Scaling Translation: Rotation: Scaling: Note: c=0

7. Combining Transformations 1) Translate 2) Rotate 3) Translate

8. Homogeneous Transforms • Rigid-body: Rotation and Translation • 2x2 rotation submatrix is orthonormal • Translation/Rotation/Scaling • are determined by rotation and scaling parameters

9. Affine and Projective Transformations • Affine: 6 free parameters • Projective: 8 free parameters • More general geometric transformations between two planes • Widely used in modeling cameras s scaling factor.

10. Inverse Transform Examples:

11. Reflections Reflect in x-axis Reflect in y-axis Reflect in line y=x

12. Reflect about a general line y=ax+b

13. Pinhole Camera Model

14. Pinhole Camera Model (shifted image plane)

15. Image Formation: pin hole model Projection of point (X,Y,Z) in 3D space to (x,y) in image plane

16. Mosaicking and Registration Reference Image Floating Image Two camerasOr one camera at two positions

17. Two Cameras Point in 3-D has two sets of coordinates

18. Rotation of Camera About One Axis

19. General Rotation of Camera Two 3-D coordinates of a point are related by a rotation matrix: rotation matrix R Decompose R into 3 rotations about the first camera axes: Counter clockwise rotation about Z axis Counter clockwise rotation about X axis Counter clockwise rotation about Y axis

20. Translation location of 2nd camera in 1st camera coordinates

21. 3-D Coordinate Transformation The two 3-D coordinates of a point are related by a 3-D rotation and a translation: Counter clockwise rotation about Z axis Counter clockwise rotation about X axis Counter clockwise rotation about Y axis from pinhole camera

22. 3-D Coordinate Transformation

23. 3-D Coordinate Transformation Independently scale each side of the equation

24. No Translation Homogeneous transformation Knowing establishes the correspondence between the images.

25. Examples

26. Flat Scene See slide 9 Homogeneous transformation Knowing establishes the correspondence between the images.

27. Planar Scene Homogeneous transformation Knowing establishes the correspondence between the images.

28. Mosaic: Firestone Library Photos: P. Ramadge, Mosaic: Y.P. Tan/R. Radke The zero translation approximation was used to register these images

29. Affine and Projective Transformations • Projective Transformations are closely connected to image formation models and camera rotation • Affine is an approximation:

30. Small Rotations Can be approximated by affine

31. Three Questions of Image Registration • How is the spatial transformation (= warping function) applied and what family of warping functions are allowed? • How do we measure how well the warped floating image aligns with the reference image? • How to we find the best warp in the allowed class to best match the images (optimization)

32. Warping Function Examples Translation Rotation Scaling Affine Projective

33. y x Goodness of Fit:How well does warped image align with reference 1)Use all pixel values: 2)Only use pixel values around selected “features”: What are features?

34. v u Features and Feature Selection Sharp  corners GOOD BAD v u

35. Registration As An Optimization Problem Consider transformations parameterized by a vector of parameters θ Find the transformation that minimizes the registration metric (e.g., above) In some special cases this can be solved analytically (in closed form) In other cases it can be approximated and solved analytically In the most complex cases it has to be solved iteratively by a gradient-like descent method.

36. Example: Translation + SSD Feature Metric 1 feature - can do this by brute force search by finding the best match More features?

37. Simplification Find the best translation for each feature individually by a brute forcesearch (can be refined) This gives (called a correspondence) Then select the translation for the entire image to minimize : This yields

38. Image Registration Reference Image Floating Image • 756x504 pixels • 16 bit color • Allowed warpings is translations • Matching metric is how well a selected subset of “features” match using SAD • best translation match. average pixels where images overlap

39. Registration Using Affine Transformation translation, rotation, scaling, shearing

40. Registration Using Projective Transformation Add: tilting