1 / 46

Image Alignment by Image Averaging

Image Alignment by Image Averaging. David Hong NCSSM, IE364 2008. Example Problem 1. Example Problem 2. Example Problem 3. Problem (Formal Statement). Motivation. Many Applications: Special Effects (Movie) Video Compression Pattern Recognition Image Stabilization (Digital Cameras)

dard
Download Presentation

Image Alignment by Image Averaging

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Image Alignment byImage Averaging David Hong NCSSM, IE364 2008

  2. Example Problem 1

  3. Example Problem 2

  4. Example Problem 3

  5. Problem (Formal Statement)

  6. Motivation Many Applications: • Special Effects (Movie) • Video Compression • Pattern Recognition • Image Stabilization (Digital Cameras) • Dead-reckoning (Mobile Robotics)

  7. State of the Art Optical Flow Lucas-Kanade (1985) Optical Flow with Smoothness Constraint Horn-Schunck(1980) Phase Correlation

  8. Lucas-Kanade U(X,Y) u(x,y,t) Y y x X

  9. Lucas-Kanade Let us consider a point on the plane. Then the floor-coordinate is (X0,Y0) and the sensor-coordinate is (x,y) at time t. From there, we can see:

  10. Lucas-Kanade Differentiating on time gives us:

  11. Lucas-Kanade Expressing (x,y) in terms of (X0,Y0) and the sensor position (Xs,Ys,Θs) gives us:

  12. Lucas-Kanade Putting the two together, we get: This is underdetermined!

  13. Algorithm u’ u

  14. Improvement by Iteration u u’ u’’’ u’’

  15. Improvement by Iteration u’(x’,y’) u’ u’’(x’’,y’’) u’’ u

  16. Improvement by Iteration

  17. Improvement by Iteration u’ (x’0, y’0) (x’1, y’0) (x’, y’) (x’0, y’1) (x’1, y’1) Places u’ is defined Place we need to evaluate u’

  18. Improvement by Iteration

  19. Improvement by Iteration u’ u’ not defined! u’’ u

  20. Improvement by Iteration u’ u’’ was not evaluated here u’’ Valid Region

  21. Improvement by Iteration (i0,1’, j0,1’ ) (i-1,0’, j-1,0’ ) (i1,0’, j1,0’ ) Places we need to evaluate u’ (i0,-1’, j0,-1’ )

  22. Performance of Algorithm Good Surface: Bad Surface: Algorithm Fails!

  23. Performance of Algorithm Surface:

  24. Performance of Algorithm Surface:

  25. Performance of Algorithm Surface:

  26. Assumptions Made • The Error Function is locally quadratic • The floor is linear

  27. Weaknesses • Many Iterations • Inherent to Technique • “Fooled” by symmetry (Aliasing problem) • Inherent to Problem

  28. Strengths • Accurate • Improvement by Iteration • Finds Error Function Root by Newton’s

  29. Phase Correlation u(xu,yu) v(xv,yv) yu yv xv xu

  30. Phase Correlation We consider the image to be like a 2-D wave. Then, displacement is simply a “phase shift” Rotation can similarly be found So, we “correlate” the “phases”

  31. Phase Correlation

  32. Phase Correlation

  33. Phase Correlation

  34. Phase Correlation

  35. Weaknesses • Inaccurate on first iteration • Boundary Problem (Repetion Assumption) • High complexity • FFT is “O(nlogn)”

  36. Strengths • Elegant • Makes a big leap • Works well on images with pattern • Separates displacement and rotation (DFT)

  37. Image Averaging u(xu,yu) v(xv,yv) yu yv xv xu

  38. Image Averaging • Find a “Center-of-Mass” of each image • Track the motion of the center-of-mass

  39. Weaknesses • Boundary Problem (Average Point Moves) • Average is affected by small discretization issues

  40. Strengths • Elegant • Makes a big leap • Very fast • Complexity of O(n) • Yields itself well to Improvement by Iteration • Using same technique as in Lucas-Kanade

  41. Handling the Weaknesses • Here we decide to take an alternative approach • Separate displacements from rotation • Do this using FFT (as in Phase Correlation)

  42. Handling the Weaknesses • We handle rotation first • Post-FFT, only rotation remains

  43. Handling the Weaknesses

  44. Weaknesses • We introduce an FFT ( O(nlogn) operation) • However, only requires 2 • Phase correlation requires up to 3 or 4

  45. Future Work • Make Image Alignment Rigorous • Use complex numbers to notate displacement • Smoothness Constraint • Pre-processing the image • Condition for Convergence

  46. Thank You!

More Related