1 / 99

Optical Flow

Optical Flow. Marc Pollefeys COMP 256. Some slides and illustrations from L. Van Gool, T. Darell, B. Horn, Y. Weiss, P. Anandan, M. Black, K. Toyama. last week: polar rectification. Last week: polar rectification. Similarity measure (SSD or NCC). Optimal path (dynamic programming ).

xiu
Download Presentation

Optical Flow

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Optical Flow Marc Pollefeys COMP 256 Some slides and illustrations from L. Van Gool, T. Darell, B. Horn, Y. Weiss, P. Anandan, M. Black, K. Toyama

  2. last week: polar rectification

  3. Last week: polar rectification

  4. Similarity measure (SSD or NCC) Optimal path (dynamic programming ) Last week: Stereo matching • Constraints • epipolar • ordering • uniqueness • disparity limit • disparity gradient limit • Trade-off • Matching cost (data) • Discontinuities (prior) (Cox et al. CVGIP’96; Koch’96; Falkenhagen´97; Van Meerbergen,Vergauwen,Pollefeys,VanGool IJCV‘02)

  5. Questions for assignment?

  6. Tentative class schedule

  7. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

  8. Optical Flow:Where do pixels move to?

  9. Motion is a basic cue Motion can be the only cue for segmentation

  10. Motion is a basic cue Even impoverished motion data can elicit a strong percept

  11. Applications • tracking • structure from motion • motion segmentation • stabilization • compression • mosaicing • …

  12. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

  13. Definition of optical flow OPTICAL FLOW = apparent motion of brightness patterns Ideally, the optical flow is the projection of the three-dimensional velocity vectors on the image 

  14. Caution required ! Two examples : 1. Uniform, rotating sphere  O.F. = 0 2. No motion, but changing lighting  O.F.  0 

  15. Caution required !

  16. Mathematical formulation I (x,y,t) = brightness at (x,y) at time t Brightness constancy assumption: Optical flow constraint equation : 

  17. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

  18. The aperture problem 1 equation in 2 unknowns 

  19. The aperture problem 0

  20. The aperture problem

  21. Remarks

  22. Apparently an aperture problem

  23. What is Optic Flow, anyway? • Estimate of observed projected motion field • Not always well defined! • Compare: • Motion Field (or Scene Flow) projection of 3-D motion field • Normal Flow observed tangent motion • Optic Flow apparent motion of the brightness pattern (hopefully equal to motion field) • Consider Barber pole illusion

  24. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

  25. Horn & Schunck algorithm Additional smoothness constraint : besides OF constraint equation term minimize es+ec 

  26. so the Euler-Lagrange equations are is the Laplacian operator Horn & Schunck The Euler-Lagrange equations : In our case , 

  27. Horn & Schunck Remarks : 1. Coupled PDEs solved using iterative methods and finite differences 2. More than two frames allow a better estimation of It 3. Information spreads from corner-type patterns 

  29. Horn & Schunck, remarks 1. Errors at boundaries 2. Example of regularisation (selection principle for the solution of illposed problems) 

  30. Results of an enhanced system

  31. Structure from motion with OF

  32. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

  33. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

  34. Optical Flow • Brightness Constancy • The Aperture problem • Regularization • Lucas-Kanade • Coarse-to-fine • Parametric motion models • Direct depth • SSD tracking • Robust flow • Bayesian flow

More Related