1 / 16

A Duality Based Approach for Realtime TV-L 1 Optical Flow

A Duality Based Approach for Realtime TV-L 1 Optical Flow. Christopher Zach 1 , Thomas Pock 2 , and Horst Bischof 2. 1 VRVis Research Center, Graz 2 Institute for Computer Graphics and Vision, TU Graz E-mail: { zach, pock, bischof }@icg.tugraz.at. time. Motivation.

ham
Download Presentation

A Duality Based Approach for Realtime TV-L 1 Optical Flow

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Duality Based Approach for Realtime TV-L1 Optical Flow Christopher Zach1, Thomas Pock2, and Horst Bischof2 1 VRVis Research Center, Graz 2 Institute for Computer Graphics and Vision, TU Graz E-mail: {zach, pock, bischof}@icg.tugraz.at

  2. time Motivation • Discontinuity preserving regularization of the flow field • Robustness to occulsions • Handle large displacements • Realtime (> 30 fps) for large images (512x512)

  3. Outline • (I) Variational Optical Flow • (II) TV-L1 optical flow • (III) Duality Based Approach • (IV) Acceleration using the GPU • (V) Performance Evaluation • (VI) Conclusion & Demo

  4. Optical Flow • Optical Flow (OF) is a major task of biological and artificial visual systems • Relates the motion of pixel intensities between consecutive image frames • Optical Flow Constraint: • Gives only the normal flow • No OFC in untextured areas u1 u2 u

  5. Variational Optical Flow • First studied by Horn and Schunck in 1981 [1] • Quadratic regularization does not allow for discontinuities and occlusions • Modifying the Horn and Schunck functional was pioneered by Black and Rangarajan [2] [1] B.K. Horn and B.G. Schunck. Determinig Optical Flow. Artificial Intelligence, 1981 [2] M.J. Black and P. Rangarajan. On the Unification of Line Processes, Outlier Rejection and Robust Statistics with Applications in Early Vision, IJCV, 1996

  6. TV-L1 Optical Flow • We use a robust variant of the Horn-Schunck formulation • Total Variation (TV) of Rudin Osher and Fatemi (ROF) [3] • L1 penalization of the OF constraint • TV-L1 has been used in many approaches • Allows for discontinuities in the flow field and outliers in the optical flow constraint • Sophisticated optimization techniques are needed • This is the major goal of this paper [3] L. Rudin and S. Osher and E. Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 1992

  7. Eθ E as Θ 0 An Approximative Formulation • Main difficulty is induced by the TV term –> ROF model [3] • Simple pointwise optimization problem -> Thresholding [3] L. Rudin and S. Osher and E. Fatemi. Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 1992

  8. Primal Formulation • Study of the ROF model • Primal Euler Lagrange equations • Degenerated if gradient vanishes • Simple solution: Replace by • Disadvantage: Large εsmoothes edges!

  9. |p| ≤ 1 p Dual Formulation • Studied by Chan [4], and later by Chambolle [5] • One arrives at two new equations • Dual Euler Lagrange Equations • Advantage: No regularization is needed! [4] T. Chan and G. Golub and P. Mulet, A Nonlinear Primal Dual Method for TV-based Image Restoration, 1999 [5] A. Chambolle, An Algorithm for Total Variation Minimization and Applications, 2004

  10. Primal vs. Dual • Convergence of the Primal and Dual formulation • Primal: fixed-point scheme of Vogel & Oman [6] • Dual: fixed-point scheme of Chambolle [5] ε=10 ε=10-15 ε=10-1 ε=1 EROF iterations [5] A. Chambolle, An Algorithm for Total Variation Minimization and Applications, 2004 [6] C. R. Vogel and M. E. Oman. Iterative Methods For Total Variation Denoising. 1996

  11. Final Algorithm • Energy minimization is embedded into a coarse-to-fine approach to handle large displacements • Solved via alternating optimization • Fix v, minimize wrt. u (Chambolle‘s algorithm) • Fix u, minimize wrt. v (Thresholding) • Goto 1 until convergence

  12. G92 Nov 2007 Implementation on Graphics Hardware • Particularly well suited to compute variational methods • High degree of parallelism • High performance processing units • All features can be accessed via C-like languages • Performance of graphics cards is steadily increasing

  13. Performance Evaluation Frames per second Error evaluation on the well known Yosemite without clouds sequence [1] B.K. Horn and B.G. Schunck. Determinig Optical Flow. Artificial Intelligence, 1981 [7] T. Nir and A.M. Bruckstein and R. Kimmel, Over-Parameterized Variational Optical Flow, IJCV 2007

  14. Conclusion & Future work • We have developed a duality based algorithm for TV-L1 optical flow computation • We have implemented this algorithm on state-of-the-art graphics hardware • In summary, we obtained an optical flow algorithm having a realtime performance of ~45 fps for 512x512 images • Implementation in CUDA should give an additional speedup • More sophisticated data terms for illumination changes • Multigrid techniques for the dual formulation

  15. Demo

  16. Solution of the ROF model • Compute the minimizer of ROF model • Solution of a huge sytem of non-linear equations • Leads to iterative algorithms • Primal formulation: • Fixed-point scheme of Vogel and Oman [6] • Dual formulation • Fixed-point scheme of Chambolle [5] [5] A. Chambolle, An Algorithm for Total Variation Minimization and Applications, 2004 [6] C. R. Vogel and M. E. Oman. Iterative Methods For Total Variation Denoising. 1996

More Related