1 / 26

Globally Optimal Wavelet-Based Motion Estimation using Interscale Edge and Occlusion Models

Globally Optimal Wavelet-Based Motion Estimation using Interscale Edge and Occlusion Models. Levent Sendur lss29@cam.ac.uk Department of Psychiatry, University of Cambridge, UK. and. Onur G. Guleryuz oguleryuz@erd.epson.com Epson Palo Alto Laboratory Palo Alto, CA.

alyson
Download Presentation

Globally Optimal Wavelet-Based Motion Estimation using Interscale Edge and Occlusion Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Globally Optimal Wavelet-Based Motion Estimation using Interscale Edge and Occlusion Models Levent Sendur lss29@cam.ac.uk Department of Psychiatry, University of Cambridge, UK and Onur G. Guleryuz oguleryuz@erd.epson.com Epson Palo Alto Laboratory Palo Alto, CA (Please view in full screen mode to see the animations.)

  2. Overview • Problem statement and the space we are trying to fill among estimation techniques. • Properties. • Probability Model: • Model for uncovered regions (using DCTs). • Observation model. • Motion field model (in wavelet domain). • Interscale dependencies. • Dynamic programming. • Examples. • Conclusion.

  3. Problem Statement Well known motion estimation techniques have two important issues • Methods that are deemed to be accurate (such as optical flow algorithms and probabilistic extensions) generate iterative estimates that are only locally optimal. • Models of uncovered regions and motion field discontinuities are incorporated as an afterthought. These models contradict regularization constraints and result in unnatural tug-of-war in the iterations. We propose a dense motion field estimation technique that is noniterative, that can be optimized globally with edges and uncovered regions built in.

  4. Block based This paper … Optical flow based, Markov random fields, … Available Estimation Techniques optimization globally optimal locally optimal crude model sophisticated model sophistication

  5. Properties of this Work • Per pixel, dense field (but flexible, can use blocks of pixels to reduce computation). • Stochastic and non-iterative. • Piecewise smooth motion field model that captures local smoothness. • Discontinuities and occlusions built in. • Motion field parameterized in terms of its wavelet transform coefficients. • Wavelet based model accounts for interscale dependencies over motion field edges. • Novel and robust occlusion model. • Deliverables: Globally optimal solution + Motion field segmentation.

  6. Table Tennis, frames 3 and 4 frame n-1 frame n : smooth : edge : uncovered Field segmentation Field

  7. Part that is explained by motion Part that is explained by uncovered regions : occlusion/uncovered region segmentation : motion field : i.i.d. noise Frame Evolution Model : current and past frames : pixel coordinates

  8. Bayes Rule Stochastic Model model each conditional probability in turn Region has motion if: Cheaper to “code” with motion vectors (motion vector cost +DFD) Region if uncovered if: Cheaper to code using DCTs.

  9. Establish a block around each pixel. • Assume the DCT coefficients of this block are independent Gaussian RVs, N N average coding cost in bits Uncovered Regions Given Occlusion Segmentation : model eachpixel using DCTs. (Given that this pixel is uncovered, what is the probability that it has the value …?) … x … … • Estimate mean and variance for each DCT coefficient. …

  10. Observation Model (i.i.d. Gaussian noise)

  11. Motion Model : Piecewise smooth Wavelet transform coefficients of the field are sparse. Most coefficients have magnitudes that are close to zero.

  12. l=4 l=3 l=2 Wavelet Transform of the Motion Field l=1 y y’ w w’ x x’ z z’

  13. Wavelet Transform of the Motion Field

  14. l=4 l=3 l=2 Tree Structure for Everything • All segmentations defined on nodes of our multiresolution tree data structure. • Probability at each node conditionally independent given the parent node (Multiresolutional Markov structure). l=1 y y’ w w’ x x’ z z’

  15. Interscale Dependency Model I Segmentation hierarchy follows our tree data structure. l=4 l=3 • Uncovered node hierarchy: l=2 • Motion node hierarchy: or l=1 y y’ w w’ x x’ z z’

  16. Motion over smooth region: • Motion over edge region: or Interscale Dependency Model II Motion node hierarchy: Why? Wavelets are sparse but not that sparse. There are interscale dependencies over wavelet coefficients over edges. l=4 l=3 l=2 l=1 y y’ w w’ x x’ z z’

  17. Dynamic Programming I 1- Exhaustively consider: • all possible motion vectors on all pixels, • all possible segmentations. 2- Evaluate the log cost: 3- Find the combination that minimizes the log cost.

  18. Dynamic Programming II It is dynamic programming because: 1- Tree structure defines additive costs at each node. 2- Multiresolutional Markov structure allows elimination at each node. … l=2 l=1 y y’ w w’ x x’ z z’ Can choose among all that sum up to the same (Bottom-up dynamic programming by propagating three types of costs at each node: uncovered cost, motion-smooth cost, motion-edge cost.)

  19. Table Tennis, frames 3 and 4 (over 4x4 blocks) frame n-1 frame n : smooth : edge : uncovered Field segmentation Field

  20. Stefan, frames 55 and 56 frame n-1 frame n : smooth : edge : uncovered

  21. Stefan, frames 55 and 56 l=11 l=14 l=13 l=12 l=10 l=9 l=8 l=7 l=6 l=5 l=4 l=3 l=2 l=1 : smooth : edge : uncovered

  22. Football, frames 128 and 129 frame n-1 frame n : smooth : edge : uncovered

  23. Football, frames 98 and 99 frame n-1 frame n : smooth : edge : uncovered

  24. Football, frames 136 and 137 frame n-1 frame n : smooth : edge foot13 : uncovered

  25. Coast Guard, frames 157 and 158 frame n-1 frame n : smooth : edge : uncovered

  26. Conclusion • Globally optimal, dense field. • Noniterative. • Field segmentation. • Robust uncovered region model. • Transform domain models. • Interscale dependencies accounted for. • Dynamic programming (1D) applied in 2D. • Results model (no optimization issues). • Better edge determination. • In conjunction with edges, better uncovered region determination.

More Related