1 / 60

Removing blur due to camera shake from images.

Removing blur due to camera shake from images. William T. Freeman Joint work with Rob Fergus, Anat Levin, Yair Weiss, Fredo Durand, Aaron Hertzman, Sam Roweis, Barun Singh. Massachusetts Institute of Technology. Overview. Our algorithm. Original. Close-up. Naïve Sharpening. Our algorithm.

teresa
Download Presentation

Removing blur due to camera shake from images.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Removing blur due to camera shake from images. William T. Freeman Joint work with Rob Fergus, Anat Levin, Yair Weiss, Fredo Durand, Aaron Hertzman, Sam Roweis, Barun Singh Massachusetts Institute of Technology

  2. Overview Our algorithm Original

  3. Close-up Naïve Sharpening Our algorithm Original

  4. Let’s take a photo Blurry result

  5. Slow-motion replay

  6. Slow-motion replay Motion of camera

  7. Image formation process ⊗ = Blur kernel Blurry image Sharp image Input to algorithm Desired output Convolutionoperator Model is approximation

  8. Why is this hard? Simple analogy: 11 is the product of two numbers. What are they? No unique solution: 11 = 1 x 11 11 = 2 x 5.5 11 = 3 x 3.667 etc….. Need more information !!!!

  9. Multiple possible solutions = ⊗ = ⊗ Sharp image Blur kernel = ⊗ Blurry image

  10. Is each of the images that follow sharp or blurred?

  11. Another blurry one

  12. Natural image statistics Characteristic distribution with heavy tails Histogram of image gradients

  13. Blury images have different statistics Histogram of image gradients

  14. Parametric distribution Histogram of image gradients Use parametric model of sharp image statistics

  15. Uses of natural image statistics • Denoising [Roth and Black 2005] • Superresolution [Tappen et al. 2005] • Intrinsic images [Weiss 2001] • Inpainting [Levin et al. 2003] • Reflections [Levin and Weiss 2004] • Video matting [Apostoloff & Fitzgibbon 2005] Corruption process assumed known

  16. Existing work on image deblurring Assumed forms of blur kernels Software algorithms: • Extensive literature in signal processing community • Mainly Fourier and/or Wavelet based • Strong assumptions about blur  not true for camera shake • Image constraints are frequency-domain power-laws

  17. A focus on image constraints, not image priors

  18. Some image constraints/priors

  19. Toy example: observed “image”: 1.0 0.0

  20. Toy example: observed “image”: 0.0 1.0

  21. Toy example: observed “image”: 1.0 0.0

  22. Three sources of information = ⊗ Estimatedblur kernel Estimated sharp image Input blurry image 3. Blur prior: Positive & Sparse 1. Reconstruction constraint: 2. Image prior: Distribution of gradients

  23. Prior on image gradients (mixture of Gaussians giving a Laplacian-like distribution) Distribution of gradients (log-scale) Green curve is our mixture of gaussians fit.

  24. Prior on blur kernel pixels (mixture of exponentials) P(b) b

  25. How do we use this information? Obvious thing to do: • Combine 3 terms into an objective function • Run conjugate gradient descent • This is Maximum a-Posteriori (MAP)

  26. Maximum A-Posteriori y – observed blurry image x – unobserved sharp image b – blur kernel Blur prior Latent image prior Likelihood Sparse and i – image patch index f – derivative filter Assumption: all pixels independent of one another

  27. Results from MAP estimation Input blurry image Maximum a-Posteriori (MAP) Our method: Variational Bayes

  28. Variational Bayes http://citeseer.ist.psu.edu/cache/papers/cs/16537/http:zSzzSzwol.ra.phy.cam.ac.ukzSzjwm1003zSzspringer_chapter8.pdf/miskin00ensemble.pdf

  29. Miskin and Mackay, 2000

  30. Setup of variational approach Need likelihood and prior in same space, so use gradients: Likelihood Prior on latent image gradients – mixture of Gaussians i – image pixel j – blur pixel We use C=4, D=4 Prior on blur elements – mixture of Exponentials Also have Gamma hyperpriors on

  31. Variational inference • Approximate posterior with • Cost function • Assume • Use gradient descent, alternating between updating while marginalizing out over and vice versa • Adapted code from Miskin & Mackay 2000 is Gaussian on each pixel is rectified Gaussian on each pixel

  32. Variational Bayesian method Based on work of Miskin & Mackay 2000 Keeps track of uncertainty in estimates of image and blur by using a distribution instead of a single estimate Helps avoid local maxima and over-fitting

  33. Variational Bayesian method Objective function for a single variable Maximum a-Posteriori (MAP) Variational Bayes Score Pixel intensity

  34. MAP vs Variational MAP Variational MAP using variational initialization

  35. Blurry synthetic image

  36. Inference – initial scale

  37. Inference – scale 2

  38. Inference – scale 3

  39. Inference – scale 4

  40. Inference – scale 5

  41. Inference – scale 6

  42. Inference – final scale

  43. Our output

  44. Ground truth

More Related