1 / 72

Single Image Blind Deconvolution

Single Image Blind Deconvolution. Presented By: Tomer Peled & Eitan Shterenbaum. Agenda. Problem Statement Introduction to Non-Blind Deconvolution Solutions & Approaches Image Deblurring PSF Estimation using Sharp Edge Prediction / Neel Joshi et. Al. MAP x,k Solution Analysis

spike
Download Presentation

Single Image Blind Deconvolution

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Single Image Blind Deconvolution Presented By: TomerPeled &Eitan Shterenbaum

  2. Agenda • Problem Statement • Introduction to Non-Blind Deconvolution • Solutions & Approaches • Image Deblurring PSF Estimation using Sharp Edge Prediction / Neel Joshiet. Al. • MAPx,kSolution Analysis Understanding and evaluating blind deconvolution algorithms / Anat Levin et. Al. C. Variational Method MAPk Removing Camera Shake from a Single Photograph / Rob Fergus et. Al • Summary

  3. Problem statement • Blur = Degradation of sharpness and contrast of the image, causing loss of high frequencies. • Technically - convolution with certain kernel during the imaging process.

  4. Camera Motion blur

  5. Defocus blur

  6. Defocus blur

  7. Defocus blur 7

  8. Blur – generative model = Sharp image Blured image Point Spread Function =  fft(Image) fft(Blured image) Optical Transfer Function

  9. Object Motion blur

  10. Local Camera Motion

  11. Depth of field – Local defocus

  12. Evolution of algorithms ? Camera motion blur Simple kernels Non blind deconvolution Volunteers ? Shan 2008 Fergus 2006 Joshi 2008 Lucy Richardson 1972 Wiener 1949

  13. Introduction to Non-Blind Deconvolution sharp image blurred image blur kernel noise Deconvolution Evolution: Simple no-Noise Case Noise Effect Over Simple Solution Wiener Deconvolution RL Deconvolution

  14. Recovered Simple no-Noise Case: Blurred Image

  15. Noisy case: Original (x) Blured + noise (y) Recovered x

  16. Noisy case, 1D Example: FT of original signal Original signal Convolved signals w/o noise FT of convolved signals sd Reconstructed FT of the signal Noisy Signal Original Signal High Frequency Noise Amplified

  17. Wiener Deconvolution Blurred noisy image Recovered image

  18. Non Blind Iterative Method : Richardson –Lucy Algorithm Assumptions: Blurred image yi~P(yi), Sharp image xj~P(xj) i point in y, j point in x Target: Recover P(x) given P(y) & P(y|x) where From Bayes theorem Object distribution can be expressed iteratively: Richardson, W.H., “Bayesian-Based Iterative Method of Image Restoration”, J. Opt. Soc. Am., 62, 55, (1972). Lucy, L.B., “An iterative technique for the rectification of observed distributions”, Astron. J., 79, 745, (1974).

  19. Richardson-Lucy ApplicationSimulated Multiple Star measurement PSF Identification reconstruction of 4th Element

  20. Solution Approaches • Image Deblurring PSF Estimation using Sharp Edge Prediction Neel Joshi Richard Szeliski David J. Kriegman • MAPx,k Solution Analysis Understanding and evaluating blind deconvolution algorithms Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman C. Variational Method MAPk Removing Camera Shake from a Single Photograph Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman

  21. PSF Estimation by Sharp Edge Prediction Given edge steps, debluring can be reduced to Kernel Optimization Suggested in PSF Estimation by Sharp Edge Prediction \ Neel Joshi et. el. in Select Edge Step (Masking) Estimate Blurring Kernel Recover Latent Image

  22. PSF Estimation by Sharp Edge Prediction - Masking Original Image Edge Prediction Masking Max ValidRegion Min

  23. Masking, Cont.Which is Best the Signals ? Original Blurred Edge Impulse

  24. PSF Estimation by Sharp Edge Prediction – PSF Estimation Blurr Model: y=x*k+n, n ~ N(0,σ2) Bayseian Framework: P(k|y) = P(y|k)P(k)/P(y) Map Model: argmaxk P(k|y) = argminkL(y|k) + L(k)

  25. PSF Estimation by Sharp Edge Prediction – Recovery Recovery through Lucy-Richardson Iterations given the PSF kernel Blurred Recovered

  26. PSF Estimation by Sharp Edge, Summary & Improvements • Handle RGB Images – perform processing in parallel • Local Kernel Variations: Sub divide image into sub-image units Limitations: • Highly depends on the quality of the edge detection • Requires Strong Edges in multiple orientations for proper kernel estimation • Assumes knowledge of noise error figure.

  27. MAPx,k , BlindDeconvolution Definition sharp image noise blurred image blur kernel Unknown, need to estimate Input (known) ? ? Courtesy of Anat Levin CVPR 09 Slides

  28. MAPx,k Cont. - Natural Image Priors Gaussian: -x2 Laplacian: -|x| -|x|0.5 Log prob -|x|0.25 x x Derivative histogram from a natural image Parametric models Derivative distributions in natural images are sparse: Courtesy of Anat Levin CVPR 09 Slides

  29. Naïve MAPx,k estimation Given blurred image y, Find a kernel k and latent image x minimizing: Convolution constraint Sparse prior Should favor sharper x explanations Courtesy of Anat Levin CVPR 09 Slides

  30. The MAPx,k paradox P( , )>P( , ) kernel kernel Latent image Latent image Let be an arbitrarily large image sampled from a sparse prior , and Then the delta explanation is favored Courtesy of Anat Levin CVPR 09 Slides

  31. The MAPx,k failure sharp blurred ? Courtesy of Anat Levin CVPR 09 Slides

  32. The MAPx,k failure 45x45 windows 25x25 windows 15x15 windows simple derivatives [-1,1],[-1;1] FoE filters (Roth&Black) Red windows =[ p(sharp x) >p(blurred x) ]

  33. The MAPx,k failure - intuition > k=[0.5,0.5] P(step edge) P(blurred step edge) cheaper sum of derivatives: < P(impulse) P(blurred impulse) cheaper sum of derivatives: Courtesy of Anat Levin CVPR 09 Slides

  34. MAPx,k Cont. - Blur Reduces Derivative Contrast < P(sharp real image) P(blurred real image) cheaper • Noise and texture behave as impulses - total derivative contrast reduced by blur Courtesy of Anat Levin CVPR 09 Slides

  35. MAPx,k Reweighting Solution High Quality Motion Debluring From Single Image / Shan et al. Alternating Optimization Between x & k Minimization term: MAPx,k

  36. MAPx,k Reweighting - Blurred 39

  37. MAPx,k Reweighting - Recovered 40

  38. Solution Approaches • Image Deblurring PSF Estimation using Sharp Edge Prediction Neel Joshi Richard Szeliski David J. Kriegman • MAPx,kSolution Analysis Understanding and evaluating blind deconvolution algorithms Anat Levin, Yair Weiss, Fredo Durand, William T. Freeman C. Variational Method MAPk Removing Camera Shake from a Single Photograph Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, William T. Freeman

  39. MAPk estimation Given blurred image y, Find a kernel minimizing: Convolution constraint Sparse prior Kernel prior Again, Should favor sharper x explanations

  40. Superiority of MAPk over MAPk,x Toy Problem : y=kx+n uncertainty of p(k|y) reduces given multiple observations yj =kxj + nj . The joint distribution p(x, k|y). Maximum for x → 0, k → ∞. p(k|y) produce optimum closer to true k∗.

  41. Evaluation on 1D signals Exact MAPk MAPx,k Favors delta solution Favor correct solution despite wrong prior! MAPk variational approximation (Fergus et al.) MAPk Gaussian prior Courtesy of Anat Levin CVPR 09 Slides

  42. Intuition: dimensionality asymmetry MAPx,k– Estimationunreliable. Number of measurements always lower than number of unknowns: #y<#x+#k MAPk– Estimation reliable. Many measurements for large images: #y>>#k kernel k sharp image x blurred image y Large, ~105 unknowns Small, ~102 unknowns ~105 measurements Courtesy of Anat Levin CVPR 09 Slides

  43. Courtesy of Rob Fergus Slides

  44. Three sources of information Courtesy of Rob Fergus Slides

  45. Image prior p(x) Courtesy of Rob Fergus Slides

  46. Blur prior p(b) Courtesy of Rob Fergus Slides

  47. The obvious thing to do Courtesy of Rob Fergus Slides

  48. Variational Bayesian approach Courtesy of Rob Fergus Slides

  49. Variational Bayesian methods • Variational Bayesian = ensemble learning, • A family of techniques for approximating intractable integrals arising in Bayesian inference and machine learning. • Lower bound the marginal likelihood (i.e. "evidence") of several models with a view to performing model selection.

  50. Setup of Variational Approach

More Related