1 / 44

Variational Image Restoration

Variational Image Restoration. Leah Bar. PhD. thesis supervised by: Prof. Nahum Kiryati and Dr. Nir Sochen*. School of Electrical Engineering *Department of Applied Mathematics Tel-Aviv University, ISRAEL. What is image Restoration?.

trella
Download Presentation

Variational Image Restoration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Variational Image Restoration Leah Bar PhD. thesis supervised by: Prof. Nahum Kiryati and Dr. Nir Sochen* School of Electrical Engineering *Department of Applied Mathematics Tel-Aviv University, ISRAEL

  2. What is image Restoration? • Image is degraded by deterministic (blur) and random (noise) processes. • Blur is assumed as linear shift invariant process with additive noise. • Camera out of focus • Motion blur • Atmospheric turbulence • Sensor noise • Quantization • Inverse problem which has been investigated for more than 40 years. • Given the image g and the blur kernel h, restore the original image f .

  3. Image Restoration - Applications Microscopy

  4. Image Restoration - Applications Astronomy

  5. Image Restoration - Applications Medical Imaging

  6. Bayesian and Variational Viewpoints Spatial domain: Frequency domain: Assuming Gaussian distribution of the noise • Maximum Likelihood • Variational • Ill-Posed Solution Noise amplification In high frequencies • Pseudo inverse Filter

  7. Bayesian and Variational Regularization • Maximum a posteriori prob. MAP smoothness prior • Variational (Tikhonov , 1977) • Solution – Wiener Filter (over smoothing)

  8. Edge Preservation observed image - g recovered image - f Image Deconvolution Image Denoising Preserve Edges Edges are very important features in image processing, and therefore have to be preserved.

  9. Total Variation Regularization Wiener Total variation Rudin, Osher, Fetami 1992

  10. Mumford-Shah Segmentation (Mumford and Shah, 1985) data fidelity gradients within segments total edge length Canny edges M-S edges Original Image is modeled as piecewise smooth function separated by edges Ω: image domainK: edge setf: recovered imageg: observed image

  11. Deconvolution with Mumford-Shah Regularization data fidelity gradients within segments total edge length L. Bar, N. Sochen, N. Kiryati, ECCV 2004 M-S functional: difficult to minimize (free-discontinuity problem). Solution is via the G-convergence framework (Ambrosio and Tortorelli 1990) Strategy: approximate the solution by approximation of the problem

  12. G-convergence • A sequence G-converges to if: • liminf inequality • existence of recovery sequence Example: Fj(u)=sin (ju) uj=1.5p n/j (De Giorgi, 1979) G-lim(Fj)=-1

  13. G-convergence Fundamental theorem of G-convergence: Suppose that and let a compact set exist such that for all j, then . Moreover if uj is a converging sequence such that then its limit is a minimum point for F.

  14. * Let satisfy Proof: There exists a subsequence converging to some u, such that This is satisfied for every u and in particular

  15. Deconvolution with Mumford-Shah Regularization data fidelity gradients within segments total edge length v(x): smooth function v(x)~0 at edges v(x)~1 otherwise (in segments)

  16. Deconvolution with Mumford-Shah Regularization • Iterate • Minimize with respect to v by Euler equation (edge detection) • Minimize with respect to f by Euler equation(image restoration)

  17. Zero padding Zero padding Zero padding Convolution Implementation • Neumann boundary conditions • FFT multiplications

  18. Deconvolution with Mumford-Shah Regularization blurred suggested edges (v) suggested restoration

  19. - The restored image is very sensitive to the recovered kernel. - The recovered kernel depends on the contents of the image. • Suggested: Gaussian kernel parameterized by s. Semi-blind Deconvolution via Mumford-Shah Regularization L. Bar, N. Sochen, N. Kiryati, IEEE Trans. Image Processing, 2006 • Blind deconvolution: the blur kernel is unknown • Chan and Wong 1998:

  20. Semi-blind Deconvolution via Mumford-Shah Regularization Chan-Wong suggested method blurred

  21. Total Variation Image Deblurring in the Presence pf Salt-and-Pepper noise L. Bar, N. Sochen, N. Kiryati, Scale Space, 2005 (best student paper) • Special care should be taken in the case of salt-and-pepper noise • L2 fidelity term in not adequate anymore

  22. Median filter 3x3 window • TV restoration • Noise remains! • Median filter 5x5 window • TV restoration • Nonlinear distortion! Image Deblurring in the Presence pf Salt-and-Pepper noise L. Bar, N. Sochen, N. Kiryati, Scale Space, 2005 (best student paper) • Special care should be taken in the case of salt-and-pepper noise • L2 fidelity term in not adequate anymore • Sequential approach: Deblurring following median-type filtering-poor

  23. data fidelity gradients within segments total edge length • Iterate • Minimize with respect to v by Euler equation (edge detection) • Minimize with respect to f by Euler equation(image restoration) Image Deblurring in the Presence pf Salt-and-Pepper noise Suggested approach: robust L1fidelity and Mumford-Shah regularization

  24. Linear operator Image Deblurring in the Presence pf Salt-and-Pepper noise Linearization via fixed point scheme: coefficients in nonlinear terms are laggedby one iteration →linear equation

  25. blurred blurred and noisy suggested 3x3 median + TV 5x5 median + TV Results - pill-box kernel (9x9), radius 4, 10% noise

  26. Results - pill-box kernel (7x7), radius 3, 1% noise blurred and noisy recovered

  27. Results - pill-box kernel (7x7), radius 3, 10% noise blurred and noisy recovered

  28. Results - pill-box kernel (7x7), radius 3, 30% noise blurred and noisy recovered

  29. Theoretical Questions L. Bar, N. Sochen, N. Kiryati, International Journal of Computer Vision • What is the theoretical explanation to the simultaneous deblurring and denoising? • Is Mumford-Shah regularization better than Total Variation? • There is discrimination between image and noise edges. • Image edges are preserved while impulse noise is removed

  30. Edge Preservation robust statistics anisotropic diffusion line process (half quadratic) Relations between: • robust statistics • anisotropic diffusion • line process (half-quadratic) were shown by • Black and Rangarajan, IJCV, 1996 • Black, Sapiro, Marimont and Heeger, IEEE T-IP, 1998 Hampel et al., 1986 Perona & Malik, 1987 Geman & Yang, 1993 Charbonnier et al., 1997

  31. Edge Preservation Influence function-y Gradient Descent: r’(s)=y(s) r(s) 1. Robust smoothness

  32. Edge Preservation • Isotropic diffusion (heat equation) • Anisotropic diffusion (Perona and Malik, 1987) g is “edge stopping” function From robust smoothness point of view Lorentzian 2. Diffusion

  33. Diffusion Illustration Original Isotropic Diffusion Anisotropic Diffusion

  34. Edge Preservation • Dual function b represents edges • Penalty function Y enforces sparse edges across edges otherwise • From robust smoothness point of view 3. Line-process (Half-Quadratic) (Geman and Yang, 1993)

  35. Example: Geman-McClure Function Anisotropic Diffusion Line Process (Half-Quadratic) Robust Smoothing GemanMcClure edge penalty edge stopping function robust r-function

  36. Relation to M-S Terms Edges are forced to be smooth and continuous image edges are preserved • The Geman-McClure function in half-quadratic form Appears in M-S terms with b = v2 M-S: extended line process = extended Geman-McClure

  37. Color Deblurring in the Presence of Impulsive Noise L. Bar, A. Brook, N. Sochen, N. Kiryati, VLSM’05 • Channels have to be coupled • One edge map for all channels

  38. Image Restoration in 3D blurred recovered edges

  39. Future Work: Space Variant Image Restoration preliminary results

  40. Conclusions Novel unified approach to variational segmentation, deblurring and denoising. Mumford-shah regularization reflects the piecewise-smooth model of natural images. Relations to robust statistics and anisotropic diffusion show that Mumford-Shah regularization is a better edge detector. Restoration outcome is superior to state-of-the-art methods

  41. Conclusions Novel unified approach to variational segmentation, deblurring and denoising. Mumford-shah regularization reflects the piecewise-smooth model of natural images. Relations to robust statistics and anisotropic diffusion show that Mumford-Shah regularization is a better edge detector. Restoration outcome is superior to state-of-the-art methods

  42. Conclusions Novel unified approach to variational segmentation, deblurring and denoising. Mumford-shah regularizationreflects the piecewise-smooth model of natural images. Relations to robust statistics and anisotropic diffusion show that Mumford-Shah regularization is a better edge detector. Restoration outcome is superior to state-of-the-art methods

  43. Conclusions Novel unified approach to variational segmentation, deblurring and denoising. Mumford-shah regularization reflects the piecewise-smooth model of natural images. Relations to robust statistics and anisotropic diffusion show that Mumford-Shah regularization isabetter edge detector. Restoration outcome is superior to state-of-the-art methods

  44. Thank you for your attention

More Related