Advanced Techniques in Image Restoration Methods

1. Lecture 3 PDE Methods for Image Restoration

2. Overview • Generic 2nd order nonlinear evolution PDE • Classification: • Forward parabolic (smoothing): heat equation • Backward parabolic (smoothing-enhancing): Perona-Malik • Hyperbolic (enhancing): shock-filtering Artificial time (scales) Initial degraded image

3. Smoothing PDEs

4. Heat Equation • PDE • Extend initial value from to • Define space be the extended functions that are integrable on C

5. Heat Equation • Solution

6. Heat Equation • Fourier transform: • Convolution theorem

7. Heat Equation • Convolution in Fourier (frequency) domain Fourier Transform Attenuating high frequency

8. Heat Equation • Convolution in Fourier (frequency) domain Fourier Transform Attenuating high frequency

9. Heat Equation • Convolution in Fourier (frequency) domain Fourier Transform Attenuating high frequency

10. Heat Equation • Isotropy. For any two orthogonal directions, we have • The isotropy means that the diffusion is equivalent in the two directions. • In particular

11. Heat Equation • Derivation: • Let • Then • Finally, use the fact that D is unitary

12. Heat Equation

13. Heat Equation • Properties: let , then

14. Heat Equation • Properties – continued • All desirable properties for image analysis. • However, edges are smeared out.

15. Nonlinear Diffusion • Introducing nonlinearity hoping for better balance between smoothness and sharpness. • Consider • How to choose the function c(x)? We want: • Smoothing where the norm of gradient is small. • No/Minor smoothing where the norm of gradient is large.

16. Nonlinear Diffusion • Impose c(0)=1, smooth and c(x) decreases with x. • Decomposition in normal and tangent direction • Let , then • We impose , which is equivalent to • For example: when s is large

17. Nonlinear Diffusion • How to choose c(x) such that the PDE is well-posed? • Consider • The PDE is parabolic if • Then there exists unique weak solution (under suitable assumptions)

18. Nonlinear Diffusion • How to choose c(x) such that the PDE is well-posed? • Consider • Then, the above PDE is parabolic if we require b(x)>0. (Just need to check det(A)>0 and tr(A)>0.) where

19. Nonlinear Diffusion • Good nonlinear diffusion of the form if • Example: or

20. The Alvarez–Guichard–Lions–Morel Scale Space Theory • Define a multiscale analysis as a family of operators with • The operator generated by heat equation satisfies a list of axioms that are required for image analysis. • Question: is the converse also true, i.e. if a list of axioms are satisfied, the operator will generate solutions of (nonlinear) PDEs. • More interestingly, can we obtain new PDEs?

21. The Alvarez–Guichard–Lions–Morel Scale Space Theory • Assume the following list of axioms are satisfied

22. The Alvarez–Guichard–Lions–Morel Scale Space Theory

23. The Alvarez–Guichard–Lions–Morel Scale Space Theory

24. The Alvarez–Guichard–Lions–Morel Scale Space Theory curvature

25. Weickert’s Approach • Motivation: take into account local variations of the gradient orientation. • Observation: is maximal when d is in the same direction as gradient and minimal when its orthogonal to gradient. • Equivalently consider matrix • It has eigenvalues . Eigenvectors are in the direction of normal and tangent direction. • It is tempting to define at x an orientation descriptor as a function of .

26. Weickert’s Approach • Define positive semidefinite matrix where and is a Gaussian kernel. • Eigenvalues • Classification of structures • Isotropic structures: • Line-like structures: • Corner structures:

27. Weickert’s Approach • Nonlinear PDE • Choosing the diffusion tensor D(J): let D(J) have the same eigenvectors as J. Then, • Edge-enhancing anisotropic diffusion • Coherence-enhancing anisotropic diffusion

28. Weickert’s Approach • Edge-enhancing Original Processed

29. Weickert’s Approach • Coherence-enhancing Original Processed

30. Smoothing-Enhancing PDEs Perona-Malik Equation

31. The Perona and Malik PDE • Back to general 2nd order nonlinear diffusion • Objective: sharpen edge in the normal direction • Question: how?

32. The Perona and Malik PDE • Idea: backward heat equation. • Recall heat equation and solution • Warning: backward heat equation is ill-posed! Backward Forward

33. The Perona and Malik PDE • 1D example showing ill-posedness • No classical nor weak solution unless is infinitely differentiable.

34. The Perona and Malik PDE • PM equation • Backward diffusion at edge • Isotropic diffusion at homogeneous regions • Example of such function c(s) • Warning: theoretically solution may not exist.

35. The Perona and Malik PDE

36. The Perona and Malik PDE • Catt′e et al.’s modification

37. Enhancing PDEs Nonlinear Hyperbolic PDEs (Shock Filters)

38. The Osher and Rudin Shock Filters • A perfect edge • Challenge: go from smooth to discontinuous • Objective: find with edge-sharpening effects

39. The Osher and Rudin Shock Filters • Design of the sharpening PDE (1D): start from

40. The Osher and Rudin Shock Filters • Transport equation (1D constant coefficients) • Variable coefficient transport equation • Example: Solution: Solution:

41. The Osher and Rudin Shock Filters • 1D design (Osher and Rudin, 1990) Can we be more precise?

42. Method of Characteristics • Consider a general 1st order PDE • Idea: given an x in U and suppose u is a solution of the above PDE, we would like to compute u(x) by finding some curve lying within U connecting x with a point on Γ and along which we can compute u. • Suppose the curve is parameterized as

43. Method of Characteristics • Define: • Differentiating the second equation of (*) w.r.t. s • Differentiating the original PDE w.r.t. • Evaluating the above equation at x(s) (*)

44. Method of Characteristics • Letting • Then • Differentiating the first equation of (*) w.r.t. s • Finally Defines the characteristics Characteristic ODEs

45. The Osher and Rudin Shock Filters • Consider the simplified PDE with • Convert to the general formulation

46. The Osher and Rudin Shock Filters • First case: . Then and • Thus • For s=0, we have and . Thus

47. The Osher and Rudin Shock Filters • Determine : • Since • Using the PDE we have • Thus, we obtain the characteristic curve and solution Constant alone characteristic curve

48. The Osher and Rudin Shock Filters • Characteristic curves and solution for case I

49. The Osher and Rudin Shock Filters • Characteristic curves and solution for case II

50. The Osher and Rudin Shock Filters • Observe • Discontinuity (shock) alone the vertical line at • Solution not defined in the white area • To not introduce further discontinuities, we set their values to 1 and -1 respectively • Final solution