IPAM-UCLA Tutorial May 14-18, 2001 GBM in Image Processing, Computer Vision, and Computer Graphics

1. IPAM-UCLA TutorialMay 14-18, 2001GBM in Image Processing,Computer Vision, andComputer Graphics Guillermo Sapiro Electrical and Computer Engineering University of Minnesota guille@ece.umn.edu Supported by IPAM, NSF, ONR

2. GBM: Moving Curves

3. Basic curve evolution • Planar curve: • General flow: • General geometric flow:

4. Mathematical morphology • Classical theory, based on Minkowsky addition. • The old and (probably wrong) way of doing geometric image analysis. • Has very important lessons to learn!!!! • Basic definitions: • A: Image in Euclidean space (R or Z) • B: Structuring element (symmetric) • Nothing else than Minkowsky addition

5. Mathematical morphology: Definitions

6. Mathematical morphology: Is it good or bad? • Advantages: • Nice mathematical properties (set theory) • Extension to Lattices • Disadvantages: • Discrete Minkowsky addition does not look good, has to be replaced by better ways of computing “discrete distances.” • Major important concept: Level-sets • Commutes with thresholding (level-sets): Do binary on each level sets or do gray-level on all the image => same result • It is in certain sense a particular case of curve evolution (before the lattices part)

7. Mathematical morphology via curve evolution Huygens principle

8. Mathematical morphology via curve evolution (cont.) • General velocity: • Examples: • Nothing else than changing the metric (distance). • Can be explained also based on dynamic programming and time of arrival • See Sapiro et al., Brocket-Maragos, Alvarez et al., Evans, Falcone

9. Euclidean invariant parametrization Affine invariant parametrization Planar differential geometry ¶ C ( s ) = 1 ¶ s < C , C > = 0 s ss C ^ C s ss k : = C ss

10. Curvature constant for circles or straight lines (=0) Curvature defines curve up to Euclidean motion At least 4 points with dk/ds=0 Defined for all curves Curvature constant for ellipses (>0), hyperbolas (<0), and parabolas (=0) Curvature defines curve up to affine motion At least 6 points with dk/ds=0 Defined only for convex curves: segment at inflection points Planar differential geometry (cont.)

11. X C(s) Planar differential geometry (cont.) X C(s) Distance has a local extrema iff X is on the normal

12. 3D Differential geometry • Remember mean and Gaussian curvatures? • Each regular surface has two principal curvatures. The average is the mean curvature, the product the Gaussian. These are also related to the tangential map, etc, etc. See DoCarmo for details.

13. Riemannian geometry, Lie theory • What about other non-Euclidean metrics? • What about invariants to other (Lie) groups, e.g., projective? • What about differential invariants? Semi-differential invariants? Are there any general theories?

14. Smoothing by classical heat flow • Linear • Equivalent to Gaussian filtering • Unique linear scale-space • Non geometric • Shrinks the shape • Implementation problems

15. Invariant shape deformations • Formulate shape deformations • Geometric • Invariant to camera transform • The “best” possible • Change only the desired features • Motivation: • Mathematics: • From static differential geometry to dynamic • Beautiful • Computer vision and image processing: • Invariant shape segmentation and analysis • Image processing via image deformations • Robotics: • Motion planning • Accurate geometric object detection and tracking • Robot manipulation and grasping

16. Basic planar differential geometry • For every Lie group we will consider, exists and invariant parametrization s, the group arc-length • For every such a group exists and invariant signature, the group curvature, k Low curvature High curvature Negative curvature

17. What and why invariant Camera motion Deformation • Camera/object movement in the space • Transformations description (for “flat” objects): • Euclidean • Motion parallel to the camera and planar projection • Affine • Planar projection • Projective

18. Euclidean geometric heat flow • Use the Euclidean arc-length: • The deformation: • Smoothly deforms to a circle (Gage-Hamilton, Grayson) • Geometric smoothing • Reduces length as fast as possible

19. Affine geometric heat flow (Sapiro-Tannenbaum) • Use the affine arc-length: • The flow:

20. Affine geometric heat flow (cont.) • Geometric smoothing (preserving area if desired) • Total curvature decreases • Maxima of curvature decreases • Number of inflections decreases • Smoothly deforms a shape into an ellipse • Decreases area as fast as possible (in an affine form) • Existence also for non-smooth curves • Viscosity framework (Alvarez-Guichard-Morel-Lions) • Polygons (Angenent-Sapiro-Tannenbaum) • Applications: • Curvature computation for shape recognition: reduce noise (Morel et al.) • Simplify curvature computation (Faugeras ‘95) • Object recognition for robot manipulation (Cipolla ‘95)

21. General invariant flows • Theorem: For every sub-group of the projective group the most general invariant curve deformation has the form • Theorem: In general dimensions, the most general invariant flow is given by • u: graph locally representing the surface • g: invariant metric • E(g): variational derivative of g • See Olver et al., Alvarez et al., Caselles-Sbert

22. General Geometric Framework For Object Segmentation

23. Introduction • Goal: Object detection • Approach: Curve/surface deformation • Geometry dependent regularization • Image dependent velocity • Characteristics: • Unifies previously considered independent approaches • Relates segmentation with anisotropic diffusion • General: • Any topology • Any type of image data • Any dimension • Holds formal results

24. Notation • Deforming curve: • Image:

25. Basic active contours approach • Terzopoulos et al., Cohen et al. • Drawbacks: • Too many parameters • Non-geometric • Handling topology changes

26. Geodesic active contours (Caselles-Kimmel-Sapiro) • Generalize image dependent energy • Eliminate high order smoothness term • Equal internal and external energies • Maupertuis and Fermat principles of dynamical systems

27. Geodesic computation • Gradient-descent • Level-sets (Osher-Sethian)

28. Further geometric interpretation • The geodesic flow

29. Model correctness • Theorem: The deformation is independent of the level-sets embedding function • Theorem: There is a unique solution to the flow in the viscosity framework • Theorem: The curve converges to ideal objects when present in the image • Related work: • Kimia-Tannenbaum-Zucker • Caselles et al. • Malladi-Sethian-Vemuri • Kichenassamy at al. • Tek-Kimia, Whitacker • New work: • Chan-Vese • Paragios-Deriche • Yezzi et al. • Faugeras et al.

30. Extensions • Gray-level values • ds -length element (geodesics) • Ordinary edge detector (gradient) • Surfaces • ds -area element (minimal surfaces) • 3D edge detector • Vector-valued images (color, texture, medical, etc) • ds - length element • Vector-valued edge detector (vector geodesics) • Eigenvalues of the first fundamental form in Riemannian space • Invariant detection (affine area geodesics) • ds - affine length element (area related) • Affine invariant edge detector • Affine norm for “gradient descent”

31. Why color edges?

32. Notation • Image • Texture: Gabor decomposition

33. Color edge computation • Given a metric (Euclidean) • Compute first fundamental form • Compute eigenvectors and eigenvalues • Edge: maximal eigenvalue and its eigenvector • Basic properties: • Eigenvectors are orthonormal • Minimal eigenvalue is not zero

34. GBM: Moving Images

35. Isotropic smoothing Anisotropic smoothing Anisotropic diffusion Isotropic vs. Anisotropic Smoothing

36. Isotropic diffusion (Koenderink, Witkin) • All “equivalent:” • Gaussian filtering of the image • Heat flow • Minimize the L2 norm

37. Isotropic diffusion: Good things • Gaussian filtering if and only if • Linear • Shift-invariant • No creation of zero crossings • Gaussian filtering if and only if • Linear • Shift-invariant • Semi-group property • Scale-invariant (dimensionless) • Unique linear filter that defines a scale-space: Do not creates information at coarser scales • Where everything started (Koenderink, Witkin)

38. Isotropic diffusion: Bad things and possible solutions • Non-geometric • Problems with implementations • Who said linear? Replace heat flow by “parabolic” PDE’s (Hummel’s original idea) • Why parabolic? Because of the maximum principle.

39. Perona-Malik anisotropic diffusion • Replace the L2 by a different norm (e.g., L1, Rudin-Osher-Fatemi; Lorentzian, Black et. al.; etc)

40. Selection of “stopping term” h • How do we select h? • h=x*x => L2 => linear => Isotropic diffusion • h=x => L1 (Rudin-Osher-Fatemi)

41. L Robust anisotropic diffusion • General theory for selection “h”, based on the theory of influence functions in robust statistics • Edges should be considered outliers: At certain point, h’, the influence, should be zero.

42. Directional diffusion • Diffuse in the direction perpendicular to the edges (Avarez et al.)

43. From active contours to anisotropic diffusion • Replace embedding function in level-sets formulation by image itself Shock-filters (Osher-Rudin) Anisotropic diffusion (Alvarez et al.)

44. Relation with Perona-Malik anisotropic diffusion Total variation, Robust estimation Anisotropic diffusion

45. Concluding remarks Terzopoulos snakes Geometric interpretation Dynamical systems Level-sets Terms elimination Curve evolution active contours Geodesic active contours Use image as embedding Geometric diffusion Self-snakes Mumford-Shah Add Shock-filters Divide by gradient Perona-Malik flow Variational interpretation Total Variation Robust Estimation

46. Anisotropic Diffusion of the Posterior

47. ADP in MRI Review: MAP Estimation • 3 classes: sulcus, gray matter, white matter • Prior probability: Pr(class=C) • Posterior probability: Pr(class=C | data) • MAP: Choose class C that maximizes posterior: • C* = arg max Pr(class=C | data) • C • Bayes’ Rule: • Pr(class=C | data) = Pr(data | class=C).Pr(class=C) • Pr(data) • What is our prior, Pr(class=C)?

48. ADP: Common Techniques MAP Estimation: Uniform Prior Classification

49. ADP: Results Anisotropic smoothing of posterior (Teo-Sapiro-Wandell) Smoothed Posterior Classification Posterior

50. ADP: Comparisons Comparison with manual segmentation Automatic (2 min) MR Image Manual (18 hours)