1 / 96

Shape and Boundaries

Shape and Boundaries. ECE 847: Digital Image Processing. Stan Birchfield Clemson University. Can you recognize these shapes?. Sometimes edge detectors find the boundary pretty well. Sometimes it’s not enough. Active contours (Snakes). User (or higher-level process) initializes contour

leache
Download Presentation

Shape and Boundaries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Shape and Boundaries ECE 847:Digital Image Processing Stan Birchfield Clemson University

  2. Can you recognize these shapes?

  3. Sometimes edge detectors find the boundary pretty well.

  4. Sometimes it’s not enough.

  5. Active contours (Snakes) • User (or higher-level process) initializes contour • Snake deforms and shrink-wraps around object boundary (Diagram courtesy “Snakes, shapes, gradient vector flow”, Xu, Prince)

  6. Active contours (Snakes) • Snake is a parameterized curve: • Snake is a type of deformable contour open curve s=0 s=1 s=1 s=0 closed curve

  7. Snakes • Goal: Match curve (boundary) to image data • Approach: minimize energy functional • Like many vision problems, this is underconstrained  regularization (impose smoothness prior) image data internal (smoothness) user-supplied Snakes are a top-down approach to segmentation

  8. elasticity 1st-order term membrane a controls tension along spline stretching balloon or elastic band stiffness 2nd-order term thin plate b controls rigidity of spline bending thin plate or bending wire Internal energy a and b may vary along curve but are usually constant

  9. Elasticity draws points together  shrinks contour Stiffness minimizes curvature Elasticity (a) and Stiffness (b) Dv v v+Dv

  10. elasticity Ideal curve is point (circle with zero radius) stiffness Ideal curve is circle (no shrinking) Elasticity (a) and Stiffness (b)

  11. Image energy We will use only edges (wline = wterm = 0) Alternatives for edge are Key: function must be monotonic non-increasing

  12. Discrete formulation • Curve represented as n points: • Energy: indices are modulo n for closed curve to handle wraparound Note: We will represent the curve simply as a polygon, with the points being the vertices

  13. How to minimize energy • Two approaches: • Finite element and calculus of variations [Kass, Witkin, and Terzopoulos, IJCV 1988] • Dynamic programming [Amini, Weymouth, and Jain, PAMI 1990] • We will only describe the latter, which is much simpler

  14. Basic framework • Start with initial contour • Allow each point to move to one of 9 positions (3x3 neighborhood) • Algorithm:

  15. Dynamic programming for snakes • Exploit Markov property of internal energy: point is conditionally independent of all other points, given its immediate neighbors • Dynamic programming can exhaustively consider all 9n possibilities with just O(92n) computations

  16. DP Algorithm • Simplifications: • Consider just first-order term (b=0) • Open curve • Construct table (n columns, 9 rows) • Cell (i,j) represents location i for point j • Each cell contains two values: • fij – Cost of best path that ends in point j being in location i • pij – Index of location (0,..,8) for previous point j-1 corresponding to this path • Fill table one column at a time from left to right • Minimum fij in last column indicates best curve

  17. DP (1st order, open curve) • Each cell must consider all 9 possibilities from previous column • First column ignores internal energy: j i

  18. Closed curves • Previous algorithm implements • For closed curves, add term to last column for P(vn-1|v0): • Optimality is lost no matter what m is chosen (could traverse path backwards to find m, but no better than assuming fixed m) • In fact, snake will go haywire near v0 and vn-1 • Solutions: • Run DP 9 times, using different choice of location for v0 each time; select the choice with the minimum cost (optimal, but expensive) • Fold curve back onto itself, so that v1 and vn-1 are both considered in the second column, etc. (optimal, but even more expensive: O(94n) instead of O(92n)) • Keep adding columns to table until path stabilizes (sounds good, but untested) • Recommended solution: Run DP twice (“probably optimal” in practice, but not guaranteed)

  19. Running DP twice • Solution: • Run once • Pick v0, run DP (with an extra term in the last column) • Find min cell of last column, trace p back to find path • Discard everything except the location for the point halfway from endpoints, vn/2 • Run again • Set vn/2 to be the new v0, shifting all other indices down accordingly • Run DP again (with an extra term in the last column) • Find min cell of last column, trace p back to find “probably optimal” path • Now adjust points on snake according to path; this is one iteration • Rationale: The errors due to the open curve assumption should have little effect on points far from the endpoints

  20. Including second-order term • DP will not let us look at future columns: |vi-1-2vi+vi+1|2 • So reformulate: |vi-2vi-1+vi-2|2 • But DP will not let us skip columns, either • So need to increase the state space • Now we have 9x9=81 rows and n columns

  21. Including second-order term • Each cell represents locations for two adjacent points • Each cell has 81 elements in previous column, but only 9 are consistent with the cell | (vi-2,vi-1) | (vi-1,vi) | • Computation is O(93n) must refer to the same location k and j are consistent

  22. DP (second-order) • Simple example: Each point can only move vertically (-1,0,1) only 3 previous cells are consistent j i

  23. Application: Medical imaging Active Contours and their applications – Julien Jomier - Comp 258 Fall 2002

  24. Examples Hands People Active Contours and their applications – Julien Jomier - Comp 258 Fall 2002

  25. More examples Highway Heart Active Contours and their applications – Julien Jomier - Comp 258 Fall 2002

  26. Drawbacks of snakes • Sensitive to initial position • Sensitive to parameters • Small capture range • Fails to detect concavities

  27. Gradient vector flow (GVF) • The GVF field is defined to be a vector field V(x,y) = • Force equation of GVF snake • V(x,y) is defined such that it minimizes the energy functional C. Xu and J. L. Prince , "Snakes, Shape, and Gradient Vector Flow“, IEEE Transactions on Image Processing, 1998. f(x,y) is the edge map of the image. [from Amyn Poonawala]

  28. A look into the vector field components of GVF u(x,y) v(x,y) Note forces also act inside the object boundary!! [from Amyn Poonawala]

  29. GVF Results Traditional snake GVF snake [from Amyn Poonawala]

  30. With GVF, the contour can also be initialized across the boundary of object!! Something not possible with traditional snakes. [from Amyn Poonawala]

  31. Level sets • Embed curve in one higher dimension; curve is given by zero level set of implicit function (i.e., intersection of function with z=0) • Due to J. Sethian and S. Osher 1988www.math.berkeley.edu/~sethian/level_set.html • Fixes several problems of snakes: • Snake intersects itself • Topology changes

  32. Moving contour using level set • Define level set function z = ((x,y,t)=0) • Move the level set function, (x,y,t), so that it rises, falls, expands, etc. • By convention, <0 inside the curve and >0 outside the curve from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  33. Level set evolution As the surfacechanges, sodoes the zero level set from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  34. Curve evolution • 2D: • Curve • Implicit function • Over time: • Curve • Implicit function • Evolution • of curve • of implicit function Simplest implementation:

  35. Advantages Advantages of level sets over snakes: • Curve may change topology and form sharp corners (“weak solutions”) • Discrete grid and finite differences approximate solution • Instrinsic geometric properties are easily determined • normal vector • curvature • Formulation is same for 2D or 3D

  36. Narrow Band Method

  37. Fast Marching Method • J. Sethian, 1996 • Special case that assumes the velocity field, F, never changes sign. That is, contour is either always expanding or always shrinking • Can convert problem to a stationary formulation on a discrete grid where the contour is guaranteed to cross each grid point at most once from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  38. Fast Marching Method • Compute T(x,y) = time at which the contour crosses grid point (x,y) • At any height, T, the surface gives the set of points reached at time T from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  39. Fast Marching Method (i) (ii) (iii) (iv) from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  40. Fast Marching Algorithm • Construct the arrival time surface T(x,y) incrementally: • Build the initial contour • Incrementally add on to the existing surface the part that corresponds to the contour moving with speed F • Builds level set surface by scaffolding the surface patches farther and farther away from the initial contour from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  41. Fast Marching Visualization from http://pages.cs.wisc.edu/~fan/LevelSet/Level_presentation/levelsets.ppt

  42. Mumford-Shah • Mumford-Shah functional captures how well reconstructed function matches original image • Regularization term enforces smoothness, but not at boundaries contour reconstruction imagedomain D. Mumford and J. Shah. Boundary detection by minimizing functionals. CVPR, 1985.

  43. Chan-Vese model • Snakes and level sets have several problems: • Small basin of attraction (edges) • initial contour must surround object (or be completely contained within it • Chan-Vese (2001) propose • segmenting image using model of interior and exterior • level set function is used to represent the curve (or equivalently the binary mask of the interior region) • for simplicity, assume interior and exterior both have approximately constant intensity (but different from each other)

  44. Chan-Vese (cont.) • Goal is to minimize energy functionalwhere • and • m, n, li, and lo are constants • G is contour • w is interior region bounded by G • ci and co are average intensities inside and outside G

  45. Heaviside function: derivative: regularized Heaviside function: derivative: Chan-Vese (cont.) vs. (other functions are possible) Dirac delta function “hard threshold, not differentiable” “soft threshold, differentiable”

  46. Chan-Vese (cont.) • H(f(x,y)) is approximately 1 inside contour, and approximately 0 outside • Therefore,

  47. Chan-Vese (cont.) • Now express the gradient of H(f(x,y)) in terms of the gradient of f(x,y): • Therefore,

  48. Chan-Vese (cont.) • Because we are minimizing an energy functional, you shouldn’t be surprised that we will be using the Euler-Lagrange equation • In preparation, note that • where

  49. Chan-Vese (cont.) • Now let us take some derivatives: • And,

More Related