On Constrained Optimization Approach To Object Segmentation

1. On Constrained Optimization Approach To Object Segmentation Chia Han, Xun Wang, Feng Gao, Zhigang Peng, Xiaokun Li, Lei He, William Wee Artificial Intelligence and Computer Vision Lab Department of Electrical & Computer Engineering and Computer Science University of Cincinnati Cincinnati, OH 45221-0030 USA

2. 1. Introduction Image understanding starts with image segmentation: extracting contour of region/object of interest. Image segmentation is built on Deformable Contour Methods (DCMs), constrained based contour energy minimization method. Integrated object segmentation and recognition: two iteratively alternating procedures of contour evolution and shape matching Model-based recognition, matching; Model used is from the model set; similarity measure

3. The limitations of model-based object segmentation methods: • Require that a shape model representing the shape of target object be given. • Shape model construction needs extensive training and is often unavailable in many situations.

4. 2. Problem Formulation • Contour searching and extraction from an image using a deformable contour strategy • Formulated as a constrained optimization problem as follows: Consider an open domain subset of image points (x,y) and an image intensity function I(x,y). Our problem is to find a close contour Cenclosing region c(t) at time t, such that the contour energy E (C) is the minimum under the constraint D(x,y) Tv for all (x, y) in the c(t) region, The D(x,y) in the constraint includes any contour interior brightness characterization features, such as smoothness, texture, and other structural features. Contour shape matching and fittings specifications can be added to the contour energy.

5. where s is the arc length, is the image brightness at and is the gradient of with on ; is a positive threshold. can be any function characterizing the interior of expected target contour.

6. As a special case, let where is the absolute value of the gradient of I(x, y) smoothed by a Gaussian filter N(0, 1) and I0 is the average intensity over target boundary interior .

7. 3. Contour Optimization Approaches • They provide global and local optimal solutions using the level set numerical implementations for extracting target contours in images. • Three constrained optimization contour extraction approaches are: the mean field annealing method, variational method, and evolution strategy.

8. Different means for guiding curve evolution • Mean field annealing. Avoid local minima through annealing process. • Variational method. A derivative-based approach to derive curve evolution solution – interior constraint acts as “balloon force” • Evolution strategy. Mimic natural evolution process with mutation and selection.

9. Key features for each approach: • the curve evolution formulation • the interior characterization and the constraint • analysis, and parameter estimation Original parametric deformable models -- Use probabilistic deformable model to represent and segment objects with irregularity and diversity. Level set based deformable models -- Embed probabilistic deformable model in the contour evolution formulation.

10. 4. Model-based shape matching • Two shape representations: Thin plate spline and implicit polynomial shape representations. • Construction of shape model set : • Select an ideal object example as the shape model to simplify the process of shape model constructions. • Define a shape similarity measure between the shape model and a given contour. The shape similarity measure is to be used for the later objectsegmentation.

11. Model Set Based Segmentation (& recognition) • Object segmentation and recognition is composed of two components: • Shape model set construction • Shape model selection (object recognition) and object segmentation based on the selected shape model. • Model based object segmentation method guided by shape model is applied for each model. • We select the resulting contour with the smallest distance to one of the shape models as the final result and the shape model to which the target object belongs.

12. Shape Model and a Given Contour Matching Using Thin Plate Spline Method • A shape matching distance measure is established between a close contour and a model close contour through a mapping for both continuous and discrete formulations. • The coordinate transformations are derived to minimize the shape matching distance by using thin plate spline method. • Model based object segmentation method is applied for • each shape model in the model set.

13. Model based object segmentation • Model based object segmentation can be formulated as find a contour , such that, is minimized subject to the region constraint if (x, y)c(t)

14. Model based object segmentation (I) • Using Lagrange approach, we have where is a constant multiplier.

15. Model based object segmentation (II) • To minimize Eq. (1), our first step (Step A) is to deform while keeping and unchanged. • The curve evolution formula corresponding is

16. Model based object segmentation (III) • is the shortest distance from point to a point in .

17. Model based object segmentation (IV) • Our next step is to keep C(q,t) constant, and then to minimize Eq. (1) by adjusting and , or equivalently minimize,

18. Model based object segmentation (V) • Minimizing Eq. (3) is to solve, (4) where , ,

19. Model based object segmentation (VI) The ith row of P is . K is a matrix composed of components, , and .

20. Model based object segmentation (VII) is an identity matrix. We solve Eq. (4) using a regular linear algebra method. The resulting , , , and are then used to form new and transformations employed in Eq. (2) to guide contour deformations.

21. 5. Algorithm Description • Select shape model from the model set. Select an initial small seed region (usually with size of 3 by 3 pixels) in the interior of the object. • With being the boundary of at , compute the mean brightness of . Set . • Obtain of N equally spaced point sequence set of in a clockwise manner.

22. Algorithm Description (II) • Evolve according to Eq. (2) using the level set narrow band algorithm for l iterations. If the maximum point velocity of is less than a velocity threshold value h, or the total number of iterations is reached, then stop the algorithm, compute the bending energy of the resulting contour and go to Step (f). If the total number of contour points is less than N, repeat step (d).

23. Algorithm Description (III) • Link contour . Form an approximately equally spaced point sequence set of in a clockwise manner. Select the best match point subsets in between and using the shape context method described in Section 3.1 of [2].

24. Algorithm Description (IV) • Save as the resulting contour using shape model . Increase g by 1. If , select the resulting contour with lowest bending energy as the segmentation results and the associated model class the recognition results.

25. 6. Some experimental results

30. Apple II Banana Cucumber Apple I Pear (a)

33. Conclusion A general contour constrained optimization approach is formulated to extract a target contour in an image for various applications. These different approaches of mean field annealing method, variational method, and evolutionary strategy are derived to provide global and local optimal solutions using level set numerical implementations. Impositions of constraints to characterize the contour interior features are employed on different specific applications. Contour shape models using either the thin plate spline matching method or the implicit polynomial representation method can be added into the optimization process to further improve contour extraction results. A set of illustrative examples for the applications ofthe approaches on biomedical images are presented.

34. 7. Experiments Apple II Cucumber Banana Apple I Pear

35. Results

36. Results (II)

37. Results (III)

38. Results (IV)

39. Results (V)

40. Results (VI)

41. Results (VII)

42. Results (VIII)

43. Results (IX)