Image Segmentation – Detection of Discontinuities

1. Image Segmentation – Detection of Discontinuities

2. Image Segmentation Outline • Detection of gray level discontinuities • Point detection • Line detection • Edge detection • Gradient operators • LoG : Laplacian of Gaussian • Edge linking and boundary detection • Hough transform • Thresholding • Region-based segmentation • Segmentation by Morphological watersheds • The use of motion in segmentation

3. Image Segmentation Revisit - Goals of image processing • Image improvement – low level IP • Improvement of pictorial information for human interpretation (Improving the visual appearance of images to a human viewer ) • Image analysis – high level IP • Processing of scene data for autonomous machine perception (Preparing images for measurement of the features and structures present )

4. Image Segmentation Image analysis – HLIP • Extracting information from an image • Step 1 : segment the image ---> objects or regions • Step 2 : describe and represent the segmented regions in a form suitable for computer processing • Step 3 : image recognition and interpretation

5. Image Segmentation Image analysis – HLIP (cont’)

6. Image Segmentation What is segmentation? • Definition • Subdivides an image into its constituent regions or objects • 从图像中提取出所需的语义对象 • 将图像划分成若干有一定涵义的区域 • Heavily rely on one of two properties of intensity values: • Discontinuity ---- Partition based on abrupt changes in intensity, e.g. edges in an image • point / line / edge / corner detection • Similarity ---- Partition based on intensity similarity, e.g. thresholding • thresholding • region growing / splitting / merging

7. Image Segmentation Segmentation We’ll discuss both approaches. Starting with the first one.

8. Image Segmentation What Should Good Image segmentation be? • Region interiors • Simple • Without many small holes • Adjacent regions • Should have significantly different values • Boundaries • Simple • Not ragged • Spatially accurate Achieving all these desired properties is difficult. There is no theory of image segmentation. Image segmentation techniques are basically ad hoc. R.M. Haralick and L.G. Shapiro, Image Segmentation Techniques. Computer Vision, Graphics, and Image Processing, 1985, 29: 100-132

9. Introduction Image Segmentation • We want to extract 3 basic types of gray-level discontinuity: • Points • Lines • Edges • What have we learnt in previous lectures to help us in this process? • CONVOLUTION! Grayscale image Mask coefficient

10. Image Segmentation - 1 Point detection

11. Image Segmentation - 1 Line detection • Masks for lines of different directions: • Respond more strongly to lines of one pixel thick of the designated direction. • High or low pass filters?

12. Line detection (cont’) Image Segmentation - 1 • If interested in lines of any directions, run all 4 masks and select the highest response. • If interested only in lines of a specific direction (e.g. vertical), use only the mask associated with that direction. • Threshold the output. • The strongest responses for lines one pixel thick, and correspond closest to the direction defined by the mask.

13. Image Segmentation - 1 Line detection (cont’)

14. Edge Detection Image Segmentation - 1 • Far more practical than line detection. • We’ll discuss approaches based on • 1st-order digital derivative • 2nd-order digital derivative

15. What is an edge? Image Segmentation - 1 • A set of connected pixels that lie on the boundary between two regions. • Local concept Edge point Any point could be an edge point Ideal/step edge Ramp-like (in real life) edge

16. 1st Derivative Image Segmentation - 1 • Positive at the points of transition into and out of the ramp, moving from left to right along the profile • Constant for points in the ramp • Zero in areas of constant gray Level • Magnitude for presence of an edge at a point in an image (i.e. if a point is on a ramp)

17. 2nd Derivative Image Segmentation - 1 • Positive at the transition associated with the dark side of the edge • Negative at the transition associated with the bright side of the edge • Zero elsewhere • Producing 2 values for every edge in an image (an undesirable feature). • Center of a thick edge is located at the zero crossing Zero crossing

18. Image Segmentation - 1 Edge detection (cont’)

19. Image Segmentation - 1 Edge detection (cont’) ---- Effect of Noise (a) • Corrupted by Random Gaussian noise of mean 0 and standard deviation of • (a) 0 • (b) 0.1 • (c) 1.0 • (d) 10.0 • Conclusion??? • ----Sensitivity of derivative to noise (b) (c) (d) grayscale 1st derivative 2nd derivative

20. Image Segmentation - 1 Edge detection (cont’) • The difference between edge and boundary • Edge detection steps • Compute the local derivative • Magnitude of the 1st derivative can be used to detect the presence of an edge • The sign of the 2nd derivative can be used to determine whether an edge pixel lies on the dark or light side of an image • Zero crossing of the 2nd derivative is at the midpoint of a transition in gray level, which provides a powerful approach for locating the edge. Edge: a “local” concept Boundary: a more global idea

21. Some Terminology Image Segmentation - 1 • An edge element is associated with 2 components: • magnitude of the gradient, and • and edge direction , rotated with respect to the gradient direction by -90 deg.

22. Image Segmentation - 1 Gradient operators (1st Derivative) • Use gradient for image differentiation • The gradient of an image f(x,y) at location (x,y) is defined as • Some properties about this gradient vector • It points in the direction of maximum rate of change of image at (x,y) • Magnitude • angle

23. Finite Gradient - Approximation Image Segmentation - 1 • Central differences (not usually used because they neglect the impact of the pixel (x,y) itself) • h is a small integer, usually 1. • h should be chosen small enough to provide a good approximation to the derivative, but large enough to neglect unimportant changes in the image function.

24. Image Segmentation - 1 Edge operator

25. Image Segmentation - 1 Sobel edge operator • Advantages : providing both differencing and a smooth effect and slightly superior noise reduction characteristics. Better noise-suppression

26. Image Segmentation - 1 Edge detection example

27. Image Segmentation - 1 Edge detection example (cont’)

28. Image Segmentation - 1 Edge detection example (cont’)

29. 2nd Derivative: Laplacian Operator Image Segmentation - 1 • Review: The Laplacian operator ( ) is a very popular operator approximating the second derivative which gives the gradient magnitude only. • We discussed this operator in spatial filtering • It is isotropic 4-neighborhood 8-neighborhood

30. Issues with Laplacian Image Segmentation - 1 • Problems: • Unacceptably sensitive to noise • Magnitude of Laplacian results in double edges • Does not provide gradient, so can’t detect edge direction • Fixes: • Smoothing • Using zero-crossing property for edge location • Not for gradient direction, but for establishing whether a pixel is on the dark or light side of and edge

31. Smoothing for Laplacian Image Segmentation - 1 • Our goal is to get a second derivative of a smoothed 2D function • We have seen that the Laplacian operator gives the second derivative, and is non-directional (isotropic). • Consider then the Laplacian of an image smoothed by a Gaussian. • This operator is abbreviated as LoG, from Laplacian of Gaussian: • The order of differentiation and convolution can be interchanged due to linearity of the operations:

32. Laplacian of Gaussian (LoG) Image Segmentation - 1 • Let’s make the substitution where r measures distance from the origin. • Now we have a 1D Gaussian to deal with • Laplacian of Gaussian becomes Normalize the sum of the mask elements to 0

33. Image Segmentation - 1 Marr and hildreth’s approach • Smooth the image to reduce noise • Then calculate the 2nd derivative • Finally, find the zero-crossing • LoG (Laplacian of Gaussian, Mexican hat function)

34. Laplacian of Gaussian (LoG) Image Segmentation - 1 • Because of its shape, the LoG operator is commonly called a Mexican hat.

35. Image Segmentation - 1 Gradient operators – examples Zero-Crossing: Advantages: noise reduction capability; edges are thinner. Drawbacks: edges form numerous closed loops (spaghetti effect); computation complex.

36. Illustration Image Segmentation - 1 • One simple method for approximating zero-crossing: • Setting all + values to white, - values to black. • Scanning the thresholded image and noting the transition between black and white. Closed loops (spaghetti effect) original LoG thresholded zero crossing

37. Image Segmentation - 1 discussion • Edge detection by gradient operations tends to work well when • Images have sharp intensity transitions • Relative low noise • Zero-crossing approach work well when • Edges are blurry • High noise content • Provide reliable edge detection

38. Summary Image Segmentation - 1 • Point detection • Line detection • Edge detection based on • 1st derivative • Provides gradient information • 2nd derivative using zero-crossing • Indicates dark/bright side of an edge

39. Image Segmentation - 1 References • J. Canny, A computational approach for edge detection. IEEE Trans. on PAMI, 1986, 8(6): 679~698 • J. Shen, An optimal linear operator for step edge detection. CVGIP: Graphical, Models and Image Processing, 1992, 54(2): 112-133