Chap. 9: Image Segmentation

1. Chap. 9: Image Segmentation Jen-Chang Liu, 2005

2. Motivation • Segmentation subdivides an image into its constituent regions or objects Example: 生物細胞在影像序列中的追蹤

3. Basics of image segmentation • Input: image => Output: attributes • Segmentation of nontrivial image is one of the most difficult tasks in image processing Ex. Separate the sky, clouds, and mountains

4. Two approaches • Based on intensity values discontinuity • Partition an image based on abrupt changes, such as edges • Based on intensity similarity in regions • Partition an image into regions that are similar according to some criteria

5. Outline • Point, line detection • Edge detection • Thresholding • Use of motion in segmentation Pixel discontinuity Region similarity

6. Detection of discontinuity • Three basic types of gray-level discontinuity • Points, lines, and edges • Method: run a mask through the image Input: 3x3 region z1, z2 , … , z9 w1 w2 w3 Output: z5 -> R w4 w6 w5 z1 z2 z3 w9 w7 w8 z4 z5 z6 z9 z7 z8

7. Point detection • Point detection mask • Mask operation over whole image • Threshold: |R|  T thresholding point mask 渦輪葉片 X 光

8. Problem with previous point detection • Assume single-pixel discontinuity • Bigger region of point? • Assume simple background: homogeneous background • Noisy environment ? • Selection of threshold T

9. Hori. R1=8 Line detection 1. Line detection mask 2. Which orientation? Thresholding: |Ri|  |Rj|  T 0 0 0 1 2 1 0 0 0 -45o R4=2 Ex. A thin line: 45o R2=2 Veri. R3=2

10. Input: binary image absolute value after –45o mask |R|  T Example: single-line detection Detect line that are one pixel thick 1 Isolated “line points” 2

11. Outline • Point, line detection • Edge detection • Boundary with arbitrary shape • Thresholding • Use of motion in segmentation

12. Definition of edges? • Intuition of edge: set of connected pixels that line one the boundary between two regions • Model of digital edge

13. Two factors affects acquired edges • Blur模糊化 • Sampling rate, illumination condition zoom • Noise雜訊

14. Method of edge detection: derivatives Edge detection method Magnitude of 1st derivatives  T 1. Sign determines Dark or light region 2. Zero-crossing

15. Methods: 1st derivatives • 1st derivatives 2-D gradient Magnitude = Direction =

16. Hor. Ver. Example: Sobel filter Original: 1200x1600 abs abs +

17. Effects of noise Unwanted Detailed Structure !

18. 2nd 1st Noisy edges Fail to detect

19. Reduce noise: Smoothing Smooth with 5x5 average filter Hor. abs abs + Ver.

20. Review 2nd derivative f(x-1) f(x) f(x)-f(x-1) 1st derivative 2nd derivative f(x+1)-2f(x)+f(x-1)

21. Method: 2nd derivative Edge detection method 2nd derivative mask Magnitude of 1st derivatives  T Laplacian 1. Sign determines Dark or light region 2. Zero-crossing

22. Example: 2nd derivative original Sobel Gaussian Smoothing Laplacian (2nd derivative) Threshold(LoG) Zero-crossing of LoG LoG

23. Outline • Point, line detection • Edge detection • Thresholding • Use of motion in segmentation

24. Foundation of thresholding • Idea: object and background pixels have gray levels grouped into two dominant modes Original image histogram

25. Foundation of thresholding • Input f(x,y), given threshold T

26. Thresholding as a multi-variable function: g(x,y) = T[ f(x,y), x, y, p(x,y) ] Adaptive: Depend on position Local: local property func. Issues of thresholding • Selection of threshold T ? • Complex environment – illumination • Multiple thresholds – more than one object • Global threshold • Local threshold

27. m1 m2 1. Automatic selection of T 1. Select an initial T • Average gray level • Mean of max. and min. gray level G2 G1 2. Segment the image using T T 3. Calculate mean of G1 and G2 T2 4. New threshold: T2 = 0.5(m1 + m2) 5. Repeat steps 2~4 until difference in successive T is small

28. Example: automatically select T Initial: gray level mean 3 iterations T = 125.4 fingerprint

29. 2. Effects of illumination • Recall: f(x,y)=i(x,y) r(x,y) illumination: reflectance: Illumination source scene reflection

30. Example: illumination x Original image Illumination source histogram histogram

31. Example: bad histogram * The gray levels of the object is mixed with background

32. 4. Motivation for adaptive thresholding A single Global threshold histogram

33. Adaptive local thresholding Subdivide image into blocks Q: Improperly segmented subimages !

34. subdivision Iterative subdivision histogram

35. Outline • Point, line detection • Edge detection • Thresholding • Use of motion in segmentation

36. Thresholded difference image 1 if |d(x,y)| > T 0 otherwise Motion as a clue to extract object • Spatial technique Reference image f(x,y,1) next image f(x,y,2) time index

37. Use more than one images in time: eliminate noise … Reference image R(x,y) Image f(x,y,2) Image f(x,y,3) d(x,y)=R(x,y)-f(x,y,t) counter a. if d(x,y) > T positive ADI b. if d(x,y) < -T negative ADI c. if |d(x,y)| > T absolute ADI counter + 1, Accumulative difference image

38. Example: Negative ADI Positive ADI Absolute ADI * Object shape * Location in ref. image