1 / 108

Pixel-Based Processing

Explore pixel-based image processing to detect and classify fruit, focusing on finding the banana stem in original images. Learn the steps involved, including binarization, region detection, and property-based classification. Enhance your understanding with the outlined methodologies and types of image transformations discussed by Stan Birchfield from Clemson University's ECE 847 course.

adamsd
Download Presentation

Pixel-Based Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pixel-Based Processing ECE 847:Digital Image Processing Stan Birchfield Clemson University

  2. Challenge ? detect objects, classify fruit, find banana stem original image Steps: binarize / threshold clean up binary image find regions corresponding to foreground objects compute properties of regions classify regions based on properties S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  3. Outline • Pixel-based image transformations(thresholding, …) • Removing noise from binary image(morphological operators) • Labeling regions(floodfill, connected components) • Computing distance in an image(chamfer distance, …) • Region properties(area and boundary properties) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  4. What is a digital image? S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  5. Accessing pixels in an image Timing Test(640 x 480 image, get/set each pixel using 2.8 GHz P4) C / C++ • Method 1: 1D arrayval = p[ y * width + x]; • Method 2: 2D arrayval = p[y][x]; • Method 3: pointersval = *p++; 6.6 ms 5.6 ms 0.5 ms (even faster with SIMD operations) • Matlab • Method 1: 2D arrayval = im(y,x); • Method 2: parallelizeim = im + constant; 21.4 ms 2.5 ms S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  6. Outline • Pixel-based image transformations(thresholding, …) • Removing noise from binary image(morphological operators) • Labeling regions(floodfill, connected components) • Computing distance in an image(chamfer distance, …) • Region properties(area and boundary properties) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  7. Types of image transformations • Graylevel transformsI’(x,y)  f( I(x,y) )(arithmetic, logical, thresholding, histogram equalization, …) • Geometric transformsI’(x,y)  f( I(x’,y’) )(flip, flop, rotate, scale, …) • Area-based transformsI’(x,y)  f( I(x,y), I(x+1,y+1), … )(morphological operators, convolution) • Global transformsI’(x,y)  f( I(x’,y’), x’,y’ )(Fourier transform, wavelet transform) A S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  8. Graylevel transforms identity inverse contrast threshold enhancement S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  9. Geometric transforms S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  10. Histograms image histogram Throws away all spatial information! S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  11. Histogram equalization S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  12. Running sum Step 1: Initialize 1D array inthist[256] ... Step 2: Compute running sum sum[0] = 0 for g = 1 to 255, sum[g] = sum[g-1] + hist[g] Example: [ 6 3 9 7 4 2]  [6 9 18 25 29 31] S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  13. Histogram equalization pdf Steps: • Compute normalized histogram (PDF)int hist[256] = 0…0float norm[256]for x, hist[ I(x) ]++for g = 0 to 255, norm[g] = hist[g] / npixels • Compute cumulative histogram (CDF)cum[0] = norm[0]for g = 1 to 255, cum[g] = cum[g-1] + norm[g] • Transformfor x, I(x) = 255*cum[ I(x) ] equal area cdf equal spacing new gray level old gray level running sum S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  14. Histogram equalization * desired PDF p PDF p’ equal area equal area a’+d a a’ new gray level k’ CDF By definition, d { q d { a’ equal spacing new gray level k’ old gray level k where p is the PDF and q is the CDF S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  15. Histogram equalization * desired PDF p PDF equal area p’ 1/(L-1) a’+d a b a’ L-1 L-1 old gray level k new gray level k’ scaled CDF L-1 q·(L-1) d { a’ new gray level k’ L-1 old gray level k S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  16. Histogram equalization * desired PDF p PDF equal area p’ 1/(L-1) a1’+d a2’+d a1 b1 a2 b2 a1’ a2’ L-1 L-1 new gray level k’ scaled CDF L-1 d { q·(L-1) a2’ d { a1’ new gray level k’ L-1 old gray level k S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  17. Integral image • Integral image is a 2D running sum • S(x,y) = SS I(x,y) • To compute, S(x,y) = I(x,y) - S(x-1,y-1) + S(x-1,y) + S(x,y-1) • To use, V(l,t,r,b) = S(l,t) + S(r,b) - S(l,b) - S(r,t)Returns sum of values inside rectangle • Note: Sum of values in any rectangle can be computed in constant time! S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  18. Thresholding Valley separates light from dark How to find it? S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  19. A simple thresholding algorithm Repeat until convergence: T  ½ (m1 + m2) T. Ridler and S. Calvard, Picture Thresholding Using an Iterative Selection Method, IEEE Transactions on Systems, Man and Cybernetics, 8(8):630--632, August 1978. S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  20. Ridler-Calvard algorithm S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  21. Otsu’s method • Threshold t splits image into two groups • Calculate within-group variance of each group • Search over all t to minimize total within-group variance; moment calculations make search efficient percentage of pixelsin first group variance of pixelsin first group S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  22. Otsu’s method S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  23. Local Entropy Thresholding (LET) * • Compute co-occurrence matrix • Threshold s divides matrix into four quadrants • Pick s that maximizesHBB(s)+HFF(s) • Uses spatial information S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  24. Otsu vs. LET * image histogram Otsu LET S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  25. Adaptive thresholding background image histogram from Gonzalez and Woods, p. 597 S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  26. Double thresholding (hysteresis) graylevel threshold too high: misses part of object object 2 object 1 threshold too low: captures noise noise pixel • Algorithm: • Threshold with high value • Threshold with low value, retaining only thepixels that are contiguous with existing ones S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  27. Double thresholding example 1. Use high threshold to get seed pixel low threshold high threshold 3. Repeat 2. Floodfill using seed pixel S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  28. Double thresholding example image low threshold retains some background combined using floodfill on low threshold with seeds from high threshold high threshold removes some foreground S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  29. Double thresholding We will explain Floodfill later S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  30. Outline • Pixel-based image transformations(thresholding, …) • Removing noise from binary image(morphological operators) • Labeling regions(floodfill, connected components) • Computing distance in an image(chamfer distance, …) • Region properties(area and boundary properties) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  31. Binary image as a set = S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  32. Set operations Fundamental operators Other operators DeMorgan’s Laws S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  33. Minkowski operators Minkowski addition: Minkowski subtraction: S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  34. Minkowski addition Add each element of B to each element of A1 S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  35. Minkowski subtraction Intersection of translating A2 to every point in B Two examples: S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  36. Minkowski addition: “Center-out” approach (“center-out” is more intuitive) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  37. Minkowski subtraction:“Center-out” approach (“center-out” is more intuitive) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  38. Morphological operations • morphology – study of form or shape • Mathematical morphology – grew out of research at Fontainebleau in 1964 • We will deal only with binary images • Two fundamental operations:Dilation and Erosion (same as Mink. add.) (same as Mink. sub.after reflection) B is the structuring element (SE) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  39. Dilation • Algorithm (center-out): • Flip B horizontally and vertically(but often B is symmetric, so nothing is changed) • Move B around • Anywhere they intersect, turn center pixel on • Looks like nonlinear convolution S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  40. Erosion • Algorithm (center-out): • Move B around • If there is not complete overlap, turn center pixel off Duality b/w dilation and erosion: S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  41. Common structuring elements • Usually SE is either B4 or B8 • Greatly simplifies algorithm(and erosion is same as Min. sub.) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  42. Dilation example Let B be 3x3 with all 1s.for y for x if ( in(x-1, y) || in(x-1, y-1) || in(x-1, y+1) ... || in(x+1, y+1)) out(x, y) = 1 else out(x, y) = 0 B A S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  43. Erosion example Let B be 3x3 with all 1s.for y for x if ( in(x-1, y) && in(x-1, y-1) && in(x-1, y+1) ... && in(x+1, y+1)) out(x, y) = 1 else out(x, y) = 0 B A S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  44. Why does dilation require flipping the structuring element? Simple example: A B Without flipping With flipping   Result is flipped! Fixed! S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  45. Closing and Opening • Dilation fills gaps. Erosion removes noise • Combine them: • First dilate, then erode (closing) • First erode, then dilate (opening) • Closing/opening retain object size • Repeated applications do nothing • Duality between closing and opening S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  46. Closing and Opening Closing: dilate, then erode Opening: erode, then dilate S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  47. Morphology Example open then close erode dilate close open thresholded difference S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  48. Accessing out-of-bounds pixels • If kernel is near boundary of image, some pixels of kernel will be out of bounds (OOB) • What to do? • Do not use OOB pixels (change kernel size near boundary) • Do not compute output for pixels near boundary • Extend image function: • zero pad – OOB pixels have value of zero • replicate – OOB pixels have value of closest pixel • reflect – pixel values are extended by mirror reflection at border • wrap – pixel values are extended by repeating • extrapolate – pixel values are extended by extrapolating function near boundary • No solution is best • Problem also occurs with convolution S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  49. Outline • Pixel-based image transformations(thresholding, …) • Removing noise from binary image(morphological operators) • Labeling regions(floodfill, connected components) • Computing distance in an image(chamfer distance, …) • Region properties(area and boundary properties) S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

  50. Binary images • Each pixel is ON or OFF • We will say that foreground pixels are ON, background pixels are OFF • In computer, each pixel has value of 0 or 1. • We adopt convention that 0=OFF, 1=ON. • Usually 0=black and 1=white, but sometimes display is reversed S. Birchfield, Clemson Univ., ECE 847, http://www.ces.clemson.edu/~stb/ece847

More Related