Introduction to Computer Vision

1. Lecture 6 Roger S. Gaborski Introduction to Computer Vision Roger S. Gaborski

2. Question 1 %First Solution I = imread('OrangeFlower.jpg'); %Must have single quotes, -1 point figure, imshow(I), title(’Orange Flowerl') Id = im2double(I); %Not double(I) figure, imshow(Id(:,:,2)) greenData = Id(:,:,2); %Extract 2nd plane of data - the green plane im= greenData(7:9,4:6); %Second Solution I = imread('OrangeFlower.jpg'); figure, imshow(I), title(’Orange Flower') Id = im2double(I); figure, imshow(Id(:,:,3)) blueData = Id(:,:,3); %Extract 3nd plane of data - the blue plane im= blueData(3:5,7:8) %%%%%%%%%% %Notes: Cannot contain loops, loops, No credit %Otherwise, -1 point per line (up to -5 points) Roger S. Gaborski

3. Question 2 im= [ .76, .7725; .76, .7725] %NOTE, I realize the actual output is a 3x2, not a 2x2 A = mat2gray(im) A = 0 1 0 1 im = [ .7725, .7725, .5; .7725, .7725, .5;.7725, .7725, .5] A = mat2gray(im) A = 1 1 0 1 1 0 1 1 0 Roger S. Gaborski

4. Question 3 Incorrect curve: -5 points No x axis label: -1 No y-axis label -1 • image2 = imadjust(image1, [.30,.90],[.15,.50]); 0 .15 .5 1.0 Output Gray Level Value 0 .30 .50 .90 1.0 Input Gray Level Value Roger S. Gaborski

5. Question 3 Incorrect curve: -5 points No x axis label: -1 No y-axis label -1 • image2 = imadjust(image1, [.30,.90],[.5,.10]); 0 .10 .5 1.0 Output Gray Level Value 0 .30 .50 .90 1.0 Input Gray Level Value Roger S. Gaborski

6. Spatial Filtering • Neighborhood processing • Define center point (x, y) • Perform operations involving only pixels in the neighborhood • Result of operation is response to process at that point • Moving the pixel results in a new neighborhood • Repeat process for every point in the image Roger S. Gaborski

7. Linear and Nonlinear Spatial Filtering • Linear operation • Multiply each pixel in the neighborhood by the corresponding coefficient and sum the results to get the response for each point (x, y) • If neighborhood is m x n , then mn coefficients are required • Coefficients are arranged in a matrix, called • filter / filter mask / kernel / template • Mask sizes are typically odd numbers (3x3, 5x5, etc.) Roger S. Gaborski

8. Image origin y Kernel coefficients mask x Image region under mask Roger S. Gaborski

9. Correlation and Convolution • Correlation • Place mask w on the image array f as previously described • Convolution • First rotate mask w by 180 degrees • Place rotated mask on image as described previously • Convolution = 180 degree rotation + correlation Roger S. Gaborski

10. Example: 1D Correlation • Assume w and f are one dimensional • Origin of f is its left most point • Place w so that its right most point coincides with the origin of f • Pad f with 0s so that there are corresponding f points for each w point (also pad end with 0s) • Multiply corresponding points and sum • In this case (example on next page) result is 0 • Move w to the right one value, repeat process • Continue process for whole length of f Roger S. Gaborski

11. Reminder • ‘full’ is the result we obtain from the operations on the previous slide. If instead of aligning the left most element of f with the right most element of w we aligned the center element of w with the left most value of f we would obtain the ‘same’ result, same indicating the result is the same length of the original w Roger S. Gaborski

12. Chapter 3 www.prenhall.com/gonzalezwoodseddins Roger S. Gaborski

13. ‘Full’ correlation Roger S. Gaborski

14. Roger S. Gaborski

15. Roger S. Gaborski

16. ‘Same’ correlation etc. Roger S. Gaborski

17. Example - Convolution • Convolution is the same procedure, but the filter is first rotated 180 degrees. • Convolution = 180 degree rotation + correlation • If the filter is symmetric, correlation and convolution results are the same Roger S. Gaborski

18. Chapter 3 www.prenhall.com/gonzalezwoodseddins This can be simply extend to images Roger S. Gaborski

19. Linear Filtering in MATLAB g = imfilter(f, w, filtering mode, boundary, size) • filters the imput image f with the filter mask w. • f is input image. It can be of any class (logical/numeric) and dimension. • g is output image • filter mode: - 'corr' : correlation, and default mode - 'conv' : convolution Roger S. Gaborski

20. Parameters • g = imfilter(f, w, filtering mode, boundary, size) Boundary options - X pad boundary with value X. Default X = 0. - 'symmetric' symmetric padding - 'replicate' replicate padding - 'circular' circular padding Size options - 'same' g is the same size of f (default mode) - 'full' g is full filtered by w, so size of g is increased Roger S. Gaborski

21. MATLAB function for filtering: imfilter • g = imfilter(f, w, ‘replicate’) • Correlation is the default filtering mode. • If filters are pre-rotated 180 degrees, can simply use default(corr) for convolution • If filter is symmetric, doesn’t matter Roger S. Gaborski

22. Chapter 3 www.prenhall.com/gonzalezwoodseddins Roger S. Gaborski

23. Example:Smoothing • w = ones(31); (31x31 filter) • % Normally the coefficients (w) are scaled to sum to one • % In this example only coefficients are not scaled by 312 • % Convolution should result in a blurred result • gd = imfilter(f, w); • % Default mode: correlation filtering • imshow(gd, [ ]); Roger S. Gaborski

24. Chapter 3 www.prenhall.com/gonzalezwoodseddins Input Default padding ‘replicate’ ‘symmetric’ ‘circular’ ‘replicate’, uint8 Roger S. Gaborski

25. Noise Issues • g(x,y) = f(x,y) + n(x,y) • Where • f(x,y) is the original image • n(x,y) is the noise term (see text, page 143 for different types of noise) • How does noise affect the image? • Can we remove the effects of noise?

26. Sources of Noise • Sensor (thermal, other) • Electronic circuitry noise • Transmission noise

27. Standard Deviation • Statistics – analyzing data sets in terms of the relationships between the individual points • Standard Deviation is a measure of the spread of the data [0 8 12 20] [8 9 11 12] • Calculation: average distance from the mean of the data set to a point s = Σi=1n(Xi – X)2 (n -1) • Denominator of n-1 for sample and n for entire population

28. Standard Deviation • For example [0 8 12 20] has s = 8.32 [8 9 11 12] has s = 1.82 [10 10 10 10] has s = 0

29. Variance • Another measure of the spread of the data in a data set • Calculation: s2 = Σi=1n(Xi – X)2 (n -1) Why have both variance and SD to calculate the spread of data? Variance is claimed to be the original statistical measure of spread of data. However it’s unit would be expressed as a square e.g. cm2, which is unrealistic to express heights or other measures. Hence SD as the square root of variance was born.

30. Gaussian Noise

31. Histogram of Noise n(x) = (1 / 22 ) * exp( -(x-u)2 / 2) where  is standard deviation and u the mean

32.  Controls the Width of the Bell-shaped Curve • Integral under curve is 1 Small  Large 

33. 2 D Gaussian

34. Data sig=ones([1 1000]); figure, stem(sig(1:15)),axis([0 15 0 2])

35. Data + Gaussian Noise

36. Data + Gaussian Noise

37. Averaging • Averaging will smooth the signal • smoothData(i) = noisyData(i-1)+noisyData(i)+noisyData(i+1) 3

38. Result of 3 Point Averaging Noisy Input Averaging with 3 Components First two responses due to edge effects (padded with zeros)

40. Image + Noise

41. Data from Row 300 Note: Noise values (amplitude) varies rapidly – high frequency component

42. Can We Recover the Original Image from the Noisy Image? Noisy Input Three Point Averaging

43. Smoothed Noisy Image

44. Simple Averaging Filter • Averaging is effective a low pass filter. High frequency noise fluctuations are ‘blocked’ by the filter. • This can be an issue for fine detail. Fine detail in image will also be smoothed. • Tradeoff between keeping fine details in image and reducing noise.

45. Compare Original and Filtered Images

46. Absolute Value of Original and Filtered Images

47. Thresholded Diff Image Details lost by low pass filtering

48. ‘Smart’ SmoothingorNoise Filtering

49. Nagao-Matsuyama FilterAn Attempt to Keep Fine Detail • Uses 9 5x5 windows (defined on next slide). • Each 5x5 window has a distinct subwindow defined • Calculate the variance for the pixels in each subwindow • Output is the mean of the pixels in the subwindow that has the lowest variance

50. Nagao-Matsuyama Filter Average pixel values under red pixels. Calculate the corresponding variance. Use the filter with the smallest variance.