Image Processing Techniques: Filtering, Edge Detection, and more

1. Last Lecture photomatix.com

2. Today • Image Processing: from basic concepts to latest techniques • Filtering • Edge detection • Re-sampling and aliasing • Image Pyramids (Gaussian and Laplacian) • Removing handshake blur from a single image

3. Represented by a matrix Image as a discreet function

4. 10 5 3 4 5 7 1 1 1 7 Some function Local image data Modified image data What is image filtering? • Modify the pixels in an image based on some function of a local neighborhood of the pixels.

5. 0 0 0 0 0.5 0 0 1 0.5 10 5 3 4 7 5 1 1 1 7 Local image data kernel Modified image data Linear functions • Simplest: linear filtering. • Replace each pixel by a linear combination of its neighbors. • The prescription for the linear combination is called the “convolution kernel”.

6. I Convolution

7. coefficient 0 Pixel offset Linear filtering (warm-up slide) ? 1.0 original

8. coefficient 0 Pixel offset Linear filtering (warm-up slide) 1.0 original Filtered (no change)

9. coefficient 0 Pixel offset Linear filtering 1.0 ? original

10. coefficient 0 Pixel offset shift 1.0 original shifted

11. coefficient 0 Pixel offset Linear filtering ? 0.3 original

12. coefficient 0 Pixel offset Blurring 0.3 original Blurred (filter applied in both dimensions).

13. 8 coefficient 0 original Pixel offset Blur Examples 2.4 impulse 0.3 filtered

14. 8 coefficient coefficient 0 0 original Pixel offset Pixel offset Blur Examples 2.4 impulse 0.3 filtered 8 8 edge 4 4 0.3 filtered original

15. Linear filtering (warm-up slide) 2.0 1.0 ? 0 0 original

16. Linear Filtering (no change) 2.0 1.0 0 0 Filtered (no change) original

17. 0.33 0 Linear Filtering 2.0 ? 0 original

18. coefficient 0 Pixel offset (remember blurring) 0.3 original Blurred (filter applied in both dimensions).

19. 0.33 0 Sharpening 2.0 0 Sharpened original original

20. Sharpening example 1.7 11.2 8 8 coefficient -0.25 -0.3 original Sharpened (differences are accentuated; constant areas are left untouched).

21. Sharpening before after

22. Spatial resolution and color R G B original

23. R G B Blurring the G component processed original

24. R G B Blurring the R component processed original

25. R G B processed Blurring the B component original

26. Lab Color Component L A rotation of the color coordinates into directions that are more perceptually meaningful: L: luminance, a: red-green, b: blue-yellow a b

27. L a b processed Bluring L original

28. L a b processed Bluring a original

29. L a b processed Bluring b original

30. Application to image compression • (compression is about hiding differences from the true image where you can’t see them).

31. Edge Detection • Convert a 2D image into a set of curves • Extracts salient features of the scene • More compact than pixels

32. How can you tell that a pixel is on an edge?

33. Image gradient • The gradient of an image: • The gradient points in the direction of most rapid change in intensity • The gradient direction is given by: • how does the gradient relate to the direction of the edge? • The edge strength is given by the gradient magnitude

34. Effects of noise • Consider a single row or column of the image • Plotting intensity as a function of position gives a signal How to compute a derivative? • Where is the edge?

35. Look for peaks in Solution: smooth first • Where is the edge?

36. Derivative theorem of convolution • This saves us one operation:

37. Laplacian of Gaussian • Consider Laplacian of Gaussian operator • Where is the edge? • Zero-crossings of bottom graph

38. Canny Edge Detector • Smooth image I with 2D Gaussian: • Find local edge normal directions for each pixel • Along this direction, compute image gradient • Locate edges by finding max gradient magnitude (Non-maximum suppression)

39. Non-maximum Suppression • Check if pixel is local maximum along gradient direction • requires checking interpolated pixels p and r

40. The Canny Edge Detector original image (Lena)

41. The Canny Edge Detector magnitude of the gradient

42. The Canny Edge Detector After non-maximum suppression

43. Canny Edge Detector original Canny with Canny with • The choice of depends on desired behavior • large detects large scale edges • small detects fine features

44. Image Scaling This image is too big to fit on the screen. How can we reduce it? How to generate a half- sized version?

45. Image sub-sampling 1/8 1/4 • Throw away every other row and column to create a 1/2 size image • - called image sub-sampling

46. Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Why does this look so crufty?

47. Even worse for synthetic images

48. Really bad in video

49. Input signal: Matlab output: WHY? x = 0:.05:5; imagesc(sin((2.^x).*x)) Aj-aj-aj: Alias! Not enough samples Alias: n., an assumed name Picket fence receding Into the distance will produce aliasing…

50. Aliasing • occurs when your sampling rate is not high enough to capture the amount of detail in your image • Can give you the wrong signal/image—an alias • Where can it happen in images? • During image synthesis: • sampling continous singal into discrete signal • e.g. ray tracing, line drawing, function plotting, etc. • During image processing: • resampling discrete signal at a different rate • e.g. Image warping, zooming in, zooming out, etc. • To do sampling right, need to understand the structure of your signal/image • Enter Monsieur Fourier…