CSCE 643 Computer Vision: Template Matching, Image Pyramids and Denoising

1. CSCE 643 Computer Vision:Template Matching, Image Pyramids and Denoising JinxiangChai

2. Today’s class • Template matching • Gaussian Pyramids • Laplacian Pyramids • Image denoising

3. Template matching • Goal: find in image

4. Template matching • Goal: find in image • Main challenge: What is a good similarity or distance measure between two patches?

5. Template matching • Goal: find in image • Main challenge: What is a good similarity or distance measure between two patches? • Correlation • Zero-mean correlation • Sum Square Difference • Normalized Cross Correlation Slide: Hoiem

6. Matching with filters • Goal: find in image • Method 0: filter the image with eye patch f = image g = filter What went wrong? Filtered Image Input Slide: Hoiem

7. Problem with Correlation

8. Problem with Correlation

9. Solution

10. Matching with filters • Goal: find in image • Method 1: filter the image with zero-mean eye mean of f True detections False detections Thresholded Image Filtered Image (scaled) Input Slide: Hoiem

11. Matching with filters • Goal: find in image • Method 2: SSD True detections Thresholded Image 1- sqrt(SSD) Input Slide: Hoiem

12. Relationship SSD and Correlation

13. Matching with filters What’s the potential downside of SSD? • Goal: find in image • Method 2: SSD 1- sqrt(SSD) Input Slide: Hoiem

14. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation - subtracting the mean - dividing by the standard deviation

15. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation mean template mean image patch Matlab: normxcorr2(template, im)

16. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation True detections Thresholded Image Input Normalized X-Correlation Slide: Hoiem

17. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation True detections Thresholded Image Input Normalized X-Correlation Slide: Hoiem

18. Q: What is the best method to use? A: Depends • SSD: faster, sensitive to overall intensity • Normalized cross-correlation: slower, invariant to local average intensity and contrast

19. Q: What if we want to find larger or smaller eyes? A: Image Pyramid

20. Gaussian Pyramid Gaussian Filter Sample Image Low-Pass Filtered Image Low-Res Image

21. Gaussian Pyramid filter mask • Repeat • Filter • Subsample • Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid)

22. Gaussian pyramid

23. Template Matching with Image Pyramids Input: Image, Template • Match template at current scale • Downsample image • Repeat 1-2 until image is very small • Take responses above some threshold, perhaps with non-maxima suppression

24. Laplacian pyramid

25. Laplacian Filter The filtering kernel for Laplacian is Finite difference allows us to approximate the Laplacian using the following filtering kernel.

26. Laplacian Pyramid Laplacian Pyramid (subband images) - Created from Gaussian pyramid by subtraction

27. is the Laplacian operator: The Laplacian of Gaussian Kernel

28. The Laplacian of Gaussian Filter Kernel

29. Laplacian Pyramid What happens in frequency domain?

30. Laplacian Pyramid What happens in frequency domain? Bandpass filters Can we reconstruct the original from the laplacian pyramid?

31. Laplacian Pyramid What happens in frequency domain? Bandpass filters Can we reconstruct the original from the laplacian pyramid? - Yes, laplacian pyramid is a complete image representation which encodes fine to course structure in a different level

32. Image representation • Pixels: great for spatial resolution, poor access to frequency • Fourier transform: great for frequency, not for spatial info such as locations. • Pyramids/filter banks: balance between spatial and frequency information

33. Major uses of image pyramids • Compression • Object detection • Scale search • Features • Detecting stable interest points • Image blending/mosaicing • Registration • Course-to-fine

34. Multi-resolution SIFT Feature Detection - Object recognition from local scale-invariant features [pdf link], ICCV 09 - David G. Lowe, "Distinctive image features from scale-invariant keypoints,"International Journal of Computer Vision, 60, 2 (2004), pp. 91-110

35. Pyramid Blending

36. laplacian level 4 laplacian level 2 laplacian level 0 left pyramid right pyramid blended pyramid

37. Hierarchical Image Registration • Compute Gaussian pyramid • Align with coarse pyramid • Successively align with finer pyramids • Search smaller range Why is this faster? - Hierarchical Model-Based Motion Estimation, ECCV 92

38. Denoising Gaussian Filter Additive Gaussian Noise

39. Gaussian input Gaussian filter

40. Median Filter • For each neighbor in image, sliding the window • Sort pixel values • Set the center pixel to the median

41. Median Filter input Gaussian filter Median filter

42. Median Filter Examples input Median 7X7

43. Median Filter Examples Median 11X11 Median 3X3

44. Median Filter Examples Straight edges kept Median 11X11 Median 3X3 Sharp features lost

45. Median Filter Properties Can remove outliers (peppers and salts) Window size controls size of structure Preserve some details but sharp corners and edges might get lost

46. Comparison of Mean, Gaussian, and Median original Mean with 6 pixels

47. Comparison of Mean, Gaussian, and Median original Gaussian with 6 pixels

48. Comparison of Mean, Gaussian, and Median original Median with 6 pixels

49. Common Problems Mean: blurs image, removes simple noise, no details are preserved

50. Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ.