Non-linear filtering

1. Non-linear filtering • Example: Median filter • Replaces pixel value by median value over neighborhood • Generates no new gray levels

2. Median filters Advantage (?): the “odd-man-out” effect e.g. 1,1,1,7,1,1,1,1  ?,1,1,1.1,1,1,?

3. Median filters: example filter width = 5 Input Median Linear (averaging)

4. Median filters: example filter width = 5 Input Discards spike Median Preserves jump exactly at correct location Linear (averaging)

5. Median filters: example filter width = 5 Input Median Smooths spike Linear (averaging) Blurs jump

6. Median filter Original 3 x 3 median filter Keeps edges sharp  Odd Man Out can destroy details  Sharpens edges  Destroys edge cusps and protrusions 

7. Median filter vs Gaussian Gaussian 3 x 3 median filter Eye has more detail. Blurrier edges

8. Median filter: summary • Purpose • Smoothimagewithout blurring boundaries Input Median Filter

9. Median filter: summary • Purpose • Smooth image without blurring boundaries • Problem • Loses small details Input Median Filter

10. Another approach: bilateral filter Input Gaussian Smoothed

11. Another approach: bilateral filter [ Input Gaussian Smoothed OK to average these brightness values

12. Another approach: bilateral filter [ Input Gaussian Smoothed Don’t average these (brightness too different, probably from different objects)

13. Another approach: bilateral filter Input Gaussian Smoothed • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

14. Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

15. Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

16. Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels (similar brightness) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

17. Another approach: bilateral filter Input Gaussian Smoothed Don’t sum over 3rd pixel (too dark) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

18. Another approach: bilateral filter Input Gaussian Smoothed Don’t sum over 3rd pixel (too bright) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

19. Another approach: bilateral filter Input Gaussian Smoothed Sum over all 3 pixels (similar brightness)) • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

20. Another approach: bilateral filter Input Bilateral Smoothing • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

21. Another approach: bilateral filter Smoothed Input Bilateral Smoothing • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

22. Another approach: bilateral filter Not smoothed Input Bilateral Smoothing • Bilateral smoothing: basic idea • Ordinary smoothing… • But only sum over pixels with similar brightness

23. Another approach: bilateral filter Input Bilateral Smoothing • Bilateral smoothing: implementation • Ordinary smoothing… • But only sum over pixels with similar brightness • Pixels with similar brightness weighted more

24. Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing

25. Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing

26. Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing Normalization (so weights sum to 1)

27. Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing • New: bilateral smoothing

28. Bilateral filter • Weight pixel contributions by nearness in distance and nearness in intensity • Old: Gaussian smoothing • New: bilateral smoothing Suppresses contribution from pixels with different brightness

29. Bilateral filter

30. Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask

31. Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3

32. Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Image (1D) … mask weights

33. Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Image (1D) … mask weights

34. Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Vector i Image (1D) … mask weights Vector w

35. Pattern detection with filters • Biggest response to brightness patterns resembling them! • Example: 1D correlation with mask Compute at position 3 Vector i Image (1D) … mask weights Vector w dot product

36. Pattern detection with filters • Gaussian filter • Detects blobs • “Derivative” filter e.g., [-1/2 0 1/2] • Detects brightness jumps • eye detector?

37. Pattern detection with filters • Summary: detect pattern (object) by correlating with similar mask • Problems when lighting changes • Overall brightness can change between images • Contrast may change • Or both…

38. Want to detect pattern Old Correlate image with Pattern “detected” wherever is large To improve detection: Compensate for changes in brightness and contrast

39. Want to detect pattern Old Correlate image with Pattern “detected” wherever is large New: at each pixel (i,j) Subtract average brightness (to eliminate problems from additive shifts) Do weighted sum To improve detection: Compensate for changes in brightness and contrast

40. New: at each pixel (i,j) Subtract average brightness Do weighted sum Normalize (avoids problems from contrast changes) To improve detection: Compensate for changes in brightness and contrast

41. Normalized Correlation Important: for each (x,y)(i.e., each pixel in result ) you must compute the average for window centered over (x,y)

42. Normalized Correlation • MATLAB demo (eye detection)

43. Detection with Normalized Correlation • Problem Result of normalized correlation Suppose you want 3 best candidates for an eye…

44. Detection with Normalized Correlation • Problem Result of normalized correlation The three highest NCvalues are all here (at best eye position plus 2 nearby pixels) Suppose you want 3 best candidates for an eye…

45. Detection with Normalized Correlation • Solution Result of normalized correlation Only consider pixels with higher NC value than at all other “nearby” pixels

46. Detection with Normalized Correlation • Solution Result of normalized correlation (Non Ma Only consider pixels with higher NC value than at all other “nearby” pixels (Non maximum suppression)

47. Detection with Normalized Correlation • Solution Result of NC and NonMax Suppression Result of normalized correlation