Computational Photography: Image Processing

1. Computational Photography:Image Processing Jinxiang Chai

2. Review 2D Fourier Transform Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal Antialiasing - prefiltering Gaussian pyramid - Mipmap

3. Review 2D Fourier Transform Nyquist sampling theory - Sampling Rate ≥ 2 * max frequency in the signal Antialiasing - prefiltering Gaussian pyramid - Mipmap

4. Review: 2D Fourier Transform Fourier Transform: Inverse Fourier transform:

5. f(x) F(u) x fmax u -fmax Fs(u) fs(x) x fmax u -fmax … … -2T -T 0 T 2T Review: Nyquist Sampling Rate Fourier transform Inverse Fourier transform Sampling Rate ≥ 2 * max frequency in the signal!

6. fmax u -fmax Review: Antialiasing Increase the sampling rate to above twice the highest frequency Introduce an anti-aliasing filter to reduce fmax Fs(u) fmax u -fmax

7. fmax u -fmax Review: Antialiasing Increase the sampling rate to above twice the highest frequency Introduce an anti-aliasing filter to reduce fmax • - Sinc filter • Mean filter • - Triangular filter • - Gaussian filter Fs(u) fmax u -fmax

8. Review: Image Pyramids • Known as a Gaussian Pyramid • MipMap (Williams, 1983)

9. Outline Image Processing - Median filtering - Bilateral filtering - Edge detection - Corner detection

10. Filtering input Gaussian filter

11. Median Filter • For each neighbor in image, sliding the window • Sort pixel values • Set the center pixel to the median

12. Median Filter input Gaussian filter Median filter

13. Median Filter Examples input Median 7X7

14. Median Filter Examples Median 11X11 Median 3X3

15. Median Filter Examples Straight edges kept Median 11X11 Median 3X3 Sharp features lost

16. Median Filter Properties Can remove outliers (peppers and salts) Window size controls size of structure Preserve some details but sharp corners and edges might get lost

17. Comparison of Mean, Gaussian, and Median original Mean with 6 pixels

18. Comparison of Mean, Gaussian, and Median original Gaussian with 6 pixels

19. Comparison of Mean, Gaussian, and Median original Median with 6 pixels

20. Common Problems Mean: blurs image, removes simple noise, no details are preserved

21. Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ.

22. Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise

23. Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise Can we find a filter that not only smooths regions but preserves edges?

24. Common Problems Mean: blurs image, removes simple noise, no details are preserved Gaussian: blurs image, preserves details only for small σ. Median: preserves some details, good at removing strong noise Can we find a filter that not only smooths regions but preserves edges? - yes, bilateral filter

25. Outline Image Processing - Median filtering - Bilateral filtering - Edge detection - Corner detection

26. What Is Bilateral Filter? Bilateral - Affecting or undertaken by two sides equally Property: - Convolution filter - Smooth image but preserve edges - Operates in the domain and the range of image

27. Bilateral Filter Example Gaussian filter Bilateral filter

28. Bilateral Filter Example Gaussian filter Bilateral filter

29. 1D Graphical Example Center Sample u I(p) Neighborhood p It is clear that in weighting this neighborhood, we would like to preserve the step

30. The Weights I(p) p

31. Filtered Values I(p) Filtered value p

32. Edges Are Smoothed I(p) Filtered value p

33. What Causes the Problem? I(p) Filtered value p

34. What Causes the Problem? Same weights for these two pixels!! I(p) Filtered value p

35. The Weights I(p) p

36. Bilateral Filtering Denoise Feature preserving Bilateral filter Normalization

37. Kernel Properties • Per each sample, we can define a ‘Kernel’ that averages its neighborhood • This kernel changes from sample to sample! • The sum of the kernel entries is 1 due to the normalization, • The center entry in the kernel is the largest, • Subject to the above, the kernel can take any form (as opposed to filters which are monotonically decreasing).

38. Filter Parameters As proposed by Tomasi and Manduchi, the filter is controlled by 3 parameters: The filter can be applied for several iterations in order to further strengthen its edge-preserving smoothing N(u) – The neighbor size of the filter support, c – The variance of the spatial distances, s – The variance of the value distances,

39. Bilateral Filter input

40. Bilateral Filter input

41. Bilateral Filter input Wc

42. Bilateral Filter input Wc Ws

43. Bilateral Filter input Ws*Wc Wc Ws

44. Bilateral Filter input Output Ws*Wc Wc Ws

45. Bilateral Filter Results Original

46. Bilateral Filter Results σc = 3, σs = 3

47. Bilateral Filter Results σc = 6, σs = 3

48. Bilateral Filter Results σc = 12, σs = 3

49. Bilateral Filter Results σc = 12, σs = 6

50. Bilateral Filter Results σc = 15, σs = 8