1 / 53

Computational Photography CSE 590

Computational Photography CSE 590. Tamara Berg Filtering & Pyramids. Image filtering. Image filtering: compute function of local neighborhood at each position Really important! Enhance images Denoise, resize, increase contrast, etc. Extract information from images

edith
Download Presentation

Computational Photography CSE 590

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computational PhotographyCSE 590 Tamara Berg Filtering & Pyramids

  2. Image filtering • Image filtering: compute function of local neighborhood at each position • Really important! • Enhance images • Denoise, resize, increase contrast, etc. • Extract information from images • Texture, edges, distinctive points, etc. • Detect patterns • Template matching

  3. Example: box filter 1 1 1 1 1 1 1 1 1 Slide credit: David Lowe (UBC)

  4. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  5. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  6. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  7. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  8. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  9. Image filtering 1 1 1 1 1 1 1 1 1 ? Credit: S. Seitz

  10. Image filtering 1 1 1 1 1 1 1 1 1 ? Credit: S. Seitz

  11. Image filtering 1 1 1 1 1 1 1 1 1 Credit: S. Seitz

  12. Box Filter 1 1 1 1 1 1 1 1 1 • What does it do? • Replaces each pixel with an average of its neighborhood • Achieve smoothing effect (remove sharp features) Slide credit: David Lowe (UBC)

  13. Smoothing with box filter

  14. 0 0 0 0 1 0 0 0 0 Practice with linear filters ? Original Source: D. Lowe

  15. 0 0 0 0 1 0 0 0 0 Practice with linear filters Original Filtered (no change) Source: D. Lowe

  16. 0 0 0 0 0 1 0 0 0 Practice with linear filters ? Original Source: D. Lowe

  17. 0 0 0 0 0 1 0 0 0 Practice with linear filters Original Shifted left By 1 pixel Source: D. Lowe

  18. 0 1 0 1 1 0 1 0 1 2 1 0 1 0 1 0 1 0 Practice with linear filters - ? Original Source: D. Lowe

  19. 0 1 0 1 1 0 1 0 1 2 1 0 1 0 1 0 1 0 Practice with linear filters - Original • Sharpening filter • Accentuates differences with local average Source: D. Lowe

  20. Sharpening Source: D. Lowe

  21. intensity function(along horizontal scanline) first derivative edges correspond toextrema of derivative Characterizing edges • An edge is a place of rapid change in the image intensity function image source: Svetlana Lazebnik

  22. Edge filters Approximations of derivative filters: Convolve filter with image to get edge map Source: K. Grauman

  23. Edge filters Approximations of derivative filters: Respond highly to vertical edges Source: K. Grauman

  24. Edge filters Approximations of derivative filters: Respond highly to horizontal edges Source: K. Grauman

  25. Edges: example source: Svetlana Lazebnik

  26. Gaussian Kernel 0.003 0.013 0.022 0.013 0.003 0.013 0.059 0.097 0.059 0.013 0.022 0.097 0.159 0.097 0.022 0.013 0.059 0.097 0.059 0.013 0.003 0.013 0.022 0.013 0.003 5 x 5,  = 1 • Constant factor at front makes volume sum to 1 (can be ignored, as we should re-normalize weights to sum to 1 in any case) Source: C. Rasmussen

  27. Example: Smoothing with a Gaussian source: Svetlana Lazebnik

  28. Gaussian filters • Remove “high-frequency” components from the image (low-pass filter) • Images become more smooth • Convolution with self is another Gaussian • So can smooth with small-width kernel, repeat, and get same result as larger-width kernel would have • Separable kernel • Factors into product of two 1D Gaussians Source: K. Grauman

  29. Separability of the Gaussian filter The filter factorsinto a product of 1Dfilters: Source: D. Lowe

  30. Practical matters • What is the size of the output? • MATLAB: imfilter(g, f, shape) • shape = ‘full’: output size is sum of sizes of f and g • shape = ‘same’: output size is same as f • shape = ‘valid’: output size is difference of sizes of f and g full same valid g g g g f f f g g g g g g g g Source: S. Lazebnik

  31. Practical matters • What about near the edge? • the filter window falls off the edge of the image • need to extrapolate • methods: • clip filter (black) • wrap around • copy edge • reflect across edge Source: S. Marschner

  32. Q? Practical matters • methods (MATLAB): • clip filter (black): imfilter(f, g, 0) • wrap around: imfilter(f, g, ‘circular’) • copy edge: imfilter(f, g, ‘replicate’) • reflect across edge: imfilter(f, g, ‘symmetric’) Source: S. Marschner

  33. Template matching • Goal: find in image • Main challenge: What is a good similarity or distance measure between two patches? • Correlation • Zero-mean correlation • Sum Square Difference • Normalized Cross Correlation Side by Derek Hoiem

  34. Matching with filters f = image g = filter What went wrong? • Goal: find in image • Method 0: filter the image with eye patch Filtered Image Input Side by Derek Hoiem

  35. Matching with filters mean of f True detections False detections • Goal: find in image • Method 1: filter the image with zero-mean eye Thresholded Image Filtered Image (scaled) Input

  36. Matching with filters True detections Thresholded Image • Goal: find in image • Method 2: SSD 1- sqrt(SSD) Input

  37. Matching with filters What’s the potential downside of SSD? • Goal: find in image • Method 2: SSD 1- sqrt(SSD) Input Side by Derek Hoiem

  38. Matching with filters mean template mean image patch • Goal: find in image • Method 3: Normalized cross-correlation Side by Derek Hoiem

  39. Matching with filters True detections Thresholded Image • Goal: find in image • Method 3: Normalized cross-correlation Input Normalized X-Correlation

  40. Matching with filters True detections Thresholded Image • Goal: find in image • Method 3: Normalized cross-correlation Input Normalized X-Correlation

  41. Q: What is the best method to use? A: Depends Zero-mean filter: fastest but not a great matcher SSD: next fastest, sensitive to overall intensity Normalized cross-correlation: slowest, invariant to local average intensity and contrast Side by Derek Hoiem

  42. Image half-sizing This image is too big to fit on the screen. How can we reduce it? How to generate a half- sized version?

  43. Image sub-sampling 1/8 1/4 • Throw away every other row and column to create a 1/2 size image • - called image sub-sampling Slide by Steve Seitz

  44. Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Aliasing! What do we do? Slide by Steve Seitz

  45. Anti-aliasing Get rid of all frequencies that are greater than half the new sampling frequency • Will lose information • But it’s better than aliasing • Apply a smoothing filter Slide by Derek Hoiem

  46. Algorithm for downsampling by factor of 2 • Start with image(h, w) • Apply low-pass filter • im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1)) • Sample every other pixel • im_small = im_blur(1:2:end, 1:2:end); Slide by Derek Hoiem

  47. Gaussian (lowpass) pre-filtering G 1/8 G 1/4 Gaussian 1/2 Solution: filter the image, thensubsample Slide by Steve Seitz

  48. Subsampling with Gaussian pre-filtering Gaussian 1/2 G 1/4 G 1/8 Slide by Steve Seitz

  49. Compare with... 1/2 1/4 (2x zoom) 1/8 (4x zoom) Slide by Steve Seitz

  50. Image Pyramids • Known as a Gaussian Pyramid [Burt and Adelson, 1983] • In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform Slide by Steve Seitz

More Related