1 / 45

Templates and Image Pyramids

09/05/13. Templates and Image Pyramids. Computational Photography Derek Hoiem, University of Illinois. Administrative stuff. Start working on project 1 (due Sept 16) Make sure you can get a project page up Can now complete first part (hybrid images ) Matlab /algebra tutorial

zyta
Download Presentation

Templates and Image Pyramids

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 09/05/13 Templates and Image Pyramids Computational Photography Derek Hoiem, University of Illinois

  2. Administrative stuff • Start working on project 1 (due Sept 16) • Make sure you can get a project page up • Can now complete first part (hybrid images) • Matlab/algebra tutorial • Friday at 5pm, SC3405 • Office hours • Derek: mon 11-12 • Kevin: Thurs 10-11

  3. Review • Match the spatial domain image to the Fourier magnitude image 2 3 1 4 5 B C E A D

  4. Today’s class: applications of filtering • Template matching • Coarse-to-fine alignment • Denoising, Compression

  5. Template matching • Goal: find in image • Main challenge: What is a good similarity or distance measure between two patches? • Correlation • Zero-mean correlation • Sum Square Difference • Normalized Cross Correlation

  6. Matching with filters • Goal: find in image • Method 0: filter the image with eye patch f = image g = filter What went wrong? Filtered Image Input

  7. Matching with filters • Goal: find in image • Method 1: filter the image with zero-mean eye mean of f True detections False detections Thresholded Image Filtered Image (scaled) Input

  8. Matching with filters • Goal: find in image • Method 2: SSD True detections Thresholded Image 1- sqrt(SSD) Input

  9. Matching with filters Can SSD be implemented with linear filters?

  10. Matching with filters What’s the potential downside of SSD? • Goal: find in image • Method 2: SSD 1- sqrt(SSD) Input

  11. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation mean template mean image patch Matlab: normxcorr2(template, im)

  12. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation True detections Thresholded Image Input Normalized X-Correlation

  13. Matching with filters • Goal: find in image • Method 3: Normalized cross-correlation True detections Thresholded Image Input Normalized X-Correlation

  14. Q: What is the best method to use? A: Depends • Zero-mean filter: fastest but not a great matcher • SSD: next fastest, sensitive to overall intensity • Normalized cross-correlation: slowest, invariant to local average intensity and contrast

  15. Q: What if we want to find larger or smaller eyes? A: Image Pyramid

  16. Review of Sampling Gaussian Filter Sample Image Low-Pass Filtered Image Low-Res Image

  17. Gaussian pyramid Source: Forsyth

  18. Laplacian filter unit impulse Gaussian Laplacian of Gaussian Source: Lazebnik

  19. Laplacian pyramid Source: Forsyth

  20. Computing Gaussian/Laplacian Pyramid Can we reconstruct the original from the laplacian pyramid? http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html

  21. Hybrid Image in Laplacian Pyramid Extra points for project 1 High frequency  Low frequency

  22. Coarse-to-fine Image Registration • Compute Gaussian pyramid • Align with coarse pyramid • Find minimum SSD position • Successively align with finer pyramids • Search small range (e.g., 5x5) centered around position determined at coarser scale Why is this faster? Are we guaranteed to get the same result?

  23. Question Can you align the images using the FFT?

  24. Compression How is it that a 4MP image can be compressed to a few hundred KB without a noticeable change?

  25. Lossy Image Compression (JPEG) Block-based Discrete Cosine Transform (DCT) Slides: Efros

  26. Using DCT in JPEG • The first coefficient B(0,0) is the DC component, the average intensity • The top-left coeffs represent low frequencies, the bottom right – high frequencies

  27. Image compression using DCT • Quantize • More coarsely for high frequencies (which also tend to have smaller values) • Many quantized high frequency values will be zero • Encode • Can decode with inverse dct Filter responses Quantization table Quantized values

  28. JPEG Compression Summary • Convert image to YCrCb • Subsample color by factor of 2 • People have bad resolution for color • Split into blocks (8x8, typically), subtract 128 • For each block • Compute DCT coefficients • Coarsely quantize • Many high frequency components will become zero • Encode (e.g., with Huffman coding) http://en.wikipedia.org/wiki/YCbCr http://en.wikipedia.org/wiki/JPEG

  29. Lossless compression (PNG) • Predict that a pixel’s value based on its upper-left neighborhood • Store difference of predicted and actual value • Pkzip it (DEFLATE algorithm)

  30. Denoising Gaussian Filter Additive Gaussian Noise

  31. Reducing Gaussian noise Smoothing with larger standard deviations suppresses noise, but also blurs the image Source: S. Lazebnik

  32. Reducing salt-and-pepper noise by Gaussian smoothing 3x3 5x5 7x7

  33. Alternative idea: Median filtering • A median filter operates over a window by selecting the median intensity in the window • Is median filtering linear? Source: K. Grauman

  34. Median filter • What advantage does median filtering have over Gaussian filtering? • Robustness to outliers Source: K. Grauman

  35. Median filter Median filtered Salt-and-pepper noise • MATLAB: medfilt2(image, [h w]) Source: M. Hebert

  36. Median Filtered Examples http://en.wikipedia.org/wiki/File:Medianfilterp.png http://en.wikipedia.org/wiki/File:Median_filter_example.jpg

  37. Median vs. Gaussian filtering 3x3 5x5 7x7 Gaussian Median

  38. Other filter choices • Weighted median (pixels further from center count less) • Clipped mean (average, ignoring few brightest and darkest pixels) • Bilateral filtering (weight by spatial distance and intensity difference) Bilateral filtering Image: http://vision.ai.uiuc.edu/?p=1455

  39. Review of Last 3 Days • Filtering in spatial domain • Slide filter over image and take dot product at each position • Remember linearity (for linear filters) • Examples • 1D: [-1 0 1], [0 0 0 0 0.5 1 1 1 0.5 0 0 0] • 1D: [0.25 0.5 0.25], [0 0 0 0 0.5 1 1 1 0.5 0 0 0] • 2D: [1 0 0 ; 0 2 0 ; 0 0 1]/4

  40. Review of Last 3 Days • Linear filters for basic processing • Edge filter (high-pass) • Gaussian filter (low-pass) [-1 1] Gaussian FFT of Gradient Filter FFT of Gaussian

  41. Review of Last 3 Days • Derivative of Gaussian

  42. Review of Last 3 Days • Filtering in frequency domain • Can be faster than filtering in spatial domain (for large filters) • Can help understand effect of filter • Algorithm: • Convert image and filter to fft (fft2 in matlab) • Pointwise-multiply ffts • Convert result to spatial domain with ifft2

  43. Review of Last 3 Days • Applications of filters • Template matching (SSD or Normxcorr2) • SSD can be done with linear filters, is sensitive to overall intensity • Gaussian pyramid • Coarse-to-fine search, multi-scale detection • Laplacian pyramid • Can be used for blending (later) • More compact image representation

  44. Review of Last 3 Days • Applications of filters • Downsampling • Need to sufficiently low-pass before downsampling • Compression • In JPEG, coarsely quantize high frequencies • Reducing noise (important for aesthetics and for later processing such as edge detection) • Gaussian filter, median filter, bilateral filter

  45. Next class • Light and color

More Related