1 / 52

Image Processing and Morphing

Image Processing and Morphing. Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski. Image Pyramids. Image Pyramids. Pyramid Creation. “Laplacian” Pyramid Created from Gaussian pyramid by subtraction L l = G l – expand(G l+1 ). filter mask. “Gaussian” Pyramid.

marjorie
Download Presentation

Image Processing and Morphing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Processing and Morphing Vision for GraphicsCSE 590SS, Winter 2001Richard Szeliski

  2. Image Pyramids

  3. Image Pyramids Vision for Graphics

  4. Pyramid Creation • “Laplacian” Pyramid • Created from Gaussianpyramid by subtractionLl = Gl – expand(Gl+1) filter mask “Gaussian” Pyramid Vision for Graphics

  5. Octaves in the Spatial Domain Lowpass Images • Bandpass Images Vision for Graphics

  6. Pyramids • Advantages of pyramids • Faster than Fourier transform • Avoids “ringing” artifacts • Many applications • small images faster to process • good for multiresolution processing • compression • progressive transmission • Known as “mip-maps” in graphics community • Precursor to wavelets • Wavelets also have these advantages Vision for Graphics

  7. laplacian level 4 laplacian level 2 laplacian level 0 Vision for Graphics left pyramid right pyramid blended pyramid

  8. Pyramid Blending Vision for Graphics

  9. Image Warping

  10. f f f h h x x x f x Image Warping • image filtering: change range of image • g(x) = h(f(x)) • image warping: change domain of image • g(x) = f(h(x)) Vision for Graphics

  11. h h Image Warping • image filtering: change range of image • g(x) = h(f(x)) • image warping: change domain of image • g(x) = f(h(x)) f g f g Vision for Graphics

  12. Parametric (global) warping • Examples of parametric warps: aspect rotation translation perspective cylindrical affine Vision for Graphics

  13. 2D coordinate transformations • translation: x’ = x + t x = (x,y) • rotation: x’ = R x + t • similarity: x’ =s R x + t • affine: x’ = A x + t • perspective: x’  H xx = (x,y,1) (x is a homogeneous coordinate) • These all form a nested group (closed w/ inv.) Vision for Graphics

  14. Image Warping • Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’)=f(h(x))? h(x) x x’ f(x) g(x’) Vision for Graphics

  15. Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? h(x) x x’ f(x) g(x’) Vision for Graphics

  16. Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? • Answer: add “contribution” to several pixels, normalize later (splatting) h(x) x x’ f(x) g(x’) Vision for Graphics

  17. Inverse Warping • Get each pixel g(x’) from its corresponding location x=h-1(x’) in f(x) • What if pixel comes from “between” two pixels? h-1(x’) x x’ f(x) g(x’) Vision for Graphics

  18. Inverse Warping • Get each pixel g(x’) from its corresponding location x=h-1(x’) in f(x) • What if pixel comes from “between” two pixels? • Answer: resample color value from interpolated (prefiltered) source image x x’ f(x) g(x’) Vision for Graphics

  19. Interpolation • Possible interpolation filters: • nearest neighbor • bilinear • bicubic (interpolating) • sinc / FIR • Needed to prevent “jaggies” and “texture crawl” (see demo) Vision for Graphics

  20. Prefiltering • Essential for downsampling (decimation) to prevent aliasing • MIP-mapping [Williams’83]: • build pyramid (but what decimation filter?): • block averaging • Burt & Adelson (5-tap binomial) • 7-tap wavelet-based filter (better) • trilinear interpolation • bilinear within each 2 adjacent levels • linear blend between levels (determined by pixel size) Vision for Graphics

  21. Prefiltering • Essential for downsampling (decimation) to prevent aliasing • Other possibilities: • summed area tables • elliptically weighted Gaussians (EWA) [Heckbert’86] Vision for Graphics

  22. Image Warping – non-parametric • Specify more detailed warp function • Examples: • splines • triangles • optical flow (per-pixel motion) Vision for Graphics

  23. Image Warping – non-parametric • Move control points to specify spline warp Vision for Graphics

  24. Image Morphing

  25. Image Morphing • How can we in-between two images? • Cross-dissolve(all examples from [Gomes et al.’99]) Vision for Graphics

  26. Image Morphing • How can we in-between two images? • Warp then cross-dissolve = morph Vision for Graphics

  27. Warp specification • How can we specify the warp? • Specify corresponding points • interpolate to a complete warping function • Nielson, Scattered Data Modeling, IEEE CG&A’93] Vision for Graphics

  28. Warp specification • How can we specify the warp? • Specify corresponding vectors • interpolate to a complete warping function Vision for Graphics

  29. Warp specification • How can we specify the warp? • Specify corresponding vectors • interpolate [Beier & Neely, SIGGRAPH’92] Vision for Graphics

  30. Warp specification • How can we specify the warp? • Specify corresponding spline control points • interpolate to a complete warping function Vision for Graphics

  31. Final Morph Result Vision for Graphics

  32. Image Mosaics

  33. Image Mosaics (stitching) • Blend together several overlapping images into one seamless mosaic (composite) + + … + = Vision for Graphics

  34. Mosaics for Video Coding • Convert masked images into a background sprite for content-based coding • + + + • =

  35. f = 180 (pixels) Image 384x300 f = 380 f = 280 Cylindrical Panoramas • Map image to cylindrical or spherical coordinates • need known focal length Vision for Graphics

  36. Given focal length f and image center (xc,yc) (X,Y,Z) (sinq,h,cosq) Y Z X Cylindrical warping Vision for Graphics

  37. Radial distortion • Correct for “bending” in wide field of view lenses Vision for Graphics

  38. Image Stitching • Align the images over each other • Blend the images together (demo) Vision for Graphics

  39. Image Stitching Assignment • Take pictures on a tripod (or handheld) • Warp to cylindrical coordinates • Automatically compute pairwise alignments • Fix up the end-to-end alignment • Blend the images together • Crop the result and import into a viewer Vision for Graphics

  40. Image registration (preview) • How do we determine alignment between images? • One possible answer: block matching (correlation), i.e., find minimum squared error Vision for Graphics

  41. Image registration (preview) • How do we determine alignment between images? • Another possible answer: Fourier-domain correlation [Brown’92] or phase correlation [Kuglin & Hines’75] • <…discuss Fourier-domain convolution formulas…> • More on alignment/registration on Wednesday [Anandan] Vision for Graphics

  42. Full-view Panorama + + + + Vision for Graphics

  43. Texture Mapped Model Vision for Graphics

  44. Image Enhancement

  45. Image Enhancement • Noise removal: • low-pass filteringg(x) = h(x) f(x) = ih(-i) f(x+i) • median filtering • anisotropic diffusion • iteratively smooth with similar neighbors [Perona & Malik’90] • Sharpening: • “unsharp masking” • remove a little bit of blurred image from original(darkroom trick) Vision for Graphics

  46. Image Enhancement • Super-resolution: • hallucinate more detail from single low-res image[Freeman & Pasztor,ICCV’99] • “learn” mapping between levels • combine several low-res images using known (linear) formation model [Irani’91;Mann’94] Vision for Graphics

  47. Image Enhancement • Super-resolution: • [Freeman & Pasztor,ICCV’99] • mapping between levels depends on “training data” Vision for Graphics

  48. Image Enhancement • Brightness and gamma correction • linearize input  brightness • Histogram equalization • balance (“flatten”) distributionof brightness values Vision for Graphics

  49. Image Enhancement • High dynamic range photography[Debevec et al.’97; Mitsunaga & Nayar’99] • combine several different exposures together Vision for Graphics

  50. Bibliography • L. Williams. Pyramidal parametrics. • Computer Graphics, 17(3):1--11, July 1983. • L. G. Brown. A survey of image registration techniques. • Computing Surveys, 24(4):325--376, December 1992. • C. D. Kuglin and D. C. Hines. The phase correlation image alignment method. • In IEEE 1975 Conference on Cybernetics and Society, pages • 163--165, New York, September 1975. • J. Gomes, L. Darsa, B. Costa, and L. Velho. Warping and Morphing of Graphical Objects. • Morgan Kaufmann Publishers, San Francisco Altos, California, 1999. • G. M. Nielson. Scattered data modeling. • IEEE Computer Graphics and Applications, 13(1):60--70, January 1993. • T. Beier and S. Neely. Feature-based image metamorphosis. • Computer Graphics (SIGGRAPH'92), 26(2):35--42, July 1992. Vision for Graphics

More Related