530 likes | 861 Views
Image Processing and Morphing. Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski. Image Pyramids. Image Pyramids. Pyramid Creation. “Laplacian” Pyramid Created from Gaussian pyramid by subtraction L l = G l – expand(G l+1 ). filter mask. “Gaussian” Pyramid.
E N D
Image Processing and Morphing Vision for GraphicsCSE 590SS, Winter 2001Richard Szeliski
Image Pyramids Vision for Graphics
Pyramid Creation • “Laplacian” Pyramid • Created from Gaussianpyramid by subtractionLl = Gl – expand(Gl+1) filter mask “Gaussian” Pyramid Vision for Graphics
Octaves in the Spatial Domain Lowpass Images • Bandpass Images Vision for Graphics
Pyramids • Advantages of pyramids • Faster than Fourier transform • Avoids “ringing” artifacts • Many applications • small images faster to process • good for multiresolution processing • compression • progressive transmission • Known as “mip-maps” in graphics community • Precursor to wavelets • Wavelets also have these advantages Vision for Graphics
laplacian level 4 laplacian level 2 laplacian level 0 Vision for Graphics left pyramid right pyramid blended pyramid
Pyramid Blending Vision for Graphics
f f f h h x x x f x Image Warping • image filtering: change range of image • g(x) = h(f(x)) • image warping: change domain of image • g(x) = f(h(x)) Vision for Graphics
h h Image Warping • image filtering: change range of image • g(x) = h(f(x)) • image warping: change domain of image • g(x) = f(h(x)) f g f g Vision for Graphics
Parametric (global) warping • Examples of parametric warps: aspect rotation translation perspective cylindrical affine Vision for Graphics
2D coordinate transformations • translation: x’ = x + t x = (x,y) • rotation: x’ = R x + t • similarity: x’ =s R x + t • affine: x’ = A x + t • perspective: x’ H xx = (x,y,1) (x is a homogeneous coordinate) • These all form a nested group (closed w/ inv.) Vision for Graphics
Image Warping • Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’)=f(h(x))? h(x) x x’ f(x) g(x’) Vision for Graphics
Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? h(x) x x’ f(x) g(x’) Vision for Graphics
Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? • Answer: add “contribution” to several pixels, normalize later (splatting) h(x) x x’ f(x) g(x’) Vision for Graphics
Inverse Warping • Get each pixel g(x’) from its corresponding location x=h-1(x’) in f(x) • What if pixel comes from “between” two pixels? h-1(x’) x x’ f(x) g(x’) Vision for Graphics
Inverse Warping • Get each pixel g(x’) from its corresponding location x=h-1(x’) in f(x) • What if pixel comes from “between” two pixels? • Answer: resample color value from interpolated (prefiltered) source image x x’ f(x) g(x’) Vision for Graphics
Interpolation • Possible interpolation filters: • nearest neighbor • bilinear • bicubic (interpolating) • sinc / FIR • Needed to prevent “jaggies” and “texture crawl” (see demo) Vision for Graphics
Prefiltering • Essential for downsampling (decimation) to prevent aliasing • MIP-mapping [Williams’83]: • build pyramid (but what decimation filter?): • block averaging • Burt & Adelson (5-tap binomial) • 7-tap wavelet-based filter (better) • trilinear interpolation • bilinear within each 2 adjacent levels • linear blend between levels (determined by pixel size) Vision for Graphics
Prefiltering • Essential for downsampling (decimation) to prevent aliasing • Other possibilities: • summed area tables • elliptically weighted Gaussians (EWA) [Heckbert’86] Vision for Graphics
Image Warping – non-parametric • Specify more detailed warp function • Examples: • splines • triangles • optical flow (per-pixel motion) Vision for Graphics
Image Warping – non-parametric • Move control points to specify spline warp Vision for Graphics
Image Morphing • How can we in-between two images? • Cross-dissolve(all examples from [Gomes et al.’99]) Vision for Graphics
Image Morphing • How can we in-between two images? • Warp then cross-dissolve = morph Vision for Graphics
Warp specification • How can we specify the warp? • Specify corresponding points • interpolate to a complete warping function • Nielson, Scattered Data Modeling, IEEE CG&A’93] Vision for Graphics
Warp specification • How can we specify the warp? • Specify corresponding vectors • interpolate to a complete warping function Vision for Graphics
Warp specification • How can we specify the warp? • Specify corresponding vectors • interpolate [Beier & Neely, SIGGRAPH’92] Vision for Graphics
Warp specification • How can we specify the warp? • Specify corresponding spline control points • interpolate to a complete warping function Vision for Graphics
Final Morph Result Vision for Graphics
Image Mosaics (stitching) • Blend together several overlapping images into one seamless mosaic (composite) + + … + = Vision for Graphics
Mosaics for Video Coding • Convert masked images into a background sprite for content-based coding • + + + • =
f = 180 (pixels) Image 384x300 f = 380 f = 280 Cylindrical Panoramas • Map image to cylindrical or spherical coordinates • need known focal length Vision for Graphics
Given focal length f and image center (xc,yc) (X,Y,Z) (sinq,h,cosq) Y Z X Cylindrical warping Vision for Graphics
Radial distortion • Correct for “bending” in wide field of view lenses Vision for Graphics
Image Stitching • Align the images over each other • Blend the images together (demo) Vision for Graphics
Image Stitching Assignment • Take pictures on a tripod (or handheld) • Warp to cylindrical coordinates • Automatically compute pairwise alignments • Fix up the end-to-end alignment • Blend the images together • Crop the result and import into a viewer Vision for Graphics
Image registration (preview) • How do we determine alignment between images? • One possible answer: block matching (correlation), i.e., find minimum squared error Vision for Graphics
Image registration (preview) • How do we determine alignment between images? • Another possible answer: Fourier-domain correlation [Brown’92] or phase correlation [Kuglin & Hines’75] • <…discuss Fourier-domain convolution formulas…> • More on alignment/registration on Wednesday [Anandan] Vision for Graphics
Full-view Panorama + + + + Vision for Graphics
Texture Mapped Model Vision for Graphics
Image Enhancement • Noise removal: • low-pass filteringg(x) = h(x) f(x) = ih(-i) f(x+i) • median filtering • anisotropic diffusion • iteratively smooth with similar neighbors [Perona & Malik’90] • Sharpening: • “unsharp masking” • remove a little bit of blurred image from original(darkroom trick) Vision for Graphics
Image Enhancement • Super-resolution: • hallucinate more detail from single low-res image[Freeman & Pasztor,ICCV’99] • “learn” mapping between levels • combine several low-res images using known (linear) formation model [Irani’91;Mann’94] Vision for Graphics
Image Enhancement • Super-resolution: • [Freeman & Pasztor,ICCV’99] • mapping between levels depends on “training data” Vision for Graphics
Image Enhancement • Brightness and gamma correction • linearize input brightness • Histogram equalization • balance (“flatten”) distributionof brightness values Vision for Graphics
Image Enhancement • High dynamic range photography[Debevec et al.’97; Mitsunaga & Nayar’99] • combine several different exposures together Vision for Graphics
Bibliography • L. Williams. Pyramidal parametrics. • Computer Graphics, 17(3):1--11, July 1983. • L. G. Brown. A survey of image registration techniques. • Computing Surveys, 24(4):325--376, December 1992. • C. D. Kuglin and D. C. Hines. The phase correlation image alignment method. • In IEEE 1975 Conference on Cybernetics and Society, pages • 163--165, New York, September 1975. • J. Gomes, L. Darsa, B. Costa, and L. Velho. Warping and Morphing of Graphical Objects. • Morgan Kaufmann Publishers, San Francisco Altos, California, 1999. • G. M. Nielson. Scattered data modeling. • IEEE Computer Graphics and Applications, 13(1):60--70, January 1993. • T. Beier and S. Neely. Feature-based image metamorphosis. • Computer Graphics (SIGGRAPH'92), 26(2):35--42, July 1992. Vision for Graphics