CSCE 441: Computer Graphics Image Warping

1. CSCE 441: Computer GraphicsImage Warping Jinxiang Chai

2. Required Readings • Section 3.6.1 (Szeliski book)

3. f f f h h x x x f x Image Warping Image filtering: change range of image g(x) = h(f(x)) Image warping: change domain of image g(x) = f(h(x))

4. h h Image Warping Image filtering: change range of image g(x) = h(f(x)) f g Image warping: change domain of image g(x) = f(h(x)) f g

5. Image Warping Why? - texture mapping - image processing (rotation, zoom in/out, etc) - image morphing/blending - image editing - image based-modeling & rendering

6. Parametric (global) Warping Examples of image warps: aspect rotation translation cylindrical affine perspective

7. Transformation Function f, g Transform the geometry of an image to a desired geometry

8. Definition: Image Warping Source Image: Image to be used as the reference. The geometry of this image is no changed Target Image: this image is obtained by transforming the reference image. (x,y): coordinates of points in the reference image (u,v): coordinates of points in the target image f,g or F,G: x and y components of a transformation function

9. Definition: Image Warping Control points: Unique points in the reference and target images. The coordinates of corresponding control points in images are used to determine a transformation function. Source Image Target Image

10. A Transformation Function Used to compute the corresponding points u = f(x,y) v = g(x.y) x = F(u,v) y = G(x.v) Source Image S(x,y) Target Image T(u,v)

11. Warping Types Simple mappings: - Similarity - Affine mapping - Projective mapping These can be applied globally over a subdivision of the plane: - Piecewise affine over triangulation - Piecewise projective over quadrilaterization - Piecewise bilinear over a rectangular grid Or other, arbitrary functions can be used, e.g. - Bieer-neely warp (popular for morphs) - Store u(x,y) and v(x,y) in large arrays

12. Similarity Transform In matrix notation: Have the form: • A combination of 2-D scale, rotation, and translation transformations. • Allows a square to be transformed into any rotated rectangle. • Angle between lines is preserved • 5 degrees of freedom (sx,sy,θ,tx,ty) • Inverse is of same form (is also similarity). Given by inverse of 3X3 matrix above

13. Affine Transform In matrix notation: Have the form: • A combination of 2-D scale, rotation, shear, and translation transformations. • Allows a square to be distorted into any parallelogram. • 6 degrees of freedom (a,b,c,d,e,f) • Inverse is of same form (is also affine). Given by inverse of 3X3 matrix above • Good when controlling a warp with triangles, since 3 points in 2D determined the 6 degrees of freedom

14. Projective Transform (a.k.a “perspective”) • Linear numerator & denominator • If g=h=0, then you get affine as a special case • Allow a square to be distorted into any quadrilateral • 8 degrees of freedom (a-h). We can choose i=1, arbitrarily • Inverse is of same form (is also projective). • Good when controlling a warp with quadrilaterals, since 4 points in 2D determine the 8 degrees of freedom In matrix notation: Have the form:

15. Image Warping x u y v T(u,v) S(x,y) Given a coordinate transform function f,g or F,G and source image S(x,y), how do we compute a transformed image T(u,v)?

16. u v Forward Warping x y T(u,v) S(x,y) Forward warping algorithm: for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v)

17. u v Forward Warping x y T(u,v) S(x,y) Forward warping algorithm: for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v) • Any problems for forward warping?

18. Forward Warping x u y v T(u,v) S(x,y) Q: What if the transformed pixel located between pixels?

19. Forward Warping x u y v T(u,v) S(x,y) Q: What if the transformed pixel located between pixels? A: Distribute color among neighboring pixels - known as “splatting”

20. Forward Warping • Iterate over source, sending pixels to destination • Some source pixels maps to the same dest. pixel • Some dest. pixels may have no corresponding source • Holes in reconstruction • Must splat etc. x u y v for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v)

21. Forward Warping • Iterate over source, sending pixels to destination • Some source pixels map to the same dest. pixel • Some dest. pixels may have no corresponding source • Holes in reconstruction • Must splat etc. - How to remove the holes? x u y v for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v)

22. Forward Warping Iterate over source, sending pixels to destination Some source pixels map to the same dest. pixel Some dest. pixels may have no corresponding source Holes in reconstruction Must splat etc. - How to remove the holes? x u y v for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v)

23. Inverse Warping x u y v T(u,v) S(x,y) Inverse warping algorithm: for v = vmin to vmax for u = umin to umax x = F(u,v); y = G(u,v) copy pixel at source S(x,y) to T(u,v)

24. Inverse Warping x u y v S(x,y) T(u,v) Q: What if pixel comes from “between” two pixels? A: Interpolate color values from neighboring pixels

25. Inverse Warping • Iterate over dest., finding pixels from source • Non-integer evaluation source image, resample • May oversample source • But no holes • Simpler, better than forward mapping x u y v for v = vmin to vmax for u = umin to umax x = F(u,v); y = G(u,v) copy pixel at source S(x,y) to T(u,v)

26. Resampling Filter

27. Resampling x u y v This is a 2D signal reconstruction problem!

28. Resampling Compute weighted sum of pixel neighborhood - Weights are normalized values of kernel function - Equivalent to convolution at samples with kernel - Find good filters using ideas of previous lectures x u y v

29. Point Sampling Nearest neighbor • Copies the color of the pixel with the closest integer coordinate • A fast and efficient way to process textures if the size of the • target is similar to the size of the reference • Otherwise, the result can be a chunky, aliased, or blurred image. x u y v

30. Bilinear Filter Weighted sum of four neighboring pixels x u y v

31. Bilinear Filter y Sampling at S(x,y): (i,j) (i,j+1) u x v (i+1,j+1) (i+1,j) S(x,y) = (1-a)*(1-b)*S(i,j) + a*(1-b)*S(i+1,j) + (1-a)*b*S(i,j+1) + a*b*S(i+1,j+1)

32. Bilinear Filter y Sampling at S(x,y): (i,j) (i,j+1) u x v (i+1,j+1) (i+1,j) S(x,y) = a*b*S(i,j) + a*(1-b)*S(i+1,j) + (1-a)*b*S(i,j+1) + (1-a)*(1-b)*S(i+1,j+1) To optimize the above, do the following Si = S(i,j) + a*(S(i,j+1)-S(i)) Sj = S(i+1,j) + a*(S(i+1,j+1)-S(i+1,j)) S(x,y) = Si+b*(Sj-Si)

33. Bilinear Filter y (i,j) (i,j+1) x (i+1,j+1) (i+1,j)

34. Inverse Warping and Resampling x u y v (x,y) (u,v) Inverse warping algorithm: for v = vmin to vmax for u = umin to umax float x = F(u,v); float y = G(u,v); T(u,v) = resample_souce(x,y,w);