Assignment 1

1. Homographies, Image Mosaics and Tracking Assignment 1

2. Homography estimation from corresponding points • Homographies describe image transformation of... • General scene when camera motion is rotation about camera center • Planar surfaces under general camera motion • Displaying tracking data on a map • Image mosaic / stitch and Texture mapping • Bilinear interpolation • Image compositing • Wii (optional) Key Parts

3. Goal: Image Mosaics • + + … + = Goal: Stitch together several images into a seamless composite

4. First part

5. Displaying Tracking Data

6. Second part

7. Texture mapping on Legobricks

8. Image Warping • Given a coordinate transform x’ = h(x) and a source image f(x), how do we compute a transformed image g(x’)=f(h(x))? • What about holes? h(x) x x’ f(x) g(x’)

9. Forward Warping • Send each pixel f(x) to its corresponding location x’=h(x) in g(x’) • What if pixel lands “between” two pixels? • Answer: add “contribution” to several pixels, normalize later (splatting) h(x) x x’ f(x) g(x’)

10. Inverse Warping • Get each pixel g(x’) from its corresponding location x’=h(x) in f(x) • What if pixel comes from “between” two pixels? h(x) x x’ f(x) g(x’)

11. Inverse Warping • Get each pixel g(x’) from its corresponding location x’=h(x) in f(x) • What if pixel comes from “between” two pixels? • Answer: resample color value from interpolated (prefiltered) source image x x’ f(x) g(x’)

12. Interpolation • Possible interpolation filters: • nearest neighbor • bilinear • bicubic (interpolating)

13. Round-off idea: Just use closest integer-valued pixel Nearest neighbor interpolation (NN)

14. Problem is that it can cause big aliasing effects Why? Because the round() function causes discontinuous switches in which pixel is nearest and hence is the color drawn NN issues

15. NN aliasing rotate 45±, scale 1.5

16. t controls “blend” of two endpoints • From parametric definition of a line segment: p(t) = p0 + t(p1 ¡ p0), wheret2[0, 1] = p0¡tp0 + tp1 = (1 ¡ t)p0 + tp1 Blending from Akenine-Möller & Haines

17. Vertical blend Horizontal blend • Idea: Blend four pixel values surrounding source, weighted by nearness • (see MO chapter 5) Bilinear Interpolation (BLI)

18. Blending eliminates abrupt color changes, reducing aliasing artifacts Bilinear interpolation rotate 45±, scale 1.5

19. Pixel Interpolation approaches:NN vs. BLI

20. Pixel Interpolation approaches:NN vs. BLI

21. Image Stitching

22. Assembling the panorama • Stitch pairs together, blend, then crop

23. With homography computed, how to render combined image? • Simply putting one image on top of the other, even with bilinear interpolation, may result in a “seam” due to different brightness levels • Auto-iris can change overall lightness of images • Vignetting can make image edges darker Image Compositing Issues courtesy of P. Haeberli

24. Image feathering • Weight each image proportional to its distance from the edge (distance map [Danielsson, CVGIP 1980] • 1. Generate weight map for each image • 2. Sum up all of the weights and divide by sum:weights sum up to 1: wi’ = wi / ( ∑iwi)

25. Image Feathering

26. + = 1 0 1 0 Feathering

27. 0 1 0 1 Effect of window size left right

28. 0 1 0 1 Effect of window size

29. 0 1 Good window size • “Optimal” window: smooth but not ghosted • Doesn’t always work...

30. Idea: Use “hat” function w indicating weight of contributions of an image to the mosaic • w is 1 at source image center, falls linearly to 0 at image boundaries • Combination of horizontal and vertical hat functions: • Normalize hat weights to get blend factor in overlaping area: Bilinear Compositing

31. Future Directions

32. various student projects (small and large)

33. Mosaics for Video Coding???? • Convert masked images into a background sprite for content-based coding • + + + =

34. Fully automated 2D stitching Demo

35. Recognizing Panoramas [Brown & Lowe, ICCV’03]