1 / 31

View Morphing by Steven M. Seitz Charles R. Dyer

Comp 767: Advanced Topics in Graphics: Image-Based Rendering. View Morphing by Steven M. Seitz Charles R. Dyer. Irwin Chiu Hau Computer Science McGill University Winter 2004. Comp 767: Advanced Topics in Graphics: Image-Based Rendering. Overview. Mona Lisa view morphs

atara
Download Presentation

View Morphing by Steven M. Seitz Charles R. Dyer

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Comp 767: Advanced Topics in Graphics: Image-Based Rendering View MorphingbySteven M. Seitz Charles R. Dyer Irwin Chiu Hau Computer Science McGill University Winter 2004

  2. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Overview Mona Lisa view morphs Source: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm • What is view morphing? • How to do view morphing? • Results • Conclusion

  3. Comp 767: Advanced Topics in Graphics: Image-Based Rendering View Morphing Virtual Cameras Source: View Morphing; Steven M. Seitz, Charles R. Dyer What is view morphing? • What is it? • Why do we care about it? • Where do we see them? • Image Morphing vs View interpolation vs View Morphing

  4. Comp 767: Advanced Topics in Graphics: Image-Based Rendering View Morphing: Key Idea Beier-Neely morph is NOT shape-preserving! • distortions • un-natural A Shape-Distorting Morph Source: Steven M. Seitz, Charles R. Dyer

  5. Comp 767: Advanced Topics in Graphics: Image-Based Rendering View Morphing: Key Idea View morphing uses 3D shape preserving morph! • no distortions • natural A morph is 3D shape preserving if the results of two different views represent new views of the same object View morphing fixes these intermediate steps! A Shape-Distorting Morph Source: Steven M. Seitz, Charles R. Dyer

  6. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Why do we care? View morphing is efficient Produces new views without • 3D modelling • Taking additional photos View morphing creates impressive effects • Camera motion • Image morphing trueSpace Source: www.caligari.com

  7. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Image Morphing vs View interpolation vs View Morphing View Morphing is an extension to Image Morphing (Beier and Neely, 1992) • Produces physically plausible new views of a scene View Morphing (Seitz and Dyer, 1996) is an improvement over View Interpolation (Chen and Williams, 1993) • Addresses non-rigid transformations problems • Does not require depth values • Creates realistic image transitions

  8. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Overview Mona Lisa view morphs Source: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm • What is view morphing? • How to do view morphing? • Results • Conclusion

  9. Comp 767: Advanced Topics in Graphics: Image-Based Rendering How to do View Morphing? View morphing in three steps • Prewarp two images • Compute a morph between the prewarped images • Postwarp each in between images produced by the morph View Morphing Procedure Source: Steven M. Seitz, Charles R. Dyer

  10. Comp 767: Advanced Topics in Graphics: Image-Based Rendering How to do View Morphing? View morphing in three steps • Prewarp two images • Compute a morph between the prewarped images • Postwarp each in between images produced by the morph View morphing in 1 steps: Assume parallel views! • Compute a morph between the parallel images

  11. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Parallel Views Basic Theory image point p0 = (x0,y0) scene point P = (X,Y,Z) p0 = Π0 P where Π0 is a projection matrix image point p1 = (x1,y1) scene point P = (X,Y,Z) p1 = Π1 P where Π1 is a projection matrix Morphing Parallel Views Source: Steven M. Seitz, Charles R. Dyer

  12. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Parallel Views Basic Theory image point p = (x,y) scene point P = (X,Y,Z) p = Π P where Π is a homogenous projection matrix Π = [ H | - HC] H: position and orientation of image plane C: euclidean position of the camera Morphing Parallel Views Source: Steven M. Seitz, Charles R. Dyer

  13. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Parallel Views Mathematics for View Interpolation Πs: Linear interpolation of Π0 and Π1 Πs = (1 – s) Π0 + s Π1 s = [0,1] Cs = (sCx, sCy, 0) fs = (1 – f) f0 + s f1 f : focal lengh C : center of a camera

  14. View Interpolation vs View Morphing Comp 767: Advanced Topics in Graphics: Image-Based Rendering View Interpolation Recap View Interpolation Source: 3D Games by Alan Watt and Fabio Policarpo Morphing Parallel Views Source: Steven M. Seitz, Charles R. Dyer

  15. Comp 767: Advanced Topics in Graphics: Image-Based Rendering How to do View Morphing? View morphing in three steps • Prewarp two images • Compute a morph between the prewarped images • Postwarp each in between images produced by the morph View morphing in 1 steps: Assume parallel views! • Compute a morph between the parallel images

  16. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Non-Parallel Views The General Case • This is where the 3-Step Algorithm • comes into play • Prewarping: I0 to Î0 and I1 to Î1 • Morphing: Î0 and Î1 into Îs • Postwarping: Îs into Is Morphing in Three Steps Source: Steven M. Seitz, Charles R. Dyer

  17. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Non-Parallel Views Mathematics for Image Reprojection ^ ĤH-1p = p H and Ĥ are 3x3 matrices that represent the position and the orientation of their image planes The resulting 3x3 matrix, ĤH-1 is a projective transformation that reprojects the image plane I into Î Morphing in Three Steps Source: Steven M. Seitz, Charles R. Dyer

  18. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Limitations • Singular Views cannot be reprojected to form parallel views • Singular configurations are settings where one of the camera resides in the field of view of another camera • Still works, just kind of conceptually hacky Singular view Source: Steven M. Seitz, Charles R. Dyer Parallel view Source: Steven M. Seitz, Charles R. Dyer

  19. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Traditional Problems Change in visibility creates • Folds • Occurs when a visible area becomes occluded • Holes • Occurs when an occluded area becomes visible Area Point Fold Source: Irwin Chiu Hau Penumbra, umbra and hole regions Source: Chen and Williams

  20. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Producing The Morph We have talked about theory behind algorithm • How to project/unproject images to parallel planes • How to warp between parallel planes • Theoretical problems Now, let’s do an actual View Dependent Morph!

  21. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Producing The Morph We need: • Two images I0 and I1 • Two perspective projection matrices Π0 and Π1 • Correspondence between pixels Note that a sequence of projection matrices Πs is required to control the entire animation, but Πs can becomputed automatically if we know Hs .

  22. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Controlling The Morph Hs can be obtained indirectly by establishing constraints • Recall Hs is position and orientation of image plane • One way is to specify four control points Note: Control points implicitly determine the postwarping transformation Four control points form the red bounding box to determine the postwarping stage Yellow lines are set of features to determine the prewarping stage View Morphing Procedure Source: Steven M. Seitz, Charles R. Dyer

  23. Comp 767: Advanced Topics in Graphics: Image-Based Rendering View Morphing sans Prewarping Prewarping is not necessary for: • Objects that aren’t closely related • Prewarping is less effective • Computation is unstable • Images that are approx. orthographic (eg. telephoto) However, postwarping should not be left out to: • Reduce image plane distorsions

  24. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Overview Mona Lisa view morphs Source: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm • What is view morphing? • How to do view morphing? • Results • Conclusion

  25. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Results Facial view morphs Source: Steven M. Seitz, Charles R. Dyer

  26. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Results Facial view morphs Source: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm

  27. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Results Mona Lisa view morphs Source: http://www.cs.washington.edu/homes/seitz/vmorph/vmorph.htm

  28. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Results Image Morphing vs View Morphing Image Morphing vs View Morphing Source: Steven M. Seitz, Charles R. Dyer

  29. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Conclusions Things to remember View Morphing • Powerful extension to image morphing • Produces new views of a scene • 3D shape preserving

  30. Comp 767: Advanced Topics in Graphics: Image-Based Rendering References • View Morphing - Seitz and Dyer, 1996 • View Interpolation - Chen and Williams, 1993 • Image Morphing - Beier and Neely, 1992 • 3D Games: Realtime rendering and Sofware Technology - Alan Watt and Fabio Policarpo

  31. Comp 767: Advanced Topics in Graphics: Image-Based Rendering Questions?

More Related