1 / 44

Paper presentation topics

Paper presentation topics. 1. Segmentation. 2. More on feature detection and descriptors. 3. Shape and Matching. 4. Indexing and Retrieval. 5. More on 3D reconstruction. depth map. 3D rendering. [Szeliski & Kang ‘ 95]. X. z. x. x ’. f. f. baseline. C. C ’. Depth from disparity.

tirza
Download Presentation

Paper presentation topics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Paper presentation topics 1. Segmentation 2. More on feature detection and descriptors 3. Shape and Matching 4. Indexing and Retrieval 5. More on 3D reconstruction

  2. depth map 3D rendering [Szeliski & Kang ‘95] X z x x’ f f baseline C C’ Depth from disparity input image (1 of 2)

  3. Real-time stereo • Used for robot navigation (and other tasks) • Several software-based real-time stereo techniques have been developed (most based on simple discrete search) • Nomad robot searches for meteorites in Antartica • http://www.frc.ri.cmu.edu/projects/meteorobot/index.html

  4. Stereo reconstruction pipeline • Steps • Calibrate cameras • Rectify images • Compute disparity • Estimate depth • Camera calibration errors • Poor image resolution • Occlusions • Violations of brightness constancy (specular reflections) • Large motions • Low-contrast image regions What will cause errors?

  5. Spacetime Stereo Li Zhang, Noah Snavely, Brian Curless, Steven Seitz CVPR 2003, SIGGRAPH 2004

  6. Stereo

  7. ? ? ? Stereo

  8. Marker-based Face Capture The Polar Express,2004 “The largest intractable problem with ‘The Polar Express’ is that the motion-capture technologyused to create the human figures has resulted in a film filled with creepily unlifelike beings.” New York Times Review,Nov 2004

  9. Stereo

  10. Stereo A Pair of Videos 640480@60fps Each Frame-by-Frame Stereo WH = 1515 Window Inaccurate & Jittering

  11. Spacetime Stereo 3D Surface

  12. Spacetime Stereo 3D Surface Time

  13. Spacetime Stereo 3D Surface Time

  14. Spacetime Stereo 3D Surface Time

  15. Spacetime Stereo Surface Motion Time

  16. Spacetime Stereo Surface Motion Time=0

  17. Spacetime Stereo Surface Motion Time=1

  18. Spacetime Stereo Surface Motion Time=2

  19. Spacetime Stereo Surface Motion Time=3

  20. Spacetime Stereo Surface Motion Time=4

  21. Spacetime Stereo Key ideas: • Matching Volumetric Window • Affine Window Deformation Surface Motion Time

  22. Spacetime Stereo Time

  23. Spacetime Stereo Time

  24. Spacetime Stereo

  25. Spacetime Stereo A Pair of Videos 640480@60fps Each Spacetime Stereo WHT = 955 Window

  26. Frame-by-Frame vs. Spacetime Stereo Frame-by-Frame WH = 1515 Window Spacetime Stereo WHT = 955 Window Spatially More Accurate Temporally More Stable

  27. Spacetime Face Capture System Black & White Cameras Color Cameras Video Projectors

  28. System in Action

  29. Input Videos (640480, 60fps)

  30. Spacetime Stereo Reconstruction

  31. Creating a Face Database

  32. … Creating a Face Database [Zhang et al. SIGGRAPH’04]

  33. … Application 1: Expression Synthesis A New Expression: [Zhang et al. SIGGRAPH’04]

  34. … Application 2: Facial Animation [Zhang et al. SIGGRAPH’04]

  35. Keyframe Animation

  36. Some Applications Entertainment: Games & Movies Medical Practice: Prosthetics

  37. Some books on linear algebra Linear Algebra, Serge Lang, 2004 Finite Dimensional Vector Spaces, Paul R. Halmos, 1947 Linear Algebra and its Applications, Gilbert Strang, 1988 Matrix Computation, Gene H. Golub, Charles F. Van Loan, 1996

  38. Multiview Stereo

  39. Choosing the stereo baseline What’s the optimal baseline? • Too small: large depth error • Too large: difficult search problem all of these points project to the same pair of pixels width of a pixel Large Baseline Small Baseline

  40. The Effect of Baseline on Depth Estimation

  41. 1/z width of a pixel width of a pixel pixel matching score 1/z

  42. Multibaseline Stereo Basic Approach • Choose a reference view • Use your favorite stereo algorithm BUT • replace two-view SSD with SSD over all baselines Limitations • Must choose a reference view (bad) • Visibility!

  43. MSR Image based Reality Project http://research.microsoft.com/~larryz/videoviewinterpolation.htm …|

More Related