1 / 12

Simultaneous super-resolution and 3D video using graph-cuts

Simultaneous super-resolution and 3D video using graph-cuts. Tony Tung Shohei Nobuhara Takashi Matsuyama Graduate School of Informatics, Kyoto University, Japan CVPR 2008. Outline. Introduction Super-resolution on multi-view images Super-resolution on single-view video frames

arav
Download Presentation

Simultaneous super-resolution and 3D video using graph-cuts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Simultaneous super-resolution and 3D video using graph-cuts Tony Tung ShoheiNobuhara Takashi Matsuyama Graduate School of Informatics, Kyoto University, Japan CVPR 2008

  2. Outline • Introduction • Super-resolution on multi-view images • Super-resolution on single-view video frames • Simultaneous super-resolution and 3D video • Results • Conclusion

  3. Introduction • In 3D video • models are in motion • limited number of cameras • details cannot always be recovered • mapped textures may lack of precision • This paper • Super-resolution(SR) for video frames • Markov Random Field(MRF) energy formulation • Graph-cut • Coarse-to-fine strategy

  4. Super-resolution on multi-view images • Super-resolution (SR) • to recover detailed information from degraded data • one can reconstruct high resolution (HR) images from several low resolution (LR) images • SR image reconstruction method • 1) magnifying every image from LR to HR • 2) accurate alignments of LR images onto HR grids to gain subpixel information in each HR image • 3) HR image regularization using MRF energy formulation and minimization with graph-cuts

  5. MRF energy formulation • enable discontinuity preservation and image regularization using graph-cuts • one energy is formulated for each R, G, B channel

  6. Super-resolution on single-view video frames • Feature matching • tracking regions of interest(ROI) in consecutive frames • the Scale Invariant Feature Transforms (SIFT) detector • Texture warping and mapping • create a mesh based on the detected features • ROI is then the mesh texture

  7. Simultaneous super-resolution and 3D video • Coarse-to-finestrategy

  8. Simultaneous energy minimization • the global compound energy • simultaneous SR • 3D shape reconstruction using graph-cuts

  9. Results • The studio • diameter is 6 m, height is 2.5 m • an object can be reconstructed without defect is approximately 3m*3m*2m • PC cluster system composed of 15 PCs and one master PC • multi-viewpoint images at 25 fps • Japanese dance performed by maikos • wears a kimono

  10. 3D model reconstruction is challenging • cloths are large • non-rigid and contain lots of details to recover • The whole pipeline is fully automatized • The most expensive step is the SR energy minimization

  11. Conclusion and further work • 3D video is a new media developed these last years to represent 3D objects in motion • Super-resolution methods dedicated to 3D video • May be combined to a 3D video compression technique • Could tune the coarse-to-finestrategy to update iteratively the texture map

More Related