1 / 16

M. Gelautz, E. Stavrakis, and M. Bleyer Interactive Media Systems Group

Stereo-based Image and Video Analysis for Multimedia Applications Application: “Computer-generated Stereoscopic Paintings“. M. Gelautz, E. Stavrakis, and M. Bleyer Interactive Media Systems Group Technical University Vienna, A-1040 Vienna, Austria Email: gelautz@ims.tuwien.ac.at.

kamal-roberson
Download Presentation

M. Gelautz, E. Stavrakis, and M. Bleyer Interactive Media Systems Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stereo-based Image and Video Analysis for Multimedia ApplicationsApplication: “Computer-generated Stereoscopic Paintings“ M. Gelautz, E. Stavrakis, and M. Bleyer Interactive Media Systems Group Technical University Vienna, A-1040 Vienna, Austria Email: gelautz@ims.tuwien.ac.at

  2. Artistic Rendering of Stereo Views • Our goal is to provide stereo views of real scenes a hand-painted appearance. • The work combines photogrammetric techniques (stereo) with computer graphic algorithms. • Available painterly rendering algorithms have been designed for single images – painterly rendering of stereo views presents a new topic of research!

  3. Traditional Stereoscopic Painting Fig1. Example of a stereoscopic painting by Salvador Dalí (“The sleeping smoker”, 1972/73).

  4. Motivation for Computer-generated Stereo Painting • The manual creation of stereo paintings is a labor intensive task. • The artist needs to reproduce the same composition twice from different viewpoints. • Some painters used stereo photography to base their compositions on. • The excessive effort associated with the manual creation of stereoscopic paintings can be reduced by computer vision techniques.

  5. Single Image Painterly Rendering Hertzmann, A., 1998: “Painterly rendering with curved brush strokes of multiple sizes” paint spilling (a) Original image (b) Rendered image (impressionist style)

  6. Requirements of Stereoscopic Painterly Rendering • Observation: Individual painting of the left and right stereo image will usually not produce satisfactory results. • Requirements: • To preserve coherence between the brush strokes of the two images. • To preserve the depth discontinuities in order to prevent „paint spilling“. • To deal with occlusions. • Solution: We incorporate a stereo-derived depth map into the painting algorithm.

  7. Stereo Matching Algorithm • “A Layered Stereo Matching Algorithm Using Color Segmentation and Global Visibility Constraints“ • Key features: • Color segmentation of the reference image. • Color segments are clustered into more robust depth layers (mean shift clustering). • Iterative refinement of the solution based on a cost function (greedy search algorithm). • Results: 2nd rank (among 30 algorithms) on the Middlebury Stereo Evaluation Website (http://www.middlebury.edu/stereo)

  8. Clustering of Color Segmentsinto Layers Fitting of planar models (depth layers) to segments (top) and layers (bottom) -> layer fitting is more robust than segment fitting!

  9. Stereo Matching Results Sawtooth stereo pair (Middlebury website) Left image Right image Evaluation: 0.2% „bad“ pixels (i.e., unoccluded pixels whose absolute disparity error is greater than 1) Stereo disparities Ground truth

  10. Stereo Painting Algorithm (1) • Modify Hertzmann‘s single image painting algorithm to prevent brush strokes from crossing depth discontinuities. • Paint the reference image using the modified single image painting algorithm. • Paint (dilated) occluded regions of the second image. • Project paint from the reference image to non-occluded regions of the second image. As a consequence, we avoid repainting large parts of the second image.

  11. Stereo Painting Algorithm (2) Original image Stereo disparities Occlusion map Occlusion paint

  12. Stereo Painting Result (1) Single image painting results Left image Right image Stereo image painting results Left image Right image

  13. Stereo Painting Result (2) Original stereo pair Painted stereo pair

  14. Stereo Painting Result (3) Creating the paintings…

  15. Summary and Outlook • We have presented • a new stereo matching algorithm that relies on clustering color-segmented regions into depth layers. • an algorithm for stereoscopic painterly rendering based on real image pairs. • We plan to use the stereo-derived depth maps to enrich the computer-generated paintings with further depth cues (e.g., different levels of detail).

  16. Acknowledgements • This work was supported by the Austrian Science Fund (FWF) under project P15663. • We wish to thank “Verwertungsgesell-schaft Bildender Künstler (VBK)“, Austria, for permission to reproduce „The sleeping smoker“.

More Related