1 / 1

Light Field Video Stabilization

Light Field Video Stabilization. Insight : Only Need Relative Transformation R f , t f. This work is sup- ported in part by Adobe System Inc. & NSF IIS-0845916. Relative Transformation. , Z f-1. p f-1. p f+1. , Z f+1. , Z f. p f. Brandon M. Smith, Li Zhang

fritz
Download Presentation

Light Field Video Stabilization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Light Field Video Stabilization Insight: Only Need Relative Transformation Rf, tf This work is sup- ported in part by Adobe System Inc. & NSF IIS-0845916. Relative Transformation , Zf-1 pf-1 pf+1 , Zf+1 , Zf pf Brandon M. Smith, Li Zhang Computer Sciences, UW-Madison Hailin Jin, AseemAgarwala Adobe Systems, Inc. • Stabilization without structure from motion R,t • Use fewer cameras (two instead of five) Original Camera Canny Edge Maps • Can handle challenging cases sensor stabilization • Motion deblurring with camera arrays UW-Madison Graphics & Vision Group Input Images Depth Maps reproj(pf-1, Zf-1,Rf-1,tf-1) reproj(pf+1, Zf+1,Rf+1,tf+1) reproj(pf, Zf,Rf,tf) Panasonic HD Stereo Camcorder Viewplus Profusion 25C • Better handle image periphery problems Virtual Camera Path Virtual Camera Path 4. Run spacetime optimization to find {R,t}f=1,…,F • Evaluate a range of camera baselines Existing applications qf-1 New View Synthesis qf qf+1 Actual Camera Path Actual Camera Path • New-view synthesis [Levoy & Hanrahan SIGGRAPH ‘96, Gortler et al. SIGGRAPH ‘96] Algorithm Outline Motivation Videos shot with a hand-held conventional video camera often appear very shaky. This shakiness is often the most distracting aspect of amateur videos that easily distinguishes them from more professional work. Spacetime optimization: Maximize smoothness of virtual video as function of {Rf, tf}f=1…F Synthesize a video along a virtual smooth camera path • Synthetic aperture[Wilburn et al., SIGGRAPH 2005] Virtual Camera Vibration Reduction (VR) Optical Flow [Bruhn et al. 2005] • Noise Removal [Zhang et al., CVPR 2009] Synthesized Image optical stabilization F-1 qf,j- -qf-1,jprev+ qf+1,jnext 2 | | 1 ( ( ) ) å å F-1 • {R,t}f=1,…,F = wf,j E • +Ereg 2 Advantage: Do not need to compute 3D input camera path Compute depth map for each time instant[Smith et al. 2009] f=2 j New application f Îf 2 | | å å 1 ( ) ) ( qf,j- -qf-1,jprev+ qf+1,jnext Video Stabilization = E • {R,t}f=1,…,F wf,j • +Ereg • Video Stabilization 2 f=2 j Îf Professional Solutions Consumer Solutions 2. Compute optical flow for each time instant[Bruhn et al. 2005] 3. Detect Canny edges, use flow to match edges over time | | • 2D-transformation based methods • Burt & Anandan, Image Stabilization by Registration to a Reference Mosaic. DARPA Image Understanding Workshop, 1994 • Hansen et al., Real-Time Scene Stabilization and Mosaic Construction. DARPA Image Understanding Workshop, 1994 • Lee et al., Video Stabilization Using Robust Feature Trajectories. ICCV 2009 5. New view synthesis • Cumbersome State-of-the-art New Approach • Rotational camera motion • Small baseline … camera crane steadicam camera dolly New View Synthesis Use special hardware to avoid camera shake • Expensive • Distant scenes • Limited DOF Why Does a Camera Array Help? How to Avoid Structure from Motion? Liu et al., SIGGRAPH 2009 Results Works well if structure from motion is successful • Background has enough visual features • Small dynamic targets Stabilization as image based rendering[Buehler et al. CVPR 2001] How to Define the Smoothness of a Video? Matching Salient Features • More input views at each time instant • Easier to work with dynamic scenes • Better handling of parallax Summary of Contributions Future Work | | • Use an array for stabilization • Nearby, dynamic targets • Increase algorithm efficiency • Large scene depth variation • Violent camera shake

More Related