Discovering Panoramas in Web Videos

Discovering Panoramas in Web Videos ACM Multimedia 2008 Feng Liu1, Yuhen-Hu1,2 and Michael Gleicher1

Outline • Introduction • Video analysis • Discovering panoramas • Panorama synthesis • Experiments • Conclusion

Creating panoramic imagery • STEP 1 image alignment • STEP2 image stitching

Image sources in a video • Not all video has appropriate sources • Not cover a wide field-of-view of a scene • Motion may be randomly • Image quality

Purpose-Discover Panoramas in Video • Three parts • video analysis • panorama source selection • panorama synthesis

Background-image homography transformation

Background-compute homography • Feature matching – SIFT • Compute homography parameters – RANSAC algo • Run k times: • (1)draw n samples randomly • (2) fit parameters Θ with these n samples • (3) for each of other N-n points, calculate its distance to the fitted model, count the number of inlier points, c • Output Θ with the largest c

Example:line fitting n=2 c=3 c=15 …………………

Video analysis (1) • Image homography • Points should match • Measure error distance and give penalty • Moving object detect • For activity synopsis • examining the discrepancy between its local motion vector and the global motion

Video analysis(2) • Visual quality measures Method of [31]Tong et al 04 Method of [35]Wang et al 02 Average differences across block boundaries.

Discovering panoramas(1) • Good panoramas • Good homography between frames • Video have high image quality • Cover a wild field view • Collision • More frame more wild field of view • More frame more accumulate error to degrade quality • , vistual quality • , extent of the scene

Discovering panoramas(2) • Visual quality measure

Discovering panoramas(3) • Scene extent measure Reference Reference

Discovering panoramas(4) • An Approximate Solution Steps • 1.Fetch a segment Sk from pool Sp • 2.Find the scene extent of Sk and corresponding reference frame. • 3.Append the panorama set according to equation(2). until . • 4.If the scene meet , , add remainder to pool Sp. • 5.If pool Sp != Ο , go to loop 1

Discovering panoramas(4) video shot boundary segments divide segments that have too penalty Repeat until done Discard those extent with too little coverage <

Panorama synthesis(1) • Scene panorama synthesis • blending – feathering • median-bilateral filtering

Panorama synthesis(2) • Activity synopsis synthesis Detect Discard Select and composite into scene

Experiments (1) • YouTube Travel and Events category– West Lake http://www.youtube.com/watch?v=6FKCHLfTns8&feature=player_embedded#! • size320 x 240

Experiments (2) • Query panorama from YouTube • 6 query , top 10 videos • 86.7% contain panoramas

Example

Example Notre Dame, Paris

Conclusion • In this paper, we presented an automatic method to discover panorama sources from casual videos. • “Query panoramas from YouTube”supports our proposal of using web videos as panorama source. • More importantly, this method contribute to presenting or summarizing imagery databases using panoramic imageries by mining the possible sources to synthesize the representations.

Reference • [31] H. Tong, M. Li, H. Zhang, and C. Zhang. Blur detection for digital images using wavelet transform.InIEEE ICME, 2004. • [35] Z. Wang, G. Wu, H. Sheikh, E. Simoncelli, E.-H.Yang, and A. Bovik. Quality-aware images. IEEE Transactions on Image Processing, 15(6):1680 -1689,2006. • Original Videos: http://pages.cs.wisc.edu/~fliu/project/discover-pano.htm

Discovering Panoramas in Web Videos