1 / 21

On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras

I. I. I. I. I. I. I. I. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. B. Client i. Vide frames. Server side. Live video. Client i.

rana-vance
Download Presentation

On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. I I I I I I I I B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B Client i Vide frames Server side Live video Client i Sphere wrapping Image alignment Client i User request [area i, at time t1~tk] 1 Rendering, Display i1 Client i im N 1 … … I B B B Decoding and composing panorama i1 … … im B B B … … N Time k CPSC 643 On-Demand Sharing of a High-Resolution Panorama Video from Networked Robotic Cameras Dezhen Song Texas A&M University Supported in part by

  2. Network PTZ Robotic Camera for Nature Observation • Panosonic HCM 280 • PTZ Robotic Camera: • 350° Pan, 120° Tilt, 42x Zoom • Maximum spatial resolution: 500 Megapixel per steradian • 3 Gigapixels panorama • Network Video Camera: • Built-in streaming server • 640x480 pixels video • >30 frames per second

  3. Fixed lens with mirror 10M Pixel CCD $ 20.0 K 2M Pixel / Steradian Pan, Tilt, Zoom (21x) 0.37M Pixel CCD $ 1.2 K 500M Pixel / Steradian Giga-pixel Motion Panorama VS. Fixed Lens Camera

  4. Existing Panoramic Video Systems

  5. Panorama Tilt Pan Frame sequence Tilt Updated Part in Panorama Panorama Live frame sequence Time Evolving Panorama: High Resolution Live Panoramic Video Using PTZ Camera

  6. Collaborative Observatories for Natural Environments (www.c-o-n-e.org) sensor networks humans: amateurs and profs. timed checks robotic video cameras 2005-2008 motion sensors

  7. I I I I I I I I B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B Vide frames Server side Live video Image alignment Sphere wrapping Client i 1 Client i Client i User request [area i, at time t1~tk] i1 Client i Rendering and Display im N 1 … … I B B B i1 Decoding and composing panorama … … im B B B … … N Time k • On-demand Panoramic Video Sharing

  8. On-demand Panoramic Video Sharing • Challenges: • Dynamic video coverage • High resolution panorama coverage • Multiple different spatial-temporal client requests.

  9. Live Live Client i … … … … k-1 k Time Live video User Request User i request: ri=[u, v, w, h, ts, te]

  10. Patch-based Panorama Video Live patch Static patch 60o 1 pjk Camera Coverage tilt Camera Coverage Camera Coverage Snapshot at time k N pan -180o -180o

  11. Live … … … … k-2 k-1 k Time Patch-based Panorama Video Live video Patch j at time k Camera coverage at time k

  12. I I I I I I I I B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B B Vide frames Server side Live video Image alignment Sphere wrapping Client i 1 Client i Client i User request [area i, at time t1~tk] i1 Client i Rendering and Display im N 1 … … I B B B i1 Decoding and composing panorama … … im B B B … … N Time k On-demand Patch-based Panorama Video Sharing

  13. Frame Insertion Algorithm • Input: Ft • Output: Updated evolving panorama video • Wrap Ftonto the spherical surface; • Estimate Ft’s registration parameters by aligning it with previous frames; • Project Ft onto the sphere panorama surface; • foreach pj and pj∩Ft≠ Ødo • Insert pjt into pj’s GOP buffer; • foreach pj, j=1, …,Ndo • ifpj’s GOP buffer is full then • Encode patch video segment; • Store patch segment start position and time data into lookup table; • Reset GOP buffer for incoming data;

  14. On-demand Patch-based Panorama Video Sharing For User i request: ri=[u, v, w, h, ts, te] Send patch data: ri ∩ Pt = { pjk | j Є{1,…,N}, k Є[ts, te], pjk ∩ ri≠Ø , pjk≠Ø}

  15. User Query Algorithm • Input: ri • Output: ri∩ P in MPEG-2 format • Identify patch set S= { pj | j Є { 1,…,N }, pj ∩ ri ≠Ø }; • foreachpjЄ Sdo • Find the nearest I frame pjb earlier or equal to ts; • Find the nearest I frame pjc later or equal to te; • Transmit the patch segments between pjb and pjc;

  16. Experiments and Results • Hardware configuration: • Dell Dimension DX, 3.2Ghz Pentium dual-core processor, 2GB RAM • Panasonic HCM 280A video camera • Software configuration: • Visual C++ in Microsoft Visual Studio 2003 .NET • MPEG-2 encoder/decoder from MPEG Software Simulation Group • Input data set: • Frame number: 609 • Frame resolution: 640x480 pixels • Frame rate: 25 fps • Raw RGB data size; 536 MB • Panorama resolution: 2742x909 pixels

  17. Experiments and Results Storage and computation speed versus different patch sizes:

  18. Experiments and Results Bandwidth for a user query (800x600 pixel) versus different patch sizes:

  19. Summary Patch-based data representation and encoder provides on-demand sharing of a high resolution panoramic video from networked robotic Pan-Tilt-Zoom cameras with: • Effective data organization • Efficient data storage. • Satisfy spatial-temporal user video requests.

More Related