1 / 32

Efficient Video Browsing

Efficient Video Browsing. Using Multiple Synchronized Views Heymo Kou. Question. What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content). Table of contents. Background Current technology Advanced technology Summary Reference.

odette
Download Presentation

Efficient Video Browsing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Efficient Video Browsing Using Multiple Synchronized Views Heymo Kou

  2. Question • What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content)

  3. Table of contents • Background • Current technology • Advanced technology • Summary • Reference

  4. Growth of digital contents data

  5. Digital video market growth

  6. Digital video becomes ubiquitous • From your • Smart phones • Notebooks • Webcams • Digital camera and camcorders • Security and monitoring cameras • With advanced streaming technology • Fast Internet access • MPEG-4 format

  7. Current technology for finding a video data • Search through categories • Similar to Internet shopping mall • We search for big categories • Then smaller categories • …and so on… • User should choose which to browse • Should check whether the selected data matches what user was finding • Time consuming! • Manual categorizing and annotation • One by one?

  8. Problem with current video search and browsing technologies • Too complicated • Lack of efficient algorithm • Time consuming • Multimedia calculation ∝ exponential • Inaccuracy • Video data is increasing exponentially • Cataloging manual has a somewhat limit point • Manually cataloging is done by human hand that mistakes can be happened

  9. Technologies for advancedimage and video retrieval • MPEG-7 Standards • Speech indexing • Shot Boundary Detection • Time Scale Modification of Audio Signals • Storyboards, Moving Storyboards and Animation • Adaptive Accelerating Fast Playback • Streaming Synchronized Views

  10. MPEG-7 standard • Standardized by ISO/IEC • International Standard Organization • International Electrotechnical Commission • Not a video encoding format • XML to store metadata • Attached to timecode in multimedia • By this tag • Able to index and search efficiently • Yet, improvement is needed

  11. Speech indexing • Search through speech transcripts • Finds familiar metaphor of free text search • Automatic speech recognition (ASR) • Indexed transcript → semantic information • Main advantage : Representation • Speech is built of words

  12. Definitions • Frame • Key frame • Shot • Group of frameswhich represents similar frames Start key frame end key frame animation

  13. Definitions • Context • Meaningful information within multimedia data • 3 levels of video browsing • Browsing a large collection of videos • Browsing a ranked list of videos • Browsing a single video to find relevant segments

  14. Shot Boundary Detection • Shot Boundary Detection(SBD) algorithm • Completely automatic • Key frames are selected and extracted • Saved as JPEG files • High Accuracy and Efficiency • Still, fault detection problem is unsolved

  15. SBD algorithm

  16. Similar to scene selection of dvd

  17. Time Scale Modification ofAudio Signals • Audio browsing is as important asvideo browsing • Except images, most digital contents are audible • Faster audio browsing is necessary • Speeding up of audio signal by • By deleting small audio segments • Especially, human speech signals are quasi-periodic

  18. Improvement of TSM Time-Domain Harmonic Scaling(TDHS) technique Time-Domain, Pitch Synchronous Overlap Add Time Scale Modification (TSM) algorithm Waveform Synchronous Overlap(WSOLA)

  19. Synchronous Overlap-Add SOLA

  20. Storyboards, Moving Storyboards and Animation • Storyboard • a set of one or more pages, each consists of a two dimensional array of key-frames, sorted in chronological order. • Animation • a quick slide show, where each of the key-frames is shown for a fixed short period (e.g., 0.6 seconds) • Moving Storyboard (MSB) • the animated key frames, fully synchronized with the original audio track. Each key-frame is shown for the entire duration of the associated shot.

  21. Adaptive Accelerating Fast Playback • Very fast video playback (without audio) • Ordinary fast forward depends only on speed • There is a chance to miss important scene • Accelerates until new scene is met • Requires less computation load

  22. Image for adaptive fast playback

  23. Real-use of adaptive fast playback • Example in surveillance camera

  24. Streaming Synchronized Views • Server preprocesses media • Keep same media, but different speed encoded • When user selects other speed • 1. pause current media • 2. open file with same content with selected speed • 3. seek to the corresponding position • 4. play the selected view • Needs no extra computational load • However, requires more storage: Tradeoff

  25. Browsing Multiple Videos: MovieDNA • Can browse multiple videos at once • Split frames every given time • (i.e 10 seconds) • Strong information scent is visible • With aggregation of occurrences

  26. Image of movieDNA

  27. Summary ofmain properties

  28. Conclusion • Streaming synchronized views and movieDNA • Less computation, multiple videos at once • Active accelerating fast playback • Most useful at analyzing surveillance videos • SBD & TSM • Efficient for implementing above technologies • Then, what is current limitation?

  29. Q & A Any questions?

  30. Answer • What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content) • Answer : The two main technologies are Shot Boundary Detection(SBD) for visual content and Time Scale Modification(TSM) for audio signals

  31. References (1/2) • Shot Boundary Detection • http://muvis.cs.tut.fi/sbd.html • Key frame • http://en.wikipedia.org/wiki/Key_frame • Synchronous Overlap-Add • http://www.surina.net/article/time-and-pitch-scaling.html • Digital Video Market Growth • http://articles.businessinsider.com/2011-12-13/research/30508929_1_fios-cable-providers-video-streaming-service/2 • Amount of Digital data • http://www.emc.com/collateral/analyst-reports/diverse-exploding-digital-universe.pdf

  32. References (2/2) • Streaming synchronized view • http://www.visus.uni-stuttgart.de/forschung/visualisierung-und-visual-analytics/visuelle-analyse-videostroeme/adaptive-fast-forward-for-video-surveillance.html • MovieDNA • http://homepage.mac.com/juggle5/WORK/publications/HICSS34.pdf

More Related