1 / 34

Video Information Retrieval

Video Information Retrieval . Mark Ruzomberka IST 497 11/07/02. Joke. Outline . What is Video Information Retrieval (VIR) ? Reasons VIR is necessary Theoretical Where we are today Examples Problems Future Work Conclusion. What is Video Information Retrieval (VIR) ?.

suchi
Download Presentation

Video Information Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Video Information Retrieval Mark Ruzomberka IST 497 11/07/02

  2. Joke

  3. Outline • What is Video Information Retrieval (VIR) ? • Reasons VIR is necessary • Theoretical • Where we are today • Examples • Problems • Future Work • Conclusion

  4. What is Video Information Retrieval (VIR) ? • Recognition technologies • Image • Voice • Text transcripts • Document retrieval technologies • Topic segmentation • Topic matching • Text summarization • Presentation Technologies • Combine Recognition and retrieval technologies • Result is an integrated application

  5. VIR-Need, or Why do I care? • Consider the task of trying to find a five minute video clip of interest in a library of 1000 hour long tapes. • Consider the “go to the part where” problem

  6. What do people want from IR D-Lib Magazine’s asks: “What do People want from Information Retrieval?” # 8 Multimedia

  7. Specificly, Reasons for Video IR • Reading is slow compared to your potential for understanding information • Humans think in pictures not words • Reading is particularly slow on a computer screen • Example: Daydreaming while some one is talking • Reading a page in a book and not remembering what it was about

  8. VIR makes for quicker human understanding. • Palm/Grafitti 25 • Hand Writing 35-40 • Typing 50-70 • Speaking 135-175 • Reading 200 • Listening 400 - 500 • Thinking 500+ Video IR allows for faster access to information

  9. Theoretical: • Think of the “Jetsons mail system” • You “talk” to the computer, • Computer intelligently “talks” back to you

  10. Where we are today • Two of Video Information Retrieval System are currently available: • Type One- keyword/text based • Type Two- Content based

  11. Type One- keyword/text based DVR- basic expansion of image IR, not as interesting

  12. Type Two- Content based MSR Video Skimmer Video Mail Informedia

  13. Example: Video Mail • University of Cambridge • 1994-1996 • AT&T • 1999 • 2000-project ended

  14. Video Mail: Medusa network • Medusa multimedia environment at Olivetti Research Ltd. In Cambridge • It takes a modular approach unlike that of a pc or workstation • Unified by a common interface to ATM network • Devices plug directly into network and include: • Cameras • Audio devices • Networked frame buffers • Processor farms • Disk drives

  15. Video Mail: Medusa Network • “The network is the computer” metaphor is used • Solves storage and network speed problems • Complicates expense problem

  16. How it works-Overview

  17. The Integrated Application • “narrow” by sender,date, time

  18. Video Mail: Video Browser • Content is now being viewed • Keywords are flagged

  19. Video Mail: Video Browser • In the latest version “thumb-nailed” pictures of key frames replace color coded line of the search keyword

  20. Informedia The Informedia Digital Video Library Project automatically combines speech, image and natural language understanding to create a full-content searchable digital video library.

  21. Informedia

  22. Informedia: human factor issues • Interaction • Motivation • Effective usage modes • Commercial compression • VHS quality playback. • Terabyte (1,000 gigabytes) of storage • 1000 hours of video.

  23. Problems • Human understanding • Spoken document retrieval • Poor video browsers • Expensive • Slow access to data • Large amounts of data

  24. Microsoft Research (MSR) Video Skimmer

  25. Enhanced Browser Controls: Time Compression Pause Removal Textual Indices: TOC, Notes Visual Indices Shot Boundary Frames Timeline Markers Jump Control (Back/Next) Microsoft Research (MSR) Video Skimmer

  26. Problem: Poor Content Based Video Browsers • Current VCR model allows for poor navigation • “go the the part where they say” problem

  27. Problem: Expensive • Hard drive space expensive • Video adds to problem • High bandwidth needs are also expensive http://www.littletechshoppe.com/ns1625/winchest.html

  28. Problem: Slow Access to Data • Broadband still not available everywhere • Availability doesn’t mean acceptance • Especially after dot com crash 2000

  29. Problem: Large Amounts of Data • Current Systems use MPEG2 • Newer compression technologies • MPEG 4-DIVX -DVD Quality • Video consumes orders of magnitude more storage than text • MPEG 7 is on horizon

  30. Future Work ? • Sky the limit ? • Sci-Fi the limit ? • Hard Drive Space, Bandwidth are current limitations.

  31. Conclusion • Not yet ready for prime time • Storage and Network Costs decreasing • Success is in day to day usage • Slowly Becoming Mainstream E.x.Tivo • Problems of “real world tests” • Idiot proof • ATM and Medusa aren’t mainstream

  32. Papers • Video Mail Retrieval Using Voice: Report on Keyword.. - Jones, Foote, Jones.. (1994) • What do people want from Information Retrieval?. Croft, Bruce W. D-Lib Magazine. (1995) • Video Skimming for Quick Browsing based on Audio and Image.. - Smith, Kanade (1995) • The VISION digital video library (context) - Gauch, Li et al. – (1997) • Informedia: News-on-Demand Multimedia Information.. - Hauptmann, Witbrock (1997) • M.G. Christel and D.J. Martin, "Information Visualization within a Digital Video Library", J. Intelligent Info. Systems 11(3), (1998), pp. 235-257 • Browsing Digital Video. Li, Gupta, Sanocki et. Al.

  33. Questions?

  34. Joke? • "There are 10 types of people in the world... • those who understand binary and those who don't."

More Related