1 / 24

Multimedia Search and Retrieval

Multimedia Search and Retrieval. Shih-Fu Chang, Qian Huang, Thomas Huang, Atul Puri, and Behzad Shahraray Published as a chapter in Advances in Multimedia: Systems, Standards, and Networks, A. Puri and T. Chen (eds.). New York: Marcel Dekker, 1999. Presented by: Reza Aghaee

blossom
Download Presentation

Multimedia Search and Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimedia Search and Retrieval Shih-Fu Chang, Qian Huang, Thomas Huang, Atul Puri, and Behzad Shahraray Published as a chapter in Advances in Multimedia: Systems, Standards, and Networks, A. Puri and T. Chen (eds.). New York: Marcel Dekker, 1999. Presented by: Reza Aghaee For Multimedia Course(CMPT820) Simon Fraser University March.2005

  2. Agenda • Introduction • Video Segmentation, Indexing and Browsing • Object-based Spatio-Temporal Search • Semantic-Level Content Classification and Filtering • Multimedia Meta Search Engines • Conclusions 2/24

  3. Introduction Applications: • WEB Large-scale Multimedia Search Engines • Media Asset Management Systems • Audio-Visual Broadcast Servers • Personal Media Servers for Consumers 3/24

  4. Video Segmentation, Indexing and Browsing • Hierarchical Segmenting • smaller retrievable data units • Hierarchical Grouping • larger yet meaningful categories • Layers of abstraction • commercials, news stories, news introductions, news summaries of the day 4/24

  5. Video Segmentation, Indexing and Browsing • Low-level segmentation of video streams • Streams are segmented into shots, clips and key frames. • They Do not correspond to the real semantic structure • Large amount of low-level structures, hence, browsing inefficiency 5/24

  6. Video Segmentation, Indexing and Browsing Semantic Segmentation of news programs 6/24

  7. Video Segmentation, Indexing and Browsing • Relationship among semantic structures 7/24

  8. Video Segmentation, Indexing and Browsing • Representation & Browsing Tools • Time lined presentation 8/24

  9. Video Segmentation, Indexing and Browsing • Representation & Browsing Tools • -Visual Pres. For stories about E1 Nino 9/24

  10. Video Segmentation, Indexing and Browsing • Representation & Browsing Tools • -Visual Pres. For stories about E1 Nino 10/24

  11. Object-based Spatio-Temporal Search & Filtering • Query by: • Example • Meaningful real world objects • Low-level image regions with uniform features • Feature • Color, Texture, Shape, Motion, Spatio-temporal structure of image regions • Sketches • Users directly draw visual sketches 11/24

  12. Object-based Spatio-Temporal Search & Filtering • Object-oriented search by feature and sketches • (a) & (c) are sketches by the user • (b) & (d) are returned as results 12/24

  13. Object-based Spatio-Temporal Search & Filtering • VideoQ search system • Video decomposed into shots • Shot separation achieved by scene change detection • Salient video regions and objects extracted • Temporal attributes of regions are indexed 13/24

  14. Object-based Spatio-Temporal Search & Filtering • Query processing architecture 14/24

  15. Object-based Spatio-Temporal Search & Filtering • Interface of AMOS semantic object search engine 15/24

  16. Semantic-Level Content Classification and Filtering • Idea is mapping images or videos to meaningful classes • Content modeling using probabilistic graphic models • Multiject (multimedia object) • Has a label • Summarizes the time sequences in from a probabilities, P( label | sequences) 16/24

  17. Semantic-Level Content Classification and Filtering • Multiject categories • Sites, Objects, Events • Multiject Lifetime • Duration of multimedia input used to determine its probability • Multinet (multiject network) • Represents probabilistic dependencies between multijects 17/24

  18. Semantic-Level Content Classification and Filtering • A multinet describes probabilistic dependencies between multijects 18/24

  19. Semantic-Level Content Classification and Filtering • Indexing Multimedia with semantic templates (STs) • Use a set of successful queries instead of a single one • There could be audio or video templates or both Semantic Visual Templates 19/24

  20. Semantic-Level Content Classification and Filtering • Components in development of STs • Generation • Used to generate STs for each semantic concept • Metric • Used to measure the fitness (similarity) of each ST • Applications • Used to develop a library of semantic concepts to facilitate video query 20/24

  21. Semantic-Level Content Classification and Filtering Semantic Template Development - Slalom 21/24

  22. Meta Search Engines • Gateways Linking users transparently to multiple search engines 22/24

  23. Meta Search Engines • Basic Components • Query dispatcher • Selects target search engines by performance scores • Query translator • Translate query to a suitable script for the target • Display interface • Merges the results of each engine using performance scores 23/24

  24. Conclusions • Semantic segmentation instead of low-level segmentation • Methods of semantic segmentation • Object-based semantic searches • Probabilistic models and template-based searches • Meta search engines architecture 24/24

More Related