1 / 23

Visual Information Retrieval

Visual Information Retrieval. Alberto Del Bimbo Dipartimento di Sistemi e Informatica Universita di Firenze Firenze, Italy. Chapter 1 Introduction. Visual Information Retrieval.

fidella-lerato
Download Presentation

Visual Information Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Visual Information Retrieval Alberto Del Bimbo Dipartimento di Sistemi e Informatica Universita di Firenze Firenze, Italy Chapter 1 Introduction

  2. Visual Information Retrieval • Information retrieval, image/video analysis and processing, pattern recognition and computer vision, visual data modeling and representation, multimedia database organization, multidimensional indexing, psychological modeling of user behavior, man-machine interaction and data visualization

  3. Visual Information Retrieval • Types of associated information • content-independent metadata (CIM) • format, author's name, date • content-dependent metadata (CDepM) • low-level features concerned with perceptual facts: color, texture, shape, spatial relationship, motion • content-descriptive metadata (CDesM) • high-level content semantics: cloud, good weather, 白雲蒼狗

  4. Visual Information Retrieval • First-generation visual information retrieval systems • CIM by alphanumeric strings, CDepM and CDesM by keywords or scripts

  5. Visual Information Retrieval • find images of paintings by Chagall with a blue background • Select IMAGE# from PAINTINGS where PAINTER = "Chagall" and BACKGROUND = "blue" • find images of paintings by Chagall with a girl in red dress and a blue background • full text retrieval

  6. Visual Information Retrieval • find images of paintings depicting similar figures in similar positions as in 收割景緻 • it is difficult for text to capture the perceptual saliency of some visual features • text is not well suited for modeling perceptual similarity • perception is mainly subjective, so is its text descriptions

  7. Visual Information Retrieval • New-generation visual information retrieval systems • retrieval not only by concepts but also by perception of visual contents • objective measurements of visual contents and appropriate similarity models • automatically extract features from raw data by image processing, pattern recognition, speech analysis and computer vision techniques

  8. Visual Information Retrieval

  9. Visual Information Retrieval • Image retrieval • by perceptual features • for each image in the database, a set of features (model parameters) are precomputed • to query the image database • express the query through visual examples • authored by the user • extracted from image samples • select features and ranges of features • choose a similarity measure • compute similarity degrees, ranking, relevance feedback

  10. Visual Information Retrieval • system architecture • extraction of perceptual features (CDepM) • extraction of high-level semantics (CDesM) from low-level features • manual annotation of CIM and CDesM • index structure • graphical query tool • retrieval engine • visualization tool • relevance feedback mechanism

  11. Visual Information Retrieval • Video retrieval • special characteristics • frames are linked together using editing effects • color, texture, shape and position (camera or object) are changed in multiple frames • richer semantics • different types of video

  12. Visual Information Retrieval • by structure • Figure 1.4 • frame: basic unit of information • shot: elementary segment of video with perceptual continuity • clip: set of frames with some semantic meaning • scene: consecutive shots with simultaneous space, time and action • episode: specific sequence of shot types such as a news episode

  13. Visual Information Retrieval • by content • perceptual properties, motion and type of an object • situations between objects • motion of camera • semantics of shots by color- or motion-induced sensations • semantics of scenes • stories • audio properties: dialogue, music or storytelling • textual information: caption or text recognized from video

  14. Visual Information Retrieval • system architecture • extraction of shots and the associated semantics, key-frames or mosaics • extraction of scenes and stories • manual annotation tool • browsing/visualization tool • video summarization • graphical query tool • index structure • retrieval engine

  15. Visual Information Retrieval • 3D image and video retrieval • WWW visual information searching • efficiency has to be emphasized due to limited network bandwidth • operate in compressed domain • visual summarization • visualization at different levels of resolution

  16. Visual Information Retrieval • Research directions • tools for automatic extraction of low-level features • tools for automatic extraction of high-level semantics • models for representing visual content • effective indexing • effective database models

  17. Visual Information Retrieval • visual interfaces • allow querying and browsing • allow querying by text and visual information • similarity models • fit human similarity judgement • psychological similarity models • Web search • 3D image and video retrieval

More Related