1 / 25

Internet Resources Discovery (IRD)

Internet Resources Discovery (IRD). Introduction to MMIR. Contents. Visual Information Retrieval (VIR) Images Video Video Information Retrieval (VIR) Music Information Retrieval (MIR). Visual Information Retrieval. Introduction VIR system VIR information domains Querying video

creda
Download Presentation

Internet Resources Discovery (IRD)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Resources Discovery (IRD) Introduction to MMIR T.Sharon

  2. Contents • Visual Information Retrieval (VIR) • Images • Video • Video Information Retrieval (VIR) • Music Information Retrieval (MIR) T.Sharon

  3. Visual Information Retrieval • Introduction • VIR system • VIR information domains • Querying video • Advanced topics T.Sharon

  4. Introduction • What is VIR? • Who needs it? • Questions and problems T.Sharon

  5. What is VIR? Query VIRSystem VIR allows users toquery and retrieve visual information. Queries will be doneaccording to informationcontent. T.Sharon

  6. Who needs VIR? • Libraries • Museums • Scientific Archives • Image Warehouses T.Sharon

  7. Questions and Problems • How can we search visual information? • How can visual and non-visual information can be searched together? • Problems: • visual information is subjectively interpreted. • few representations: images, graphics, video, animations, stereoscopic images. • requires substantial amount of resources. Dog?? T.Sharon

  8. VIR System • Architecture • Query formulation • Match and ranking • Query answer • Refinement (relevance feedback) T.Sharon

  9. Architecture of VIR System T.Sharon

  10. Query Formulation • Query by example: • sketch an example • give an image • Query by giving values to visual features: • % colors • texture • describe textuallybut use visual tools to define values. T.Sharon

  11. Query matching and ranking • Similarity test • Using combination of features, for example: • colors • texture • shape • motion • additional information • Actions in feature space can be: • maximal distance • K nearest neighbors Feature 1 . . . Feature 3 . . . . . . . . . . Feature 2 T.Sharon

  12. Query Answer • Thumbnails: • Images • DC images • Video • built fromselected DC images (key frames) T.Sharon

  13. Query Refinement • Using a result image from previous query. • Launch a new query. • Modifying a result image with an image processing tool to specify an an additional criteria. • Changing relative weights of visual features and get a new ranking to the previous results. T.Sharon

  14. VIR Information Domains • Information domain • Queries at Pixels Level • examples • problems • Implementations • color • color complex • shape T.Sharon

  15. Information Domains • Metadata information • alphanumeric, database scheme • Visual characteristic • contained in the object • achieved by using computational process, usually image processing T.Sharon

  16. Queries at Pixels Level - Examples • Find all objects for which the 100th to 200th pixels are orange (RGB=255,130,0). • Find all the images that have about the same color (certain RGB) in the central region (relative or absolute). • Find all images that are a shifted version of this particular image, in which the maximum allowable shift is D. T.Sharon

  17. Queries at Pixels Level -Problems • Pixel queries are noise sensitive • couple of noise pixels can cause to discard a good image. • Do not work on rotations. • Changes in lightning and imaging conditions effect pixels significantly and bias queries. T.Sharon

  18. Implementations • Pixels location combined with • Database scheme built by humans • Example techniques: • Color • Color complex • Texture • Shape T.Sharon

  19. Color • Method: • Color definition • Hue (color spectrum) • Saturation (gray) • Calculate histograms • Enables queries: • Find all images for which more than 30% is sky blue and more than 25% is grass green • Sort histogram drawers, find 5 most frequent colors, find all other images with these color features • Find all images far from this image only D T.Sharon

  20. Color Complex • Method: Create histograms quad-tree: • Calculate image histogram • Divide image to quarters and calculate histogram for each quarter • Continue recursively till 16x16 squares • Enables queries: • Find images for which: • more than 20% red-orange pixels in the right upper quarter • more than 20% yellow pixels in the left upper quarter • about 30% brown pixels in the bottom half of the picture • Find all images with red patch in the middle and blue patch around. T.Sharon

  21. Shape • Method: • Suppose we have graphics collection (clip arts) • contain pure colors (little hue changes, no saturation) • Divide image to color areas so that each area contains pixels with the same pure color • Calculate features for each area: • color, area, elongation (sqrt(perimeter)/area), centrality (distance of shape centroid to image center) • Enables queries: • Find all images containing white squares in the center • Find all images containing 2 blue circles close to the center T.Sharon

  22. Examples: Existing Systems • SaFe http://disney.ctr.columbia.edu/SaFe/ • Virage http://www.virage.com/virdemo.html • QBIC http://wwwqbic.almaden.ibm.com/ (stamps) download! http://wwwqbic.almaden.ibm.com/cgi-bin/pcd-demo/drawpicker (photos) • MetaSEEK http://mahler.ctr.columbia.edu:8080/cgi-bin/MetaSEEk_cate • WebSEEK • VisualSEEK • MELDEX http://www.nzdl.org/cgi-bin/gw?c=meldex&a=page&p=coltitle T.Sharon

  23. SaFe T.Sharon

  24. QBIC - Histogram Query T.Sharon

  25. QBIC - Color Layout Query T.Sharon

More Related