1 / 14

CM613 Multimedia storage and retrieval Content-based image retrieval

CM613 Multimedia storage and retrieval Content-based image retrieval. D.Miller. Fundamental problem of search by machine v. search by a person. Sensory gap Gap between the “object in the world” and its digital representation Semantic gap

janine
Download Presentation

CM613 Multimedia storage and retrieval Content-based image retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CM613 Multimedia storage and retrievalContent-based image retrieval D.Miller

  2. Fundamental problem of search by machine v. search by a person • Sensory gap • Gap between the “object in the world” and its digital representation • Semantic gap • Gap between information that a machine extract from the image and what the the image means to someone.

  3. Heirinymus Bosch: The Garden of Earthly Delights (detail from the panel “Hell”) • Hieronymus, or Jerome, Bosch, b. c.1450, d. August 1516, spent his entire artistic career in the small Dutch town of Hertogenbosch, from which he derived his name. • At the time of his death, Bosch was internationally celebrated as an eccentric painter of religious visions who dealt in particular with the torments of hell.

  4. Statistical Techniques: colour histogram • Basically just a count of how many pixels of each (R, G and B) value are present • Positive • Not sensitive to orientation • Negative • Spatial information is lost. http://www.cultivate-int.org/issue7/artiste/

  5. Statistical Techniques: Colour layout • Divides an image into a grid and searches for similar colours in equivalent grid position. • E.g blue sky

  6. Statistical Techniques: texture • Using orientation and spacing of edges • Also a histogram of edge features can be used • Also texture layout (as colour layout) http://www.cultivate-int.org/issue7/artiste/

  7. Techniques: searching for an object • Much more difficult than previous techniques http://www-rocq.inria.fr/~gouet/HCP/index.htm l

  8. C-Bird case study • Content-Based Image Retrieval from Digital Libraries • Colour histogram • Colour Density • Colour layout • Texture layout • Illumination invariance • Object model

  9. C-Bird case study: Colour histogram • “coarse” histogram • 3 bits for red • 3 bits for green • 2 bits for blue • Giving 256 “bins” for the histogram. • Returns matches above a threshold, rather top few matches • So sometimes may return nothing. • Similarity is measured by histogram intersection • A histogram is generated for each image in the database • Each histogram is a sequence of numbers • Each value is divided by the sum of all the values in the histogram to remove the size of the image as a factor (normalisation). • The histogram is stored in the database. • When a comparison of two histograms is made, the minimum value of equivalent bins is summed for all the bins. The closer the sum is to 1, the better the match.

  10. C-Bird case study: Colour layout • User sets up a scheme of how colour should appear • Coarse blocks of colour in grid sizes 1x1, 2x2, 4x4, 8x8. • Cells can be specified with any RGB colour value or not. If no value is specified that grid is ignored in the search. • Each image in the database is analysed for each of the 4 grid sizes and the 5 most frequent colours in each cell are stored.

  11. C-Bird case study: Texture layout • User sets up a scheme of how colour should appear • Coarse blocks grid sizes 1x1, 2x2, 4x4, 8x8. • Cells can be specified with pre-defined textures, such as: • Comparison can then be carried out in a way analogous to colour layout..

  12. C-Bird case study: Illumination invariance • Differences in illumination of an image can lead to RGB sensors picking up different colour values – pink under daylight might be detected as purple under fluorescent lighting, for example. • It is desirable to remove the effect of differences in illumination from image when searching. • This is dealt with by: • normalising each of the RGB bands of the image (analogous to the way normalisation was used with colour histogram), • Creating a 128x128 colour histogram of chromaticity (capturing only colour information, not brightness).

  13. C-Bird case study: search by object model • This is described by Li and Drew as “the most important search type C-BIRD” supports. • The user picks a sample image and interactively selects a region for object searching – “object query-by-example”. • Uses technique of “feature localization” as opposed to “image segmentation”. • Both use concept of “tiles” • However “locales” are: • collections of tiles that share similar features • and can be found in a an area of the image, • but need not necessarily be contiguous (they can be scattered about a bit) • Li and Drew outline a combination of techniques for creating the tile and locale features from raw image data.

  14. Sources • Li and Drew (2004) Fundamentals of Multimedia (module recommended book) • http://www.cs.sfu.ca/cbird/ (demonstration site for C-BIRD) • Addis,M. Lewis, P. and Martinez, K. (2002) ARTISTE image retrieval system puts European galleries in the picture. http://www.cultivate-int.org/issue7/artiste/ (Accessed 4/05/06)

More Related