1 / 38

Image Mining – Intricacies and Innovations

Image Mining – Intricacies and Innovations. SUNDARAM R M D M.Tech Computer Vision & Image Processing RE cognition A nd L earning Lab Amrita School of Engineering. Agenda. Introduction 5-min Recap Problem Definition – CBIR Need for CBIR Flow of work Feature Extraction – A brief view

nerita
Download Presentation

Image Mining – Intricacies and Innovations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Mining – Intricacies and Innovations SUNDARAM R M D M.Tech Computer Vision & Image Processing REcognition And Learning Lab Amrita School of Engineering

  2. Agenda Introduction 5-min Recap Problem Definition – CBIR Need for CBIR Flow of work Feature Extraction – A brief view Promising directions and open issues Performance Evaluation Conclusion Content Based Image Retrieval

  3. Introduction Content Based Image Retrieval

  4. 5-min Recap Why is Image Information Retrieval more important? • “A Picture is worth thousand words” • Alternative form of communication • Not everything can be described in text • Popular medium of information on the Internet • Well known search engines like “Google” fails to retrieve images based on their contents. Content Based Image Retrieval

  5. 5-min Recap Content Based Image Retrieval

  6. 5-min Recap Content Based Image Retrieval

  7. 5-min Recap Content Based Image Retrieval

  8. Problem Definition Given a query image, with single / multiple object present in it; mission of this work is to retrieve similar kind of images from the database based on the features extracted from the query image. - Content based Image Retrieval (CBIR) Content Based Image Retrieval

  9. Need for CBIR • Early work on image retrieval can be tracked back to the late 1970s. Techniques used were not generally based on visual features but on the textual annotation of the images. • Through text descriptions, images can be organized by topical or semantical hierarchies to facilitate easy navigation and browsing based on standard Boolean queries. • But automatically generating texts for a wide spectrum of images is not feasible. Also, annotating images manually is a cumbersome and expensive task for large image databases, and is often subjective, context-sensitive and incomplete. • Hence it is widely recognized that a more efficient and intuitive way to represent and index visual information is needed. This gave birth to a new concept called CBIR. Content Based Image Retrieval

  10. Applications of CBIR • Crime prevention: Automatic face recognition systems, used by police forces. • Security Check: Finger print or retina scanning for access privileges. • Medical Diagnosis: Using CBIR in a medical database of medical images to aid diagnosis by identifying similar past cases. • Intellectual Property: Trademark image registration, where a new candidate mark is compared with existing marks to ensure no risk of confusing property ownership. Content Based Image Retrieval

  11. Flow of Work Flow Chart Content Based Image Retrieval

  12. Feature Extraction [1] Extract Features (Primitives) Similarity Measure Matched Results Query Image Image Database Relevance Feedback Algorithm Features Database Content Based Image Retrieval

  13. Feature Extraction [2] -- Color • Color is the most extensively used visual content for image retrieval. • Its three-dimensional values make its discrimination potentiality superior to the single dimensional gray values of images. • The Various parameters that can be extracted from color information are as follows: 1) Color Moments 2) Color Histogram Content Based Image Retrieval

  14. Feature Extraction [3] -- Color Color is the most extensively used visual content for image retrieval Finding the Color moments 1) μi = (1/N) Σ fij …Mean 2) σi = { (1/N) Σ (fij - μi )2 } ½ …Variance 3) Si = { (1/N) Σ (fij - μi )3 } 1/3 …Skew ness Where fij is the value of the ith color component of the image pixel j, and N is the number of pixels in the image. Through Color Histogram Since any pixel in the image can be described by three components in a certain color space, a histogram, i.e., the distribution of the number of pixels for each quantized bin, can be defined for each component. Clearly, the more bins a color histogram contains, the more discrimination power it has. Content Based Image Retrieval

  15. Feature Extraction [4] -- Color Content Based Image Retrieval

  16. Why Fuzzy Color Histogram(FCH)? Existing Algorithm: Given a color space containing n color bins, the color histogram of image containing N pixels is represented as, H(I) = [h1, h2, h3, ….hn] where hi = Ni / N is the probability of a pixel in the image belonging to the ith color bin, and Ni is the total number of pixels in the ith color bin. Fuzzy Color Histogram: Instead of using the normal probability values, we consider each of the N pixels in the image I being related to all the n color bins via Fuzzy – set membership functions such that the degree of belongingness or association of the jth pixel to the ith color bin is determined by distributing the membership value of the jth pixel,μij to the ith color bin. Content Based Image Retrieval

  17. Feature Extraction [5] -- Color Context Sensitivity Content Based Image Retrieval

  18. Feature Extraction [6] -- Texture • Texture is another important property of images. Basically, texture representation methods can be classified into two categories: structural and statistical. • Structural methods, including morphological operator and adjacency graph, describe texture by identifying structural primitives and their placement rules. They tend to be most effective when applied to textures that are very regular. • Statistical methods include, for example, in areas with smooth texture, the range of values in the neighborhood around a pixel will be a small value; in areas of rough texture, the range will be larger. Similarly, calculating the standard deviation of pixels in a neighborhood can indicate the degree of variability of pixel values in that region. Content Based Image Retrieval

  19. Feature Extraction [7] -- Texture Grey – level co-occurrence matrix Content Based Image Retrieval

  20. Feature Extraction [8] -- Texture Tamura Features – Three main features including Coarseness, contrast and directionality are extracted. • Coarseness is the measure of granularity of an image, or average size of regions that have the same intensity. • Contrast is the measure of brightness of the texture pattern. Therefore, the bigger the blocks that makes up the image, the higher the contrast. • Directionality is the measure of directions of the gray values within the image. Content Based Image Retrieval

  21. Why Tamura Features ? [1] Following are the features calculated from the normalized co-occurrence matrix P(i,j): Content Based Image Retrieval

  22. Why Tamura Features ? [2] • An image will contain textures at several scales; coarseness aims to identify the largest size at which a texture exists, even where a smaller micro texture exists. • Here we first take averages at every point over neighborhoods, the linear size of which are powers of 2. The average over the neighborhood of size 2k * 2k at the point (x,y) is Content Based Image Retrieval

  23. Why Tamura Features ? [3] • Then at each point one takes differences between pairs of averages corresponding to non-overlapping neighborhoods on opposite sides of the point in both horizontal vertical orientations. In the horizontal case this is • At each point, one then picks the best size which gives the highest output value, where k maximizes E in either direction. The Coarseness measure is then the average of Sopt (x,y)= 2kopt over the picture. Content Based Image Retrieval

  24. Why Tamura Features ? [4] • Contrast aims to capture the dynamic range of gray levels in an image, together with the polarization of the distribution of black and white. Here μ4 is the fourth moment about the mean and σ2 is the variance. • Directionality is a global property over a region. Detect the edges in an image. At each pixel the angle and magnitude are calculated. A histogram of edge probabilities is then built up by counting all points with magnitudes greater than a threshold and quantizing by the edge angle. The histogram will reflect the degree of directionality. Content Based Image Retrieval

  25. Feature Extraction [9] -- Texture Content Based Image Retrieval

  26. Feature Extraction [10] -- Shape • Compared with Color and Texture features, shape features are usually described after images have been segmented into regions or objects. • Since robust and accurate image segmentation is difficult to achieve, the use of shape features for image retrieval has been limited to special applications where objects or regions are readily available. • The state-of-art methods for shape description can be categorized into either boundary-basedpolygonal approximation, finite element models and Fourier-based shape descriptors or region-basedmethods (statistical moments.) • A good shape representation feature for an object should be invariant to translation, rotation and scaling. Phase Congruency is one such feature. Content Based Image Retrieval

  27. Feature Extraction [11] -- Shape Algorithm phase_congruency: Input: Amplitude An(x) Phase angle n(x) Weighted mean phase angle (x) Weights for frequency spread w(x) To avoid division by zero, add a small  Compute PC1 (x) = E (x) / n An(x) WhereE (x) = n An (cos ((x) - (x)) To produce more localized response, PC1 (x) = w (x)E (x) -T / n An(x) +  Compute distance measurement between query image and image in the database. Output: Retrieve similar images. Content Based Image Retrieval

  28. Similarity / Distance Measure Instead of exact matching, content-based image retrieval calculates visual similarities between a query image and images in a database. Accordingly, the retrieval result is not a single image but a list of images ranked by their similarities with the query image. If each dimension of image feature vector is independent of each other and is of equal importance, the Minkowski-form distance is appropriate for calculating the distance between two images. This distance is defined as: D( i, j ) = Σ fi (I) - fi (J) p1/p Content Based Image Retrieval

  29. Highlights and Novelty in the work • Using Rodriguez formula to normalize the images before taking the color Histogram. • Analysis of Statistical methods, including Fourier power spectra, co-occurrence matrices, Tamura feature, and multi-resolution filtering techniques such as Gabor and wavelet transform, for characterizing texture. • Phase Congruency measurement for Shape Information. Content Based Image Retrieval

  30. Promising Directions and Open Issues • Relevance feedback is proposed as a technique for overcoming many of the problems faced by fully automatic systems by allowing the user to interact with the computer to improve retrieval performance. • More interestingly, the semantic gap is need to be bridged. Semantic gap here refers to “the large disparity between the low-level features or content descriptors that can be computed automatically by current machines and algorithms, and the richness and subjectivity of semantics in user queries and high-level human interpretations of audiovisual media” Content Based Image Retrieval

  31. Performance Evaluation • The proposed technique is tested on two types of data sets, first one consisting of different animals and the second dataset consisting of birds, flowers and buildings. • The retrieval accuracy is found to be 96.4% and 92.2% for a database size of 55 each. Content Based Image Retrieval

  32. Database (Partial Set) Content Based Image Retrieval

  33. Results [1] QUERY IMAGE Content Based Image Retrieval

  34. Results [2] -- Color Content Based Image Retrieval

  35. Results [3] – Color + Texture Content Based Image Retrieval

  36. References • M. Flickner, H. Sawhney, W. Niblack, and P. Yanker, “Querying by image and video content: The QBIC system,” IEEE Trans. Comput., vol. 28, pp. 23–32, 1995. • James Z. Wang, Jia Li and Gio, “SIMPLIcity: Semantics – sensitive Integrated Matching for picture Libraries,” IEEE Transactions on pattern Analysis and Machine Intelligence, Vol.23,no.9,pp.947-963,2001. • Smeulders, Senior Member, IEEE, Santini, Member, IEEE, “Content-Based Image Retrieval at the End of the Early Years”, IEEE transactions on pattern analysis and machine intelligence, vol. 22, no. 12, december 2000 • K.Satya Sai Prakash, RMD. Sundaram, “Shape Information from Phase Congruency and its application in Content based Image Retrieval”, Proceedings of the 3rd Workshop on Computer Vision, Graphics and ImageProcessing – WCVGIP 2006, Hyderabad PP.88-93, 2006. Content Based Image Retrieval

  37. References • J. van de Weijer and J. M. Geusebroek. Color edge and corner detection by photometric quasi-invariants. IEEE Trans. Pattern Anal. Machine Intel. 27(4):625-630, 2005. • Barbeau Jerome, Vignes-Lebbe Regine, and Stamon Georges, “A Signature based on Delaney Graph and Co-occurrence Matrix,” Laboratories Informatique et Systematique, University of Paris, Paris, France, July 2002, Found at: http://www.math-info.univ-paris5.fr/sip-lab/barbeque/barbeque.pdf Content Based Image Retrieval

  38. Discussions???? THANK YOU sundaram.rmd@gmail.com Content Based Image Retrieval

More Related