310 likes | 456 Views
Image Categorization by Learning and Reasoning with Regions. Yixin Chen, University of New Orleans James Z. Wang, The Pennsylvania State University Published on Journal of Machine Learning Research 5 (2004). Presented by: Jianhui Chen 09/26/2006. Contents. Introduction
E N D
Image Categorization by Learning and Reasoning with Regions Yixin Chen, University of New Orleans James Z. Wang, The Pennsylvania State University Published on Journal of Machine Learning Research 5 (2004) Presented by: Jianhui Chen 09/26/2006
Contents • Introduction • Image segmentation and representation • An extension of Multiple Instance Learning - Diverse Density SVM (DD-SVM) • Comparison between DD-SVM & MI-SVM • Experimental results
Introduction • Image categorization • Images (a) and (b) are mountains and glaciers images. • Images (c), (d) and (e) are skiing images. • Images (f) and (g) are beach images.
Introduction • A new learning technique for region-based image categorization. • An extension from Multiple-Instance Learning (MIL) : DD-SVM.
Image Segmentation and Representation • Basic steps Step 1 - Divide the images into subblocks and extract LUV features. Step 2 - Do clustering using k-means and form regions. Step 3 - Form features vectors for regions (Classes).
Image Segmentation and Representation • Step 1: (1) Partition the image into non-overlapping blocks of size 4 x 4 pixels. (2) Extract LUV features from each of the blocks, denoted as L, U, V.
Image Segmentation and Representation • Step 1: (3)Apply Daubechies-4 wavelet transform and compute features from LH, HL, HH bands as fhl, flh and fhh. Suppose the coefficients are {ckl, ck,l+1, ck+1,l, cK+1,l+1}, the feature is computed as: (4) Form feature vector for each of the subblocks as:
Image Segmentation and Representation • Step 2: (1) Apply k-means and do clustering. (2) Each resulting class corresponds to one region.
Image Segmentation and Representation • Step 3: (1) Compute the mean of the feature vectors for each region. (2) Compute the normalized inertia of order 1,2,&3 for each region. Normalized inertia Shape feature of region Rj Feature vector on Region Rj
An Extension of Multiple Instance Learning • Basic idea of DD-SVM: (1) An images is referred as a bag which consists of instances. (2) Each bag is mapped to a point in a new feature space. (3) Standard SVMs are trained in the bag feature space.
An Extension of Multiple Instance Learning • Diverse-Density SVM (DD-SVM) (1) Maximum Margin Formulation of MIL. (2) Construct bag feature space based Diverse Density. (3) Compute region features vectors from instance features vectors. (4) A label is attached to a bag, instead of instances.
An Extension of Multiple Instance Learning • Objective function for DD-SVM: , define bag feature space. , a kernel function. C: controls the trade-off between accuracy and regularization.
An Extension of Multiple Instance Learning • The bag classifier is defined by as: * Assume the bag feature space is given.
An Extension of Multiple Instance Learning • Constructing a Bag Feature Space (1) Diverse Density (2) Learning Instance Prototypes (3) Computing Bag Features
Constructing a Bag Feature Space • Diverse Density x is a point in the instance feature space; W is a weight vector; Ni is the number in the i-th bag;
Constructing a Bag Feature Space • Learning Instance Prototypes (1) A large value of DD at a point indicates it may fit better with the instances from positive bags than with those from negative bags. (2) Choose local maximizers as instance prototypes. (2) An instance prototype represents a class of instances that is more likely to appear in positive bags than in negative bags.
Constructing a Bag Feature Space • Learning Instance Prototype
Computing Bag features (1) Each bag feature is defined by one instance prototype and one instance from the bag. (2) A bag feature gives the smallest distance between any instance and the corresponding instance prototype. (3) A bag feature can be viewed as a measure of the degree that an instance prototype shows up in the bag.
Comparison between DD-SVM & MI-SVM • Learning process of DD-SVM (1) Input is a collection of bags with binary labels. (2) Output is SVM classifier.
Comparison between DD-SVM & MI-SVM • Learning process of MI-SVM
Comparison between DD-SVM & MI-SVM • Learning process of MI-SVM Input: A collection of labeled bags. Output: a SVM classifier.
Comparison between DD-SVM & MI-SVM • DD-SVM: (1)Negative bag – all instances are negative (2) Positive bag – at least one instance is positive • MI-SVM: (1)One instance is selected to represent the whole positive bag. (2) Train SVM using all of the negative instances and selected positive instances.
Experimental Results • Experimental Setup and Data set (1) The data set consists of 2000 images from 20 image categories. (2) All images are in JPEG format with the size of 384 x 256 or 256 x 384. (3) Manually set parameters, ie. C (4) Comparison among DD-SVM, MI-SVM & Hist-SVM
Experimental Results • Categorization Result in terms of accuracy
Experimental Results • Classification result in terms of confusion matrix
Experimental Results All “Beach” images contain mountains or mountain-like regions All “Mountain and glaciers” images contain regions corresponding to river, lake or oceans.
Experimental Results • Sensitivity to image segmentation
Experimental Results • Sensitivity to the number of Categories in the Data Set
Experimental Results • Sensitivity to the size and diversity of training images