Region Based Image Annotation Through Multiple-Instance Learning

1. Region Based Image Annotation Through Multiple-Instance Learning By: Changbo Yang Wayne State University Department of Computer Science

2. Annotation by Region An image contains several regions, and each region may have different contents. The incomplete information provided by the training images. The annotation information for a training image is usually available at the concept level (annotation to the image ), but NOT at the content level (annotation to region) . A large number of irrelevant noisy regions, exist in the training set for keyword “tiger”.

3. Bayesian Framework for image annotation • Consider image annotation as a problem of image classification, in which each keyword is treated as a distinct class label. • Given the feature vector of a testing image I, what is the probability that I belongs to class w? • One of the key steps in Bayesian classification is to select the most representative image region for a keyword . • a large amount of irrelevant noisy regions in training images. MIL could be used to predict the representative region

4. What MIL can do? • Multiple-instance learning: a variation of supervised learning. • MIL Problem: Each bag may contain many instances. A bag is labeled positive even if only one of the instances is positive. A bag is labeled negative only if all the instances in it are negative. • In region based image annotation, each region is an instance, and the set of regions that comes from the same image can be treated as a bag. • One way to solve MIL problem is to examine the distribution of these instances, and look for an instance that is close to all the instances in the positive bags and far from those from negative bags. • MIL algorithm: Diverse Density, MIL-SVM, ASVM and so on.

5. Region based image annotation Predict the most possible representative region of a semantic meaning by relevant and irrelevant images.