Abstract

1. 5 Classification of painting styles Van Gogh Shishkin Cluster I M - current mixture model Current models in M Aivazovsky Old models removed from M Models obtained from tentative split Picasso Rembrandt M0 35 pictures (about 600x800 pixels) drawn by 5 different painters. The algorithm successfully identifies the painters by their drawing style. Score(M) = P(M) JS(M1,M2) Cluster II M “Tentative” split JS M2 M1 1 Image window Wavelet transform Normalized coefficient histograms 2 5 4 3 Unsupervised Clustering of Images using their Joint Segmentation Yevgeny Seldin, Sonia Starik, Michael Werman School of Computer Science and Engineering The Hebrew University of Jerusalem, Israel www.cs.huji.ac.il/~seldin/SCTV2003 {seldin,starik,werman}@cs.huji.ac.il 1 Abstract We present a method for unsupervised content based classification of images. Our major idea is to start from joint segmentation of all the images in the input set. We then draw an analogy between segments/words and images/documents and apply advanced algorithms from the field of unsupervised document classification to cluster the images. Outline • Input: set of images • photos from traveling, image database organization, movie frames • Preprocessing step:Texture modeling • cover each image by an overlapping net of small square windows • model each window as a texture • Step1: Unsupervised joint segmentation • represent the images as a mixture of a small number of textures. • Step 2: Image clustering based on the segmentation map: Use co-occurrence statistics of mixture components and images in order to cluster the images 3 Step 1: Unsupervised joint image segmentation (Unsupervised feature selection) • Represent all the images in the set with single mixture model M. • Use forced hierarchy deterministic annealing framework for soft hierarchical top-down segmentation at increasing levels of resolution Schematic illustration of segmentation framework Example Classification based on the Segmentation map Joint Segmentation (Texture based) 6 Classification of the Hebrew university campus views Segmented images (Similar textures have the same color) Classified images Original images 4 2 Preprocessing: Texture modeling • Texture parametric model: Set of histograms of wavelet coefficients, normalized to probability distributions • Distance between textures • Kullback-Leibler divergence • Centroid Model • A centroid model M for a weighted set of textures is a normalized weighted sum of wavelet histograms Hi Other statistical properties, for example color histograms for a window, can be used instead or together with texture modeling Step 2: Image classification according to the segmentation map (Unsupervised classification based on the selected features) Indoor scenes Perspective views Idea: Use image-document, model-word analogy Goal: represent the images with a small number of clusters s.t. the distribution of models inside the clusters will be maximally close to the original distribution of the models inside the images Solution: sequential Information Bottleneck algorithm of Slonim, Friedman, Tishby 2. Reassign it to cluster C* for which I(C;M) is maximized 1. Pick a random image I from a random cluster C Tree branches on sky background Bushes and flowers, close view In this example the algorithm classifies 55 view images (640x480 pixels) by their content