Download
1 / 69
690 likes | 703 Views
Learn about image segmentation, the process of partitioning an image into related regions. Discover the applications of image segmentation, such as image compression, medical diagnosis, and mapping and measurement. Explore the Bayesian view and the challenges of specifying prior knowledge. Dive into the Gestalt approach to perceptual grouping and the graph theoretic formulation of the segmentation problem. Understand the concept of normalized cut and its application in image segmentation.
E N D
Image segmentation using Eigenvectors Speaker : Sameer Agarwal Course : Learning and Vision Seminar Date : 09/10/2001
“Theoretically I might say there are 327 brightnesses and nuances of color. Do I have “327” No. I have sky, house, and trees. It is impossible to achieve “327” as such. And yet even though such droll calculations are possible--- and implied, say, for the house 120, the trees 90 and the sky 117– I should at least have this arrangement and division of the total, and not, say, 127 and 100 and 100; or 150 and 177.” Laws of Organization in Perceptual Forms Max Wertheimer (1923)
What is Image Segmentation ? • Partitioning of an image into related regions.
Why do Image Segmentation ? • Image Compression - Identify distinct components within an image and use the most suitable compression algorithm for each component to get a higher compression ratio. • Medical Diagnosis - Automatic segmentation of MRI images for identification of cancerous regions • Mapping and Measurement - Automatic analysis of remote sensing data from satellites to identify and measure regions of interest. e.g. Petroleum reserves.
How many groups ? Out of the various possible partitions, which is the correct one ?
The bayesian view Given prior knowledge about the structure of the data, choose the partition which is most probable. Problem : How do you specify a prior for knowledge which is composed of knowledge on multiple scales. e.g. • Coherence • Symmetry
A simple implementation • Assume that the image was generated by a mixture of multiple models • Segmentation is done in two steps : • Estimate the parameters of the mixture model • For each point calculate the posterior probabilities of it belonging to a cluster. Assign to the cluster with the maximum posterior.
Why doesn’t it work ? • The model selection problem. • Number of components ? • The structure of the components? • Estimation problem transforms into a hard optimization problem. No guarantee of convergence to the global optima.
Prior Work • k-means • Mixture Models (Expectation Maximization) • k-Medoid • k-Harmonic • Self Organizing Maps • Neural Gas • Linkage based graph methods.
Outline of the talk • The Gestalt approach to perceptual grouping • Graph theoretic formulation of the segmentation problem • The normalized cut • Experimental results • Relation to other methods • Conclusions
The Gestalt approach • Gestalt : a structure, configuration, or pattern of physical, biological, or psychological phenomena so integrated as to constitute a functional unit with properties not derivable by summation of its parts “The whole is different from the sum of the parts”
The Gestalt Movement • Formed by Max Werthheimer, Wolfgang Kohler and Kurt Koffka. • Rejected structuralism and its assumptions of atomicity and empiricism. • Adopted a “Holistic” approach to perception.
An Example Emergent properties of a configuration. The arrangement of several dots in a line gives rise to emergent properties, such as length, orientation and curvature, that are different from the properties of the dots that compose it.
And the moral of the story is .. • Image segmentation based on low level cues cannot and should not aim to produce a complete final “correct” segmentation. • Instead use low-level attributes like color, brightness to sequentially come up with hierarchical partitions. • Mid and high-level knowledge can be used to either confirm or select some partition for further attention.
A graph theoretic approach • A weighted undirected graph G = (V,E) • Nodes are points in the feature space • Fully connected graph • Edge weight w(i,j) is a function of the similarity between nodes i and j. Task: Partition the set V into disjoint sets V1,..,Vn, s.t. similarity among nodes in Vi is high and similarity across Vi and Vj is low.
Issues • What is a good partition ? • How can you compute such a partition efficiently ?
Graph Cut • G=(V,E) • Sets A and B are a disjoint partition of V Cut(A,B) is a measure of similarity between the two groups.
The temptation Cut is a measure of association Minimizing it will give a partition with the maximum disassociation. Efficient poly-time algorithms algorithms exist to solve the MinCut problem. So why not use it ?
The Normalized Cut Given a partition (A,B) of the vertex set V. Ncut(A,B) measures similarity between two groups, normalized by the “volume” they occupy in the whole graph.
Matrix formulation Definitions: D is an n x n diagonal matrix with entries W is an n x n symmetrical matrix
NP-Complete After some linear algebra we get.. • Subject to the constraints: • y(i)ε {1,-b} • ytD1=0
Real numbers to the rescue • Relax the constraints on y, and allow it to take real value. • Claim : The real valued MinNcut(G) can then be solved for by solving the generalized eigenvalue problem for the second smallest generalized eigenvector.
Proof • Rewrite the equation as Here Lemma 1: is an eigenvector of the above eigensystem with eigenvalue 0.
Proof(contd.) Lemma 2 : is a positive definite matrix since (D-W) is known to be positive semi-definite. Lemma 3 : z0 is the smallest eigenvector of eigensystem. Lemma 4 : z1 is perpendicular to z0
Proof (contd.) Lemma 5 : Let A be a real symmetric matrix, Under the constraint that x is orthogonal to the j-1 smallest eigenvectors x1,…,xj-1,the quotient is minimized by the next smallest eigenvector.
Finally.. • By lemma 1 we have y0=1 is an eigenvector of the eigensystem with eigenvalue 0. • It is the “smallest” eigenvector. • Hence by lemma 2, the second smallest eigenvector (y1) will minimize the Ncut equation. • By lemma 3 and 4 z1tz0= y1tD1=0
What about the first constraint ? • The second smallest eigenvector is only an approximation to the optimal normalized cut. y1 minimizes Y will take similar values for nodes with with high similarity value.
The grouping algorithm • Given an image, set up the weighted graph G=(E,V). Set the weight on the edges connecting two nodes as a measure of the similarity between the nodes. • Solve (D-W)x=λDx for eigenvectors with the smallest eigenvalues. • Use the second smallest eigenvector to bipartition the graph.
Details.. • The eigenvector takes continuous values, how do use it to segment the image ? • Choose 0 as the splitting point. • Find the median of the eigenvector and use that as the splitting point • Search amongst l evenly spaced points for one which gives the best exact Ncut value. • Impose a stability criterion on the eigenvector.
Stability ? • Since we allow the eigenvectors to take real values. Some eigenvectors might take a smooth continuous form. • We want vectors that have sharp discontinuities, indicating separation between regions. • Measure the smoothness of the vector, and stop partitioning when the smoothness value falls below a threshold.
Detail.. (Contd.) • How do you partition images with multiple segments ? 1. The higher order eigenvectors contain information about sub-partitions. Keep splitting till Ncut exceeds some pre-specified value. Problem : Numerical Error 2. Recursively run the algorithm on successive subgraphs. Problem : Computationally Expensive and the stability criterion might prevent correct partitioning.
Simultanous P-way cut • Use the first n eigenvectors as n-dimensional indicator vectors of each point. This is equivalent to imbedding each point in an n-dimensional space. • Perform k-means clustering in this new space to create p’>p clusters. • Use the original 2-way Ncut or a greedy strategy to merge these p’ partitions into p partitions.
How good is the approximation ? The normalized cheeger constant h is defined as : We know that the second eigenvalue is bounded by : This is only a qualitative indication of the quality of approximation, it does not say anything about how close the eigenvector is to the optimal Ncut vector.
