1 / 19

Distance Metric Learning with Spectral Clustering

Distance Metric Learning with Spectral Clustering. By Sheil Kumar. Spectral Clustering . Based on the MinCut Problem Cuts deal with pairwise similarity measures, and thus can capture non-linear relationships. Spectral Clustering Cont. . sigma = .1. Spectral Clustering Cont. . sigma = .1.

onan
Download Presentation

Distance Metric Learning with Spectral Clustering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distance Metric Learning with Spectral Clustering By Sheil Kumar

  2. Spectral Clustering • Based on the MinCut Problem • Cuts deal with pairwise similarity measures, and thus can capture non-linear relationships.

  3. Spectral Clustering Cont. sigma = .1

  4. Spectral Clustering Cont. sigma = .1

  5. Spectral Clustering Cont. • sigma = 3

  6. Motivation • Finding the sigma parameter automatically and optimally will give us better clustering of the data. • It is hard to formulate a RBF distance metric such that the sigma is easily isolatable. • Mahalanobis Distance?

  7. Defining a Distance Metric 1) The distance metric must represent similarities between data points. 2) Commonly RBF Kernels are used as distance metrics in SC. 3) The Mahalanobis Distance must form positive distance values representing similarity, NOT dissimilarity.

  8. The MinCut Problem The about eqn is subject to constraints yi must take on discrete binary values, and yTD1 = 0. If y is relaxed to take on real values, this minimization is equivalent to the eigenvalue system This eqn is easily shown by substituting z = D1/2y. z0 = D1/21 is an eigenvector, with eigenvalue = 0.

  9. Minimizing Eigenvectors D-1/2(D-W)D-1/2 is a symmetric semi-positive definite matrix because (D-W) (also known as the Laplacian Matrix) is known to be symmetric semi-positive definite. z0 is the smallest eigenvector of D-1/2(D-W)D-1/2, and all other eigenvectors are perpendicular to it. z1, the second smallest eigenvector has the property z1Tz0 = 0 = y1TD1

  10. Minimizing Eigenvectors Cont. Thus we obtain arg.min zTz0 = 0 : zTD-1/2(D-W)D-1/2z zTz and equivalently: arg.min yTD1 = 0 : yT(D-W)y yTDy Minimizing the second smallest eigenvector solution of this equation is guaranteed to give us the normalized cut solution with the second constraint satisfied.

  11. Trace SDP Given that the lambdas are eigenvector solutions to matrix K, we see that minimizing over the lambdas is equivalent to minimizing over the tr(KB). Minimizing our second eigenvector can be rewritten as a Procrustes Problem. B = a weighted outer product of the eigenvectors such that the eigenvectors are normalized, and the weights are in strictly increasing order.

  12. Trace SDP Cont. Because we know that we want to only minimize our second smallest and smallest eigenvector, we can set alpha n and n-1 to 1, and the rest to 0.

  13. Solving the SDP

  14. The K Matrix

  15. Solving the SDP Cont.

  16. Solving the SDP

  17. Some Results (more coming)

  18. More results

  19. Conclusions • Unclear as of right now, whether linear transformations *help* clustering. • More interesting Distance Metrics

More Related