1 / 15

Heat Diffusion Classifier on a Graph

Heat Diffusion Classifier on a Graph. Haixuan Yang , Irwin King, Michael R. Lyu The Chinese University of Hong Kong Group Meeting 2006. Outline. Introduction Heat Diffusion Model on a Graph Three Graph Inputs Connections with Other Models Experiments Conclusions and Future Work.

yoko
Download Presentation

Heat Diffusion Classifier on a Graph

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Heat Diffusion Classifier on a Graph Haixuan Yang,Irwin King, Michael R. Lyu The Chinese University of Hong Kong Group Meeting 2006

  2. Outline Introduction Heat Diffusion Model on a Graph Three Graph Inputs Connections with Other Models Experiments Conclusions and Future Work

  3. Introduction • Kondor & Lafferty (NIPS2002) • Construct a diffusion kernel on a graph • Apply to a large margin classifier • Lafferty & Kondor (JMLR2005) • Construct a diffusion kernel on a special manifold • Apply to SVM • Belkin & Niyogi (Neural Computation 2003) • Reduce dimension by heat kernel and local distance • Tenenbaum et al (Science 2000) • Reduce dimension by local distance

  4. Introduction • The ideas we inherit • Local information • relatively accurate in a nonlinear manifold. • Heat diffusion on a manifold • a generalization of the Gaussian density from Euclidean space to manifold. • heat diffuses in the same way as Gaussian density in the ideal case when the manifold is the Euclidean space. • The ideas we think differently • Establish the heat diffusion equation directly on a graph • three proposed candidate graphs. • Construct a classifier by the solution directly.

  5. Heat Diffusion Model on a Graph • Notations

  6. Heat Diffusion Model on a Graph • Assumptions

  7. Heat Diffusion Model on a Graph • Solution

  8. Heat Diffusion Model on a Graph • Three candidate graphs • KNN Graph • Connect points j and i from j to i if j is one of the K nearest neighbors of i, measured by the Euclidean distance. • SKNN-Graph • Choose the smallest K*n/2 undirected edges, which amounts to K*n directed edges. • Minimum Spanning Tree • Choose the subgraph such that • It is a tree connecting all vertices; the sum of weights is minimum among all such trees.

  9. Heat Diffusion Model on a Graph • Illustration • Manifold • KNN Graph • SKNN-Graph • Minimum Spanning Tree

  10. Heat Diffusion Model on a Graph • Advantages and disadvantages • KNN Graph • Democratic to each node • Resulting classifier is a generalization of KNN • May not be connected • Long edges may exit while short edges are removed • SKNN-Graph • Not democratic • May not be connected • Short edges are more important than long edges • Minimum Spanning Tree • Not democratic • Long edges may exit while short edges are removed • Connection is guaranteed • Less parameter • Faster in training and testing

  11. Heat Diffusion Classifier (HDC) • Choose a graph • Compute the heat kernel • Compute the heat distribution for each class according to the initial heat distribution • Classify according to the heat distribution

  12. Connections with other models • The Parzen window approach (when the window function takes the normal form) is a special case of the HDC for the KNN and SKNN graphs (whenγis small, K=n-1). • KNN is a special case of the HDC for the KNN graph (whenγis small, 1/β=0). • In Euclidean space, the proposed heat diffusion model for the KNN graph (when K is set to be 2m, 1/β=0) is a generalization of the solution deduced by Finite Difference Method. • Hopefield Model (PNAS, 1982) is the original one which determines class by looking at immediate neighbors. (Thanks to the anonymous reviewer)

  13. Experiments • Experimental Setup • Experimental Environments • Hardware: Nix Dual Intel Xeon 2.2GHz • OS: Linux Kernel 2.4.18-27smp (RedHat 7.3) • Developing tool: C • Data Description • 3 artificial Data sets and 6 datasets from UCI • Comparison • Algorithms: • Parzen windowKNNSVM KNN-HSKNN-HMST-H • Results: average of the ten-fold cross validation

  14. Experiments • Results

  15. Conclusions and Future Work • KNN-H, SKNN-H and MST-H • Candidates for the Heat Diffusion Classifier on a Graph. • Future Work • Apply the asymmetric exp{γH} to SVM. • Extend the current heat diffusion model further (from inside). • DiffusionRank is a generalization of PageRank

More Related