Minimum Spanning Trees and Clustering

1. Minimum Spanning Trees and Clustering By Swee-Ling Tang April 20, 2010

2. Spanning Tree • A spanning tree of a graph is a tree and is a subgraph that contains all the vertices. • A graph may have many spanning trees; for example, the complete graph on four vertices has sixteen spanning trees:

3. Spanning Tree – cont.

4. Minimum Spanning Trees • Suppose that the edges of the graph have weights or lengths. The weight of a tree will be the sum of weights of its edges. • Based on the example, we can see that different trees have different lengths. • The question is: how to find the minimum length spanning tree?

5. Minimum Spanning Trees • The question can be solved by many different algorithms, here is three classical minimum-spanning tree algorithms : • Boruvka's Algorithm • Kruskal's Algorithm • Prim's Algorithm

6. Kruskal's Algorithm • Joseph Bernard Kruskal, Jr • Kruskal Approach: • Select the minimum weight edge that does not form a cycle • Kruskal's Algorithm: • sort the edges of G in increasing order by length • keep a subgraph S of G, initially empty • for each edge e in sorted order • if the endpoints of e are disconnected in S • add e to S • return S

7. Kruskal's Algorithm - Example

8. Kruskal's Algorithm - Example

9. Prim’s Algorithm • Robert Clay Prim • Prim Approach: • Choose an arbitrary start node v • At any point in time, we have connected component N containing v and other nodes V-N • Choose the minimum weight edge from N to V-N • Prim's Algorithm: • let T be a single vertex x • while (T has fewer than n vertices) • { • find the smallest edge connecting T to G-T • add it to T • }

10. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

11. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

12. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

13. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

14. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

15. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

16. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

17. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

18. Prim's Algorithm - Example 8 12 9 13 2 11 14 40 20 7 6 50 10 3 1

19. Boruvka's Algorithm • Otakar Borůvka • Inventor of MST • Czech scientist • Introduced the problem • The original paper was written in Czech in 1926. • The purpose was to efficiently provide electric coverage of Bohemia.

20. Boruvka’s Algorithm • Boruvka Approach: • Prim “in parallel” • Repeat the following procedure until the resulting graph becomes a single node. • For each node u, mark its lightest incident edge. • Now, the marked edges form a forest F. Add the edges of F into the set of edges to be reported. • Contract each maximal subtree of F into a single node.

21. 3 5 3.1 4 3.3 2.4 Boruvka’s Algorithm - Example 2.1 3.2 1.5 5.1 2.2 2.5 1 2.3 2.6 4.1 1.3 1.2 1.4 2.7

22. Usage of Minimum Spanning Trees • Network design: • telephone, electrical, hydraulic, TV cable, computer, road • Approximation algorithms for NP-hard (non-deterministic polynomial-time hard) problems: • traveling salesperson problem • Cluster Analysis

23. Clustering • Definition • Clustering is “the process of organizing objects into groups whose members are similar in some way”. • A cluster is therefore a collection of objects which are “similar” between them and are “dissimilar” to the objects belonging to other clusters. • A data mining technique

24. Why Clustering? • Unsupervised learning process • Pattern detection • Simplifications • Useful in data concept construction

25. The use of Clustering • Data Mining • Information Retrieval • Text Mining • Web Analysis • Marketing • Medical Diagnostic • Image Analysis • Bioinformatics

26. Image Analysis – MST Clustering An image file before/after color clustering using HEMST (Hierarchical EMST clustering algorithm) and SEMST (Standard EMST clustering algorithm).

27. References • Minimum Spanning Tree Based Clustering Algorithms - http://www4.ncsu.edu/~zjorgen/ictai06.pdf • Minimal Spanning Tree based Fuzzy Clustering - http://www.waset.org/journals/waset/v8/v8-2.pdf • Minimum Spanning Tree – Wikipedia - http://en.wikipedia.org/wiki/Minimum_spanning_tree • Kruskal's algorithm - http://en.wikipedia.org/wiki/Kruskal%27s_algorithm • Prim's Algorithm – Wikipedia - http://en.wikipedia.org/wiki/Prim%27s_algorithm • Design and Analysis of Algorithms - http://www.ics.uci.edu/~eppstein/161/960206.html