Randomized Algorithms CS648

1. Randomized AlgorithmsCS648 Lecture 17 Miscellaneous applications of Backward analysis

2. Minimum spanning tree

3. Minimum spanning tree 16 d 17 9 v x 4 2 5 h 19 6 11 y 3 3 22 c 12 a 18 15 1 10 u b 7 13

4. Minimum spanning tree 16 d 17 9 Algorithms: • Prim’salgorithm • Kruskal’s algorithm • Boruvka’s algorithm v x 4 2 5 h 19 6 11 y 3 3 22 c 12 a 18 15 1 10 u b 7 13 Less known but it is the first algorithm for MST

5. Minimum spanning tree : undirected graph with weights on edges , . Deterministic algorithms: Prim’s algorithm • O(( + ) log) using Binary heap • O( + log) using Fibonacci heap Best deterministic algorithm: O( + ) bound • Too complicated to design and analyze • Fails to beat Prim’s algorithm using Binaryheap

6. Minimum spanning tree When finding an efficient solution of a problem appears hard, one should strive to design an efficient verification algorithm. MST verification algorithm: [King, 1990] Given a graph and a spanning tree , it takes O( + ) time to determine if is MST of . Interestingly, no deterministic algorithm for MST could use this algorithm to achieve O( + ) time.

7. Minimum spanning tree : undirected graph with weights on edges , . Randomized algorithm: Karger-Klein-Tarjan’s algorithm [1995] • Las Vegas algorithm • O( + ) expected time This algorithm uses • Random sampling • MST verification algorithm • Boruvka’s algorithm • Elementary data structure

8. Minimum spanning tree : undirected graph with weights on edges , . Randomized algorithm: Karger-Klein-Tarjan’s algorithm [1994] • Las Vegas algorithm • O( + ) expected time • Random sampling : How close is MST of a random sample of edges to MST of original graph ? The notion of closeness is formalized in the following slide.

9. Light Edge 16 d 17 9 Definition:Let . An edge is said to be light with respect to if Question: If and ||=, how many edges from are light with respect to on expectation ? Answer: ?? v x 4 2 5 h 19 6 31 11 y 3 3 22 c 12 a 18 15 1 10 u b 7 13 MST() MST()

10. USING Backward analysis forMiscellaneous Applications

11. problem 1SMALLEST Enclosing circle

12. Smallest Enclosing Circle

13. Smallest Enclosing Circle Question: Suppose we sample points randomly uniformly from a set of points, what is the expected number of points that remain outside the smallest circle enclosing the sample? For =, the answer is

14. problem 2smallest length interval

15. Sampling points from a unit interval Question: Suppose we select points from interval [0,1], what is expected length of the smallest sub-interval ? • for, it is ?? • for, it is ?? • General solution : ?? This bound can be derived using two methods. • One method is based on establishing a relationship between uniform distribution and exponential distribution. • Second method (for nearly same asymptotic bound) using Backward analysis. 1 0

16. problem 3Minimum spanning tree

17. Light Edge 16 d 17 9 Definition:Let . An edge is said to be light with respect to if Question: If and ||=, how many edges from are light with respect to on expectation ? v x 4 2 5 h 19 6 31 11 y 3 3 22 c 12 a 18 15 1 10 u b 7 13 MST() MST()

18. using Backward analysis forThe 3 problems :A General framework

19. A General framework Let be the desired random variable in any of these problems/random experiment. • Step 1: Define an event related to the random variable . • Step 2: Calculate probability of event using standard method based on definition. (This establishes a relationship between ) • Step 3: Express the underlying random experiment as a Randomized incremental construction and calculate the probability of the event using Backward analysis. • Step 4: Equate the expressions from Steps1 and 2 to calculate E[].

20. problem 3Minimum spanning tree

21. A Better understanding of light edges

22. Minimum spanning tree 16 d 17 19 v x 4 2 5 h 19 6 31 11 y 3 3 22 c 42 a 18 15 1 10 u b 7 13 Random sampling 16 d v x 4 5 h 19 31 11 y 22 c a 18 15 1 10 u b

23. Minimum spanning tree 16 d 17 19 v x 4 2 5 h 19 6 31 11 y 3 3 22 c 42 a 18 15 1 10 u b 7 13 16 MST() d v x 4 5 h 19 31 11 y 22 c a 18 15 1 10 u b

24. Minimum spanning tree 16 d 17 19 v x 4 2 5 h 19 6 31 11 y 3 3 22 c 42 a 18 15 1 10 u b 7 23 16 MST() d 17 19 v x 4 2 5 h 19 6 11 y 3 3 c 42 a 15 1 10 u b 7 23

25. Minimum spanning tree 16 d 17 19 v x 4 2 5 h 19 6 31 11 y 3 3 22 c 42 a 18 15 1 10 u b 7 23 16 MST() d 17 19 v x 4 2 5 h 19 6 11 y 3 3 c 42 a Light 15 1 10 u b 7 23

26. First useful insight Lemma1: An edge is light with respect to if and only if belongs to MST().

27. Minimum spanning tree 16 d 17 19 v x 4 2 5 h 19 6 31 11 y 3 3 22 c 42 a 18 15 1 10 u b 7 23 16 MST() d 17 19 v x 4 2 5 h 19 6 11 y 3 3 c 42 a heavy Light 15 1 10 u b 7 23

28. Minimum spanning tree d v x 4 2 5 h 6 MST() y 3 3 c a 1 u b 7 Is there any relationship among MST(), MST() and Light edges from ? 16 MST() d 17 19 v x 4 2 5 h 19 6 11 y 3 3 c 42 a Light heavy 15 1 10 u b 7 23

29. Second useful insight Lemma2: Let • and • be the set of all edges from that are light with respect to . Then MST()=MST() This lemma is used in the design of randomized algorithm for MST as follows (just a sketch): • Compute MST of a sample of edges (recursively). Let it be T’. • There will be expected edges light edges among all unsampled edges. • Recursively compute MST of T’ edges which are less than on expectation.

30. Light Edge 16 d 17 9 Definition:Let . An edge is said to be light with respect to if Question: If and ||=, how many edges from are light with respect to on expectation ? v x 4 2 5 h 19 6 31 11 y 3 3 22 c 12 a 18 15 1 10 u b 7 13 MST() MST() We shall answer the above question using the Generic framework. But before that, we need to get a better understanding of the corresponding random variable.

31. MST()

32. Light MST()

33. Light heavy MST()

34. : random variable for the number of light edges in whenis a random sample of edges. : set of all subsets of of size . : number of light edges in when . = ?? Can you express in terms of and only ?

35. Step 1 Question: Let be a uniformly random sample of edges from . What is the prob. that an edge selected randomly from is a light edge ? Two methods to find

36. Step 2 Calculatingusing definition

37. Step 2 Calculatingusing definition Light edges = Light heavy MST()

38. Step 2 Calculatingusing definition : set of all subsets of of size . The probability is equal to

39. Step 3 Expressing the entire experiment as Randomized Incremental Construction A slight difficulty in this process is the following: • The underlying experiment talks about random sample from a set. • But RIC involves analyzing a random permutation of a set of elements.  Question:What is relation between random sample from a set and a random permutation of the set ? Spend some time on this question before proceeding further.

40. random sample and random permutation Random permutation of Observation: is indeed a uniformly random sample of

41. Step 3 The underlying random experiment as Randomized Incremental Construction: • Permute the edges randomly uniformly. • Find the probability that th edge is light relative to the first edges. Question: Can you now calculate probability ? Spend some time on this question before proceeding further.

42. Step 3 Random permutation of …

43. Step 3 Random permutation of : a random variable taking value 1 if is a light edge with respect to MST(). …

44. Step 3 Random permutation of : a random variable taking value 1 if is a light edge with respect to MST(). Question: What is relation between and ’s? Answer: ?? …

45. Calculating ). Random permutation of : set of all subsets ofof size . ) =  depends upon … Forward analysis MST()

46. Calculating ). Random permutation of : set of all subsets of of size . )= … Backward analysis

47. Random permutation of = ?? ?? … Use Lemma 2. Backward analysis MST()

48. Calculating ) Random permutation of : set of all subsets of of size . )= … Backward analysis

49. Combining the two methods for calculating Using method 1: Using method 2: ) Hence:

50. Theorem: If we sample edges uniformly randomly from an undirected graph on vertices and edges, the number of light edges among the unsampled edges will be less than on expectation.