Randomized Algorithms for Selection and Sorting

1. Randomized Algorithms for Selection and Sorting Analysis of Algorithms Prepared by John Reif, Ph.D.

2. Randomized Algorithms for Selection and Sorting • Randomized Sampling • Selection by Randomized Sampling • Sorting by Randomized Splitting: Quicksort and Multisample Sorts

3. Readings • Main Reading Selections: • CLR, Chapters 9

4. Comparison Problems • Input set X of N distinct keys total ordering < over X • Problems • For each key x Xrank(x,X) = |{x’ X| x’ < x}| + 1 • For each index i {1, …, N)select (i,X) = the key x X where i = rank(x,X) • Sort (X) = (x1, x2, …, xn) where xi = select (i,X)

5. Randomized Comparison Tree Model

6. Algorithm Samplerank s(x,X) begin Let S be a random sample of X-{x} of size soutput 1+ N/s [rank(x,S)-1] end

7. Algorithm Samplerank s(x,X) (cont’d) • Lemma 1 • The expected value of samplerank s(x,X) is rank (x,X)

8. Algorithm Samplerank s(x,X) (cont’d)

9. S is Random Sample of X of Size s

10. More Precise Bounds on Randomized Sampling (cont’d)

11. More Precise Bounds on Randomized Sampling • Let S be a random sampling of X • Let ri = rank(select(i,S),X)

12. More Precise Bounds on Randomized Sampling (cont’d) • Proof We can bound ri by a Beta distribution, implying • Weak bounds follow from Chebychev inequality • The Tighter bounds follow from Chernoff Bounds

13. Subdivision by Random Sampling • Let S be a random sample of X of size s • Let k1, k2, …, ks be the elements of S in sorted order • These elements subdivide X into

14. Subdivision by Random Sampling (cont’d) • How even are these subdivisions?

15. Subdivision by Random Sampling (cont’d) • Lemma 3 If random sample S in X is of size s and X is of size N, then S divides X into subsets each of

16. Subdivision by Random Sampling (cont’d) Proof • The number of (s+1) partitions of X is • The number of partitions of X with one block of size  v is

17. Subdivision by Random Sampling (cont’d) • So the probability of a random (s+1) partition having a block size  v is

18. Randomized Algorithms for Selection • “canonical selection algorithm” • Algorithm can select (i,X) input set X of N keys index i  {1, …, N} [0] if N=1 then output X [1] select a bracket B of X, so that select (i,X) B with high prob. [2] Let i1 be the number of keys less than any element of B [3] output can select (i-i1, B) B found by random sampling

19. Hoar’s Selection Algorithm • Algorithm Hselect (i,X) where 1  i  N begin if X = {x} then output x else choose a random splitter k  X let B = {x  X|x < k} if |B| i then output Hselect(i,B) else output Hselect(i-|B|, X-B) end

20. Hoar’s Selection Algorithm (cont’d) • Sequential time bound T(i,N) has mean

21. Hoar’s Selection Algorithm (cont’d) • Random splitter k  X Hselect (i,X) has two cases

22. Hoar’s Selection Algorithm (cont’d) • Inefficient: each recursive call requires N comparisons, but only reduces average problem size to ½ N

23. Improved Randomized Selection • By Floyd and Rivest • AlgorithmFRselect(i,X) begin if X = {x} then output x else choose k1, k2 X such that k1< k2 let r1 = rank(k1, X), r2 = rank(k2, X) if r1 > i then FRselect(i, {x  X|x < k1}) else if r2 > i then FRselect(i-r1, {x  X|k1x k2}) else FRselect(i-r2, {x  X|x > k2}) end

24. Choosing k1, k2 • We must choose k1, k2 so that with high likelihood, k1  select(i,X)  k2

25. Choosing k1, k2 (cont’d) • Choose random sample S  X size s • Define

26. Choosing k1, k2 (cont’d) • Lemma 2 implies

27. Expected Time Bound

28. Small Cost of Recursions • Note • With prob  1 - 2N- each recursive call costs only O(s) = o(N) rather than N in previous algorithm

29. Randomized Sorting Algorithms • “canonical sorting algorithm” • Algorithm cansort(X) begin if x={x} then output X else choose a random sample S of X of size s Sort S S subdividesX into s+1 subsets X1, X2, …, Xs+1 output cansort (X1) cansort (X2)… cansort(Xs+1) end

30. Using Random Sampling to Aid Sorting • Problem: • Must subdivide X into subsets of nearly equal size to minimize number of comparisons • Solution: random sampling!

31. Hoar’s Randomized Sorting Algorithm • Uses sample size s=1 • Algorithm quicksort(X) begin if |X|=1 then output X else choose a random splitter k  X output quicksort ({x  X|x<k})· (k)· quicksort ({x  X|x>k}) end

32. Expected Time Cost of Hoar’s Sort • Inefficient: • Better to divide problem size by ½ with high likelihood!

33. Improved Sort using • Better choice of splitter is k = sample selects (N/2, N) • Algorithm samplesorts (X) begin if |X|=1 then output X choose a random subset S of X size s = N/log N k  select (S/2,S) cost time o(N) output samplesorts ({x  X|x<k})· (k) · samplesorts ({x  X|x>k}) end

34. Randomized Approximant of Mean • By Lemma 2, rank(k,X) is very nearly the mean:

35. Improved Expected Time Bounds • Is optimal for comparison trees!

36. Open Problems in Selection and Sorting • Improve Randomized Algorithms to exactly match lower bounds on number of comparisons • Can we de randomize these algorithms – i.e., give deterministic algorithms with the same bounds?

37. Randomized Algorithms for Selection and Sorting Analysis of Algorithms Prepared by John Reif, Ph.D.