Limitations of sorting by comparison (chapter 8)

1. Limitations of sorting by comparison(chapter 8) Prove the lower bound on worst-case runtime is W(nlgn) Unless the input has special properties, cannot expect comparison sorting to be faster than nlgn For special types of inputs that allow faster sorting Counting sort uses array indexing based on assumption that the input is in the set {1,2,...k} T(n) = O(n) Radix sort used fact that input is integers T(n) = O(n) Bucket sort uses info about the probability distribution of numbers in the input array <T(n)> = O(n)

2. Overview of Sort Algorithms Sorting lineup Insertion sort (worst case is Q(n2)) sorts in place Merge sort is faster (Q(nlgn)) but does not sort in place Heap sort (chapter 6) sorts in place with runtime bounded by O(nlgn) uses important data structure called a “heap” heaps have other uses (e.g. priority queue) Quicksort (chapter 7) worst case runtime is Q(n2) average runtime is Q(nlgn) All of the above use comparison of elements as basis for sorting

3. Chapter 7: Quicksort Quicksort is one of the rare cases where analysis of average runtimes is useful. Usually average runtime and worst-case runtime have the same order of growth Worst-case runtime = Q(n2) Runtime for a Randomized Quicksort = Q(nlgn) Quicksort “sorts in place” by recursive calls to Partition

5. Continue until j = r-1, then exchange A[i+1] and A[r] Note: upper and lower sub-arrays are not the same size and not sorted Note: pivot is sorted relative elements in upper and lower sub-arrays Note: T(n) =Q(n) why?

6. Performance of Quicksort Pseudocode Quicksort(A,1,n=length[A]) if 1 < n then q  Partition(A,1,n) Quicksort(A,1,q-1) Quicksort(A,q+1,n) Worst-case partitioning: (always unbalanced) q  Partition(A,1,n) returns q = 1, then call Quicksort(A,1,0) which sorts subarray of size 0 Quicksort(A,2,n) which sorts subarray of size n-1 If every recursive call to partition were this unbalanced then runtime characterized by the recurrence (explain Q(n)) T(n) < T(n-1)+Q(n), which has solution T(n) = O(n2) Best-case partitioning: (always balanced) q  Partition(A,1,n) returns q = n/2, then T(n) > 2T(n/2) + Q(n), which has solution T(n) = W(nlgn)

7. If neglect cost of leaves, analysis of either branch gives the same result. Why?

8. Rigorous Analysis of Quicksort Performance Worst Case Assuming T(n) = T(n-1) + Q(n) is “hand waving” worst-case analysis of Quicksort Another way to define worst case is • T(n) = (T(q) + T(n-q-1)) + Q(n) This say “regardless of the value of q, choose the larger partition” This recurrence can be solved by the substitution method

9. T(n) = (T(q) + T(n-q-1)) + Q(n) Guess T(n) = O(n2) Write cost of partition as dn T(n) < (cq2 + c(n-q-1)2) + dn < c ( (q2 + (n-q-1)2)) + dn f(q)=q2+((n-1)-q)2 f(0)=(n-1)2 < c (n – 1)2 + dn f(n-1)=(n-1)2 < cn2 – 2cn + c + dn f(((n-1)/2)=0.5(n-1)2 < cn2 – (c(2n – 1) – dn) < cn2 if c(2n – 1) – dn > 0 true if c = d > 0 and n > 1 Similarly show that T(n) = W(n2) (exercise 7.4-1, p184)  T(n) = Q(n2).

10. Randomized Quicksort Randomized by a random choice of pivot. Randomized-Partition(A, p, r) i  Random(p, r) exchange A[r]  A[i] return Partition Randomized-Quicksort(A, p, r) if p < r then q  Ramdomized-Partition(A, p, r) Randomized-Quicksort(A, p, q-1) Randomized-Quicksort(A, q+1, r) Randomized-Partition just calls Partition after randomly choosing a pivot. Find the expected run time of Randomized Quicksort What is the expected number of comparison from all calls to Randomized-Partition in a complete execution of Randomized-Quicksort(A,1,n)?

11. Last line of Partition stores pivot between partitioned subarrays Once used, a pivot is never involved in subsequent recursive calls • In sorting an array of length n, Partition called at most n times Elements compared to pivots only Elements in separate subarrays are never compared

12. What is the expected number of comparison from all calls (at most n) to Randomized-Partition in a complete execution of Randomized-Quicksort? Rename the elements <a1, a2, ..., an> of array A by their order statistic Call then z1, z2, ..., zn, where zi is the ith smallest element z1 is the minimum element, zn is maximum element Define subsets of elements Zij = {zi, zi+1, ..., zj} to be the elements (by size) between zi and zj inclusively What is the probability that zi and zj are compared in Ramdonized-Quicksort(A,1,n=length[A])? Since elements are compared to pivots only and elements in separate subarrays are never compared, zi and zj are compared, at most, once when one or the other is a pivot.

13. Let Aij be the event when zi compared to zj Xij = I{Aij} Let X be random variable for number of comparisons make in all calls to partition in Ramdonized-Quicksort(A,1,n=length[A]) Since any pair of elements compared, at most, once, X = (i.e. sum over all ordered pairs) By the linearity of expected values E[X] = By the property of indicator random variables E[X] =

14. Let pivot x be used to create subarrays A[p,...q-1] and A[q+1,...r] x will be compared to every element in A[p,...r] no element in A[p,...q-1] will be compared to an element in A[q+1,...r] • if zi < x < zj then zi and zj will never be compared to each other zi and zj are compared if and only if the 1st pivot chosen from the set Zij={zi, zi+1,...zj} is either zi and zj. If zi is the 1st pivot chosen from the set Zij then zj cannot be the 1st pivot chosen from that set two possibilities leading to comparison of zi and zj are mutually exclusive.  Pr{ zi is compared to zj} = Pr{ zi is the 1st pivot chosen from Zij} + Pr{ zj is the 1st pivot chosen from Zij} = 2/(j – i + 1) All elements in Zij equally likely to be first chosen

15. E[X] = E[X] = (by change of variable k = j-i) E[X] < E[x] < O(lg n) E[x] = O(n lg n) Harmonic sum

16. CptS 450 Spring 2014 All problems are from Cormen et al, 3rd Edition Homework Assignment 6: due 3/26/14 1. ex 7.2-2 p 178 2. ex 7.2-5 p 178 3. ex 7.4-1 p 184 4. ex 7.4-2 p 184