Non-Comparison Based Sorting

1. Non-Comparison Based Sorting

2. How fast can we sort? • Insertion Sort: O(n2) • Merge Sort, Quick Sort (expected), Heap Sort: O(nlgn) • Can we sort faster than O(nlgn)?

3. Comparison-Based Sort • Can prove that any comparison-based sort MUST make Ω(nlgn) comparisons. • Comparison-based sort: algorithm directly compares elements of the array to one another, makes some decision based upon <, =, > • This means that heapsort and mergesort are optimal. • Not always possible to prove that your algorithm is the best possible! • Proof idea • Use a decision tree to represent comparisons of a sorting algorithm • Actual sorting algorithm maps to the decision tree

4. Decision Tree Example • Three elements to sort: a1, a2, a3 Decision tree must have n! leaves, one for each permutation of the input

5. Decision Tree • For any comparison-based sorting algorithm • One tree exists for each input of length n • An algorithm “splits” at each decision/comparison unwinding the actual execution into a tree path • The tree is for all possible execution traces for an input of length n • Height of the tree tells us the minimum number of comparisons necessary to sort the input • There must be n! leaves • But we also have a binary tree; binary tree of height h has no more than 2h leaves • This means that n!  2h

6. Height of Tree • n!  2h • lg(n!)  lg(2h) • Stirlings approximation says: • Giving us: This means h is Ω(nlgn)

7. Better than O(nlgn)? • This means any comparison-based sorting algorithm must require at least time O(nlgn) • But we can do better if we use a non-comparison-based sorting algorithm! • Count Sort • Radix Sort • Bucket Sort • Postman Sort

8. Count Sort • May run in O(n) time • But requires assumptions about the size and nature of the input • Input: A[1..n] where a[i] is an integer in the range 1..k • Output: B[1..n], array input values sorted • Uses: C[1..k] auxiliary storage • Idea: Using the random access of the array C, count up the number of times each input element appears and then collect them together

9. Count Sort Algorithm Count-Sort(A,n) for i=1 to k do C[i]=0 ; Initialize to 0 for j=1 to n do C[A[j]] ++ ; Count j=1 for i=1 to k do if (C[i]>0) then for z=1 to C[i] do B[j]=i j++ Ex: A=[1 5 3 2 2 4 9] ; Initial values C=[0 0 0 0 0 0 0 0 0] ; Initialize C to 0 C=[1 2 1 1 1 0 0 0 1] ; After count loop B=[1 2 2 3 4 5 9] ; After last loop Runtime: O(n+k). If k is close to n, then runs in O(n) time. A bad example would be a list such as: A[1,2,999999999999]

10. Stable Count Sort • A sorting algorithm is stable if the occurrence of a value I appears in the same value in the output as they do in the input • Ties broken by the rule of whichever appeared first in the input appears first in the output • Desired for various algorithms where sorting a key of a record (e.g. a zip code) • Example: • A[3,5a,9,2,4,5b,6] • Sorts to : A[2,3,4,5a,5b,6,9] not A[2,3,4,5b,5a,6,9] • Prior algorithm was not stable but we can modify to make it stable

11. Stable Count Sort Key idea: fetch from original array to make it stable.

12. Radix Sort • Works like the punch card readers of the early 1900’s • Only works on input items with digits • E.g. numbers, letters • Idea: Sort on the least significant digit first ; Array A of n elements, each element d digits Radix-Sort(A,d,n) for i = 1 to d Stable-Sort(A) with digit i as the key

13. Radix Sort Example

14. Radix Sort Notes • Internal sort must be stable to preserve order of elements. Based on the observation that if all the lower order digits are already sorted, then we only need to sort on the highest order digit. • Good choice to use if each digit is small in magnitude • If k is the maximum value of a digit • If using stable count sort internally • Stable-Count-Sort takes Θ(k+n) time for one pass • With d passes total, runtime is Θ(dk+dn) • Runtime is Θ(n) if d is a constant, k is same magnitude as n

15. Bucket Sort • Bucket sort works similarly to count sort, but uses a “bucket” to hold a range of inputs. This allows it to operate on real numbers. • Runs in O(n) time with the following assumptions: • Input elements are in the interval [0..1]. • If not, in many cases we can divide by some “max” value to scale the input to be between 0 and 1 • Input elements are evenly distributed • Uniform probability over [0..1]

16. Bucket Sort Algorithm • Idea: • Divide [0..1] into n equal sized “buckets” • Put each of the n inputs into the proper bucket; some may be empty and others may have more than 1 element • Sort each bucket using insertion sort • To produce output, go through the buckets in order, listing the elements in each • Algorithm: Bucket-Sort(A,n) for i=1 to n do ; implicitly make buckets 0..n-1 Insert A[i] into bucket B[n*A[i]] for i =0 to n-1 do sort bucket B[i] using insertion sort ; < O(n) if small bucket concatenate buckets B[0], B[1], .. B[n-1] in order

17. Bucket Sort Example

18. Bucket Sort Notes • Expected runtime: • If any element in A comes from [0..1] with equal probability, then the probability that an element e is in bucket B[i] is 1/n • Expect on average each bucket to have a single element • Call to insertion sort for each bucket will be extremely fast; essentially constant time • Rest of the algorithm runs in O(n) • Overall runtime O(n) if the uniform probabilty assumption holds • Worst case O(n2) if everything ends up in one bucket

19. Postman Sort • Algorithm proposed by Robert Ramey, August 1992 issue of C Programming Journal • Analysis left as a homework exercise • From the article: • When a postal clerk receives a huge bag of letters he distributes them into other bags by state. Each bag gets sent to the indicated state. Upon arrival, another clerk distributes the letters in his bag into other bags by city. So the process continues until the bags are the size one man can carry and deliver. This is the basis for my sorting method which I call the postman's sort.

20. Postman Sort Idea • Given a large list of records to sort alphabetically • Make one pass, reading each record into one of 26 lists depending on the first letter in the field • First list contains all records starting with “A” • Second list contains all records starting with “B”, etc. • Recurse on each of the 26 sublists if it contains more than one element, but this time using the next letter as the field to split up the sublists • In-order traversal of the resulting tree gives a sorted list

21. Postman Example A = [bob, bill, boy, cow, dog] c d b Empty for all other letters A = [bob, bill, boy] A = [cow] A = [dog] i o Empty for all other letters A = [bill] A = [bob, boy] b y Empty for all other letters A = [boy] A = [bob] Runtime? Left as a homework exercise