Donald E. Knuth (1938---)

1. Donald E. Knuth (1938---) The father of algorithm analysis popularizing asymptotic notation. The author of TAOCP -- The Art of Computer Programming • Volume 1: Fundamental Algorithms • Volume 2: Seminumerical Algorithms • Volume 3: Sorting and Searching (among tons of others) “ I have been a happy man ever since January 1, 1990, when I no longer had an email address.” IT 279

2. Donald E. Knuth TAOCP – The Art of Computer Programming • Volume 1: Fundamental Algorithms Basic Concepts and Information Structures • Volume 2: Seminumerical Algorithms Random numbers and Arithmetic • Volume 3: Sorting and Searching Sorting and Searching • Volume 4: Combinatorial Algorithms Combinatorial Searching and Recursion • Volume 5: Syntactical Algorithms Lexical Scanning and Parsing Techniques • Volume 6: Theory of Languages Mathematical Linguistics • Volume 7: Compiler Programming Language Translation Highlighted in the preface of Vol. 1 IT 279

3. -- Donald E. Knuth -- “An algorithm must be seen to be believed” Sorting algorithms IT 279

4. Sorting: Arranging items into a certain order. Items can be stored in an one dimensional array or linked list, or trees. We prefer using one dimensional array Why? IT 279

5. Insertion Sort: Insert an item into a sorted the right place. sorted 7 IT 279

6. Insertion Sort: Insert an item into the right place. sorted IT 279

7. Insertion Sort: Insert an item into the right place. sorted 7 8 9 10 12 7 IT 279

8. Insertion Sort: Insert an item into the right place. IT 279

9. Selection Sort: Select the minimum item into the end of the sorted segment. sorted the minimum one the minimum one IT 279

10. Select the minimum item into the end of the sorted segment. Selection Sort: IT 279

11. Analysis of Insertion/Selection Sorts: T(n) = 1 +2 + 3 + ... + n = O(n2) Shell Sort D. L. Shell, (1959) O(n3/2) O(n log n) Lower bound of sorting algorithms IT 279

12. Incremental insertion (selection) sort. Shell Sort: h = 4 IT 279

13. Shell Sort: hk-sorted h = 4 4-sorted 2-sorted 1-sorted IT 279

14. Shell Sort: Incremental insertion with Increment sequence, hk-sorted 1-sorted ht-sorted ht-1-sorted hk-1-sorted  (implies) hk-sorted hk-1-sorted 10 9 30 20 hk-sorted Shell suggested: Not a good suggestion, worst case: O(n2) IT 279

15. Shell Sort: using Shell’s suggestion Worst case analysis: O(n2) Idea: Right before the final sort, let the smallest n/2 be distributed in the even position. 2 2 4 4 8 8 final sort 2-sorted IT 279

16. Shell Sort: using Shell’s suggestion What kind of initial array will result in such 2-sorted? Let Ah[s] = A[s], A[s+h], A[s+2h], .... odd positions Ah[1] Ah[0] even positions 5 13 1 11 3 Ah[2] 12 7 4 15 9 10 8 Ah[3] 6 2 14 16 Ah[5] Ah[6] Ah[4] Ah[7] elements will never cross the even-/odd-cell before the final sort 8-sort , 4-sort , 2-sort IT 279

17. void shellsort(int a[], int size) { int h = size/2; while (h > 0) { // h-sort for (int i=h; i<size;i++) { int t=a[i], j=i; while (j >= h && a[j-h] > t) { a[j] = a[j-h]; j -= h; } a[j]=t; } h /= 2; } // end increment } Shell sort algorithm bad strategy h = 4 h = 4 h = 4 i h = 4 j j j j j-h j,i j-h j, i t t IT 279

18. hk+1-sorted hk+2-sorted hk+1 hk+1 hk+1 hk+1 hk+1 hk+2 hk+2 hk+2 A[p] A[p-(xhk+2+yhk+1)]  A[p-(xhk+2+yhk+1)] < A[p] If j is a liner combination of hk+2and hk+1 with positive coefficients, then there is no need to check A[p-j] < A[p] xp+yq= gcd(p,q) if gcd(p,q) = 1, then any number is a liner combination of p and q IT 279

19. A[p-(xhk+2+yhk+1)] < A[p] hk+1-sorted hk+2-sorted hk+1 hk+1 hk+1 hk+1 hk+1 hk+2 hk+2 hk+2 A[p] A[p-j] If j is a liner combination of hk+2and hk+1 with positive coefficients, then there is no need to check A[p-j] < A[p] while (h > 0) { // h-sort for (int i=h; i<size;i++) { int t=a[i], j=i; while (j >= h && a[j-h] > t) { a[j] = a[j-h]; j -= h; } a[j]=t; } while (h > 0) { // h-sort for (int i=h; i<size;i++) { int t=a[i], j=i; while (x > j && j >= h && a[j-h] > t) { a[j] = a[j-h]; j -= h; } a[j]=t; } IT 279

20. A[p-(xhk+2+yhk+1)] < A[p] hk+1-sorted hk+2-sorted hk+1 hk+1 hk+1 hk+1 hk+1 hk+2 hk+2 hk+2 A[p] A[p-j] xp+yq= gcd(p,q) If gcd(hk+1, hk+2) = 1, then for sufficiently large j, we have that j is a liner combination of hk+2and hk+1 with positive coefficients, there is no need to check A[p-j] < A[p] IT 279

21. gcd(hk+1, hk+2) = 1,   A[p-j] < A[p] IT 279

22. while (h > 0) { // h-sort for (int i=h; i<size;i++) { int t=a[i], j=i; while (x > j && j >= h && a[j-h] > t) { a[j] = a[j-h]; j -= h; } a[j]=t; }  IT 279

23. Heapsort Percolating a non-heap Worst case: O(n) 34 2 2 15 5 30 7 7 6 15 2 23 5 9 30 21 7 25 6 23 9 21 15 6 32 23 5 40 11 31 30 21 25 7 15 32 34 40 11 31 30 25 IT 279

24. Heapsort O(log n) 2 Percolating a non-heap O(log n-1) 5 Worst case: O(n) O(log n-2) 6 7 2 5 7 6 23 9 21 40 O(log 1) 15 32 34 40 11 31 30 25 O(n+n log n) = O(n log n) IT 279

25. Mergesort Merge two sorted lists 3 5 7 8 1 2 6 O(n) IT 279

26. Merge Sort O(log n) O(n log n) IT 279

27. Quick Sort < < < < < < < < < < < < < < < < < < < < < < < < < < IT 279

28. Quick Sort 4 3 2 16 14 8 IT 279

29. /* Quick Sort */ void SwapTwo(int A[], int i, int j){ int temp = A[i]; A[i]=A[j]; A[j]=temp; } void QuickSort(int A[], int h, int t){ // h: head // t: tail if (h == t) return; inti=h+1, j=t; if (i==j) { if (A[h] > A[i]) SwapTwo(A,h,i); return; } while (i < j) { while (A[h] >= A[i] && i < t) i++; while (A[h] <= A[j] && j > h) j--; if (i < j) SwapTwo(A,i++,j--); if (i==j && A[h]>A[i]) SwapTwo(A,h,i); } QuickSort(A,h,i-1); QuickSort(A,i,t); } Quick Sort IT 279