Chapter 7 Sorting

1. Chapter 7 Sorting Instructors: C. Y. Tang and J. S. Roger Jang All the material are integrated from the textbook "Fundamentals of Data Structures in C" and  some supplement from the slides of Prof. Hsin-Hsi Chen (NTU).

2. Sequential Search /* maximum size of list plus one */ # define MAX_SIZE 1000 typedef struct { int key;/* other fields */ } element; element list[MAX_SIZE];

3. Sequential Search Int seqsearch(int list[], int searchnum, int n) { /* search an array, list, that has n numbers. Return i, if list[i] = searchnum. Return -1, if searchnum is not in the list */ int i; // sentinel that signals the end of the list list[n].key = searchnum; for (i = 0; list[i].key != searchnum; i++) ; return ((i < n) ? i : -1); }

4. Analysis of Seq Search • Example 44, 55, 12, 42, 94, 18, 06, 67 • unsuccessful search • n+1 • The average number of comparisons for a successful search is

5. Binary Search • Input : sorted list(i.e. list[0].key≦list[1].key≦ ... ≦list[n-1].key) • compare searchnum and list[middle].key, where middle=(n-1)/2, There are three possible outcomes: • searchnum < list[middle].key • search list[0] and list[middle-1] • searchnum = list[middle].key • search terminal successfully • searchnum > list[middle].key • search list[middle+1] and list[n-1]

6. Binary Search • Input : 4,15,17,26,30,46,48,56,58,82,90,95 • It is easily to see that binary search makes no more than O(log n) comparisons (i.e. height of tree ). [5] 46 [2] [8] 17 58 [0] [3] [6] [10] 4 26 48 90 [1] [9] [11] [4] [7] 15 30 56 82 95 Decision tree for binary search

7. Binary Search int binsearch(element list[], int searchnum, int n) { int left = 0, right = n-1, middle; while (left <= right){ middle = (left + right) / 2; switch (COMPARE(list[middle].key, searchnum)){ case -1 : left = middle + i; break; case 0 : return middle; case 1 : right = middle - 1; } } return -1; }

8. List Verification • Compare lists to verify that are identical, or to identify the discrepancies. • The problem of list verification is an instance of repeatedly searching one list, using each key in the other list as the search key. • example • international revenue service (e.g., employee vs. employer) • complexities • random order: O(mn) • ordered list: O(tsort(n)+tsort(m)+m+n)

9. Verifying using a sequential search /* compare two unordered lists list1 and list2 */ void verify1(element list1[], element list2[ ], int n, int m) { int i, j; int marked[MAX_SIZE]; for(i = 0; i<m; i++) marked[i] = FALSE; for (i=0; i<n; i++) if ((j = seqsearch(list2, m, list1[i].key))<0) printf("%d is not in list 2\n", list1[i].key); else marked[j] = TRUE; for ( i=0; i<m; i++) if (!marked[i]) printf("%d is not in list1\n", list2[i]key); }

10. Fast verification of two lists /* Same task as verify1, but list1 and list2 are sorted */ void verify2(element list1[ ], element list2 [ ], int n, int m) { int i, j; sort(list1, n); sort(list2, m); i = j = 0; while (i< n && j< m) if (list1[i].key<list2[j].key) { printf ("%d is not in list 2 \n", list1[i].key); i++; }

11. Fast verification of two lists else if (list1[i].key == list2[j].key) { i++; j++; } else { printf("%d is not in list 1\n", list2[j].key); j++; } for(; i<n; i++) printf ("%d is not in list 2\n", list1[i].key); for(; j<m; j++) printf("%d is not in list 1\n", list2[j].key); }

12. complexities • verify1︰ • O(mn) (randomly arranged) • verify2︰ • O(tsort(n) + tsort(m) + m + n) (sorted list)

13. Sorting Problem Definitions • We are given a list of records (R0, R1, ... , Rn-1).each record, Ri, has a key value, Ki. • Find a permutation σ such that︰ • sorted • Kσ(i-1) ≦ Kσ(i), for 0 < i ≦ n-1. • stable • If i<j and Ki=Kj in the list, then Ri precedes Rj in the sorted list. • criteria • # of key comparisons • # of data movements

14. Insertion sort • Insert a record Ri into a sequence of ordered records, R0,R1,...,Ri-1.(K0≦K1≦ ... ≦Ki-1). • Time Complexity : O(n2) • Stable • The fastest sort when n is small (n≦20)

15. Insertion sort

16. Insertion sort /* perform a insertion sort on the list */ void insertion_sort(element list[], int n) { int i, j; element next; for (i=1; i<n; i++) { next= list[i]; for (j=i-1; j>=0 && next.key<list[j].key ; j--) list[j+1] = list[j]; list[j+1] = next; } }

17. Insertion sort worse case i 0 1 2 3 4 - 5 4 3 2 1 1 4 5 3 2 1 2 3 4 5 2 1 3 2 3 4 5 1 4 1 2 3 4 5

18. Insertion sort O(n) best case i 0 1 2 3 4 - 2 3 4 5 1 1 2 3 4 5 1 2 2 3 4 5 1 3 2 3 4 5 1 4 1 2 3 4 5 left out of order (LOO)

19. Insertion sort Ri is LOO if Ri < max{Rj} 0j<i k: # of records LOO Computing time: O((k+1)n) 44 55 12 42 94 18 06 67 * * * * *

20. Variation • Binary insertion sort • sequential search --> binary search • reduce # of comparisons, # of moves unchanged • List insertion sort • array --> linked list • sequential search, move --> 0

21. Quick Sort (C.A.R. Hoare) • Given (R0, R1, ... , Rn-1), pick up a pivot key Ki, if Ki is placed in position s(i), then︰ Kj≦Ks(i) for j<s(i) Kj≧Ks(i) for j>s(i) • R0, …, RS(i)-1, RS(i), RS(i)+1, …, RS(n-1) two partitions

22. Quick sort example • Given 10 records with keys (26, 5, 37, 1, 61, 11, 59, 15, 48, 19)

23. Quick sort void quicksort(element list[], int left, int right) { int pivot, i, j; element temp; if (left < right) { i = left; j = right+1; pivot = list[left].key; do { do i++; while (list[i].key < pivot); do j--; while (list[j].key > pivot); if (i < j) SWAP(list[i], list[j], temp); } while (i < j); SWAP(list[left], list[j], temp); quicksort(list, left, j-1); quicksort(list, j+1, right); } }

24. Analysis for quick sort • Assume that each time a record is positioned, the list is divided into the rough same size of two parts. • Position a list with n element needs O(n) • T(n) is the time taken to sort n elementsT(n)<=cn+2T(n/2) for some c <=cn+2(cn/2+2T(n/4)) ... <=cnlog n+nT(1)=O(nlog n)

25. Analysis for quick sort • Time complexity: • Worst case: O(n2) • Best case: O(nlogn) • Average case: O(nlogn) • Space complexity • Worst case: O(n) • Best case: O(logn) • Average case: O(logn) • Unstable

26. Variation • Our version of quick sort always picked the key of the first record in the current sublist as the pivot. • A better choice for this pivot is the median of the first, middle, and last keys in the current sublist.

27. Optimal sorting time • How quickly can we hope to sort a list of n object? • If we restrict our question to algorithms that permit only the comparison and interchange of keys, then the answer is O(n log2n)

28. Decision tree for insertion sort K0≦K1 [0,1,2] Yes No K0≦K2 [1,0,2] K1≦K2 [0,1,2] Yes No Yes No stop [0,1,2] K0≦K2 [0,1,2] K1≦K2 [1,2,0] stop [1,0,2] Yes No Yes No stop [0,1,2] stop [0,1,2] stop [2,1,0] stop [1,2,0]

29. Optimal sorting time • Theorem 7.1: Any decision tree that sorts n distinct elements has a height of at least log2(n!) +1 • Corollary: Any algorithm that sorts by comparisons only must have a worst case computing time ofΩ(n log2n).

30. Merge sort • How to merge two sorted lists (list[i], ... , list[m] and list[m+1], ... , list[n]) into single sorted list, (sorted[i], ... , sorted[n])? • There are two algorithm to do this, one need O(n) space complexity, another only require O(1).

31. Merge sort (O(n) space) void merge(element list[], element sorted[], int i, int m, int n) { int j, k, t; j = m+1; k = i; while (i<=m && j<=n) { if (list[i].key<=list[j].key) sorted[k++]= list[i++]; else sorted[k++]= list[j++]; } if (i>m) for (t=j; t<=n; t++) sorted[k+t-j]= list[t]; else for (t=i; t<=m; t++) sorted[k+t-i] = list[t]; }

32. Analysis • Time complexity: O(n) • Space complexity: O(n)

33. *Figure 7.5:First eight lines for O(1) space merge example (p337) file 2 (sorted) file 1 (sorted) discover keys exchange preprocessing exchange sort Sort by the rightmost records of -1 blocks compare exchange compare exchange compare

34.    0 1 2 y w z u x 4 6 8 a v 3 5 7 9 b c e g i j k d f h o p q l m n r s t    0 1 2 3 w z u x 4 6 8 a v y 5 7 9 b c e g i j k d f h o p q l m n r s t    0 1 2 3 4 z u x w 6 8 a v y 5 7 9 b c e g i j k d f h o p q l m n r s t    0 1 2 3 4 5 u x w 6 8 a v y z 7 9 b c e g i j k d f h o p q l m n r s t

35. *Figure 7.6:Last eight lines for O(1) space merge example(p.338) 6, 7, 8 are merged Segment one is merged (i.e., 0, 2, 4, 6, 8, a) Change place marker (longest sorted sequence of records) Segment one is merged (i.e., b, c, e, g, i, j, k) Change place marker Segment one is merged (i.e., o, p, q) No other segment. Sort the largest keys.

36. O(1) Space merge sort

37. Iterative Merge Sort • We assume that the input sequence has n sorted lists each of length 1. • We merge these lists pairwise to obtain n/2 list of size 2. • We then merge the n/2 lists pairwise, and so on, until a single list remains.

38. Iterative Merge Sort example

39. merge_pass function void merge_pass(element list[], element sorted[],int n, int length) { int i, j; for (i=0; i<n-2*length; i+=2*length) merge(list,sorted,i,i+length-1,i+2*lenght-1); if (i+length<n) merge(list, sorted, i, i+lenght-1, n-1); else for (j=i; j<n; j++) sorted[j]= list[j]; } One complement segment and one partial segment Only one segment i i+length-1 i+2length-1 ... 2*length ...

40. merge_sort function void merge_sort(element list[], int n) { int length=1; element extra[MAX_SIZE]; while (length<n) { merge_pass(list, extra, n, length); length *= 2; merge_pass(extra, list, n, length); length *= 2; } } l l l l ... 2l 2l ... 4l ...

41. Recursive Formulation of Merge Sort 26 5 77 1 61 11 59 15 48 19 copy copy 5 26 11 59 copy copy 5 26 77 1 61 11 15 59 19 48 1 5 26 61 77 11 15 19 48 59 (1+5)/2 (6+10)/2 1 5 11 15 19 26 48 59 61 77 (1+10)/2 Data Structure: array (copy subfiles) vs. linked list (no copy)

42. Simulation of merge sort start=3

43. Recursive Merge Sort lower upper int rmerge(element list[], int lower, int upper) { int middle; if (lower >= upper) return lower; else { middle = (lower+upper)/2; return listmerge(list, rmerge(list,lower,middle), rmerge(list,middle, upper)); } } Point to the start of sorted chain lower middle upper

44. List Merge int listmerge(element list[], int first, int second) { int start=n; while (first!=-1 && second!=-1) { if (list[first].key<=list[second].key) { /* key in first list is lower, link this element to start and change start to point to first */ list[start].link= first; start = first; first = list[first].link; } first ... second

45. List Merge else { /* key in second list is lower, link this element into the partially sorted list */ list[start].link = second; start = second; second = list[second].link; } } if (first==-1) list[start].link = second; else list[start].link = first; return list[n].link; } first is exhausted. second is exhausted. O(nlog2n)

46. Nature Merge Sort 26 5 77 1 61 11 59 15 48 19 1 11 59 61 15 19 48 5 26 77 15 19 48 1 5 11 26 59 61 77 1 5 11 15 19 26 48 59 61 77

47. Heap Sort • The heap sort algorithm will require only a fixed amount of additional storage and at the same time will have as its worst case and average computing time O(n log n). • While heap sort is slightly slower than merge sort using O(n) additional space, it is faster than merge sort using O(1) additional space.

48. Heap Sort Example [1] 26 [3] [2] 77 5 [4] [5] [6] [7] 1 61 11 59 [8] [9] [10] 15 48 19 Array interpreted as a binary tree

49. Heap Sort Example initial heap [1] 77 [3] [2] 59 61 exchange [4] [5] [6] [7] 48 19 11 26 [8] [9] [10] 15 1 5 Max heap following first for loop of heapsort

50. Heap Sort Example [1] 61 [3] [2] 59 48 [4] [5] [6] [7] 15 19 11 26 [8] [9] [10] 5 1 77