1 / 77

Big-O and Sorting

Big-O and Sorting. February 6, 2006. Administrative Stuff. Readings for today: Ch 7.3-7.5. Readings for tomorrow: Ch 8. Sorting!. Very common to need data in order Viewing, printing Faster to search, find min/max, compute median/mode, etc. Lots of different sorting algoritms

aubrey-livingston
Download Presentation

Big-O and Sorting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Big-O and Sorting February 6, 2006

  2. Administrative Stuff • Readings for today: Ch 7.3-7.5 • Readings for tomorrow: Ch 8

  3. Sorting! • Very common to need data in order • Viewing, printing • Faster to search, find min/max, compute median/mode, etc. • Lots of different sorting algoritms • From the simple to very complex • Some optimized for certain situations (lots of duplicates, almost sorted, etc.) • Typically sort arrays, but algorithms usually can be adapted for other data structures (e.g. linked lists)

  4. Selection sort • Sort by "selecting" smallest and putting in front • Search entire array for minimum value • Min is placed in first slot • Could move elements over to make space, but faster to just swap with current first • Repeat for second smallest, third, and so on

  5. Selection sort code void SelectionSort(int arr[], int n) { for (int i = 0; i < n-1; i++) { int minIndex = i; for (int j = i+1; j < n; j++) { if (arr[j] < arr[minIndex]) minIndex = j; } Swap(arr[i], arr[minIndex]); } }

  6. Analyzing selection sort for (int i = 0; i < n-1; i++) { int minIndex = i; for (int j = i+1; j < n; j++) { if (arr[j] < arr[minIndex]) minIndex = j; } Swap(arr[i], arr[minIndex]); } • Count statements • First time inner loop N-1 comparisons • N-2 second time, then N-3, … • Last iteration 1 comparison

  7. Analyzing selection sort • N-1 + N-2 + N-3 + … + 3 + 2 + 1 • "Gaussian sum" • Add sum to self Sum =

  8. Analyzing selection sort • N-1 + N-2 + N-3 + … + 3 + 2 + 1 • "Gaussian sum" • Add sum to self N-1 + N-2 + N-3 + … + 3 + 2 + 1 + 1 + 2 + 3 + …. + N-2 + N-1 = N + N + N + …. + N + N = (N-1)N Sum = 1/2 * (N-1)N O(N2)

  9. Quadratic growth • In clock time • 10,000 3 sec • 20,000 13 sec • 50,000 77 sec • 100,000 5 min • Double input -> 4X time • Feasible for small inputs, quickly unmanagable • Halve input -> 1/4 time • Hmm… • If two sorted half-size arrays, how to produce sorted full array?

  10. Mergesort • "Divide and conquer" algorithm • Divide array in half • Recursively sort each half • Merge two halves together • "Easy-split hard-join" • No complex decision about which goes where, just divide in middle • Merge step preserves ordering from each half

  11. 6 2 8 5 1 9 3 7 4 10

  12. void MergeSort(int array[], int n) { if (n > 1) { int n1 = n/2; int n2 = n - n1; int *arr1 = CopySubArray(array, 0, n1); int *arr2 = CopySubArray(array, n1, n2); MergeSort(arr1, n1); MergeSort(arr2, n2); Merge(array, arr1, n1, arr2, n2); delete[] arr1; delete[] arr2; } }

  13. CopySubArray // Create a new array in memory void CopyArray(int arr[], int n, int * & copy) { copy = new int[n]; for(int i = 0; i < n; i++) { copy[i] = arr[i]; } }

  14. Merge code void Merge(int array[], int arr1[], int n1, int arr2[], int n2){ int p = 0, p1 = 0, p2 = 0; while (p1 < n1 && p2 < n2) { if (arr1[p1] < arr2[p2]) array[p++] = arr1[p1++]; else array[p++] = arr2[p2++]; } while (p1 < n1) array[p++] = arr1[p1++]; while (p2 < n2) array[p++] = arr2[p2++]; }

  15. void Merge(int array[], int arr1[], int n1, int arr2[], int n2) { int p, p1, p2; p = p1 = p2 = 0; while (p1 < n1 && p2 < n2) { // Merge until hit if (arr1[p1] < arr2[p2]) { // end of one array array[p++] = arr1[p1++]; } else { array[p++] = arr2[p2++]; } } while (p1 < n1) { // Merge rest of array[p++] = arr1[p1++]; // remaining array } while (p2 < n2) { array[p++] = arr2[p2++]; } } arr1 n1 p1 array 4 7 8 12 4 arr2 n2 p2 5 9 16 18 4 p

  16. Merge sort analysis void MergeSort(int array[], int n) { if (n > 1) { int n1 = n/2; int n2 = n - n1; int *arr1 = CopySubArray(array, 0, n1); int *arr2 = CopySubArray(array, n1, n2); MergeSort(arr1, n1); MergeSort(arr2, n2); Merge(array, arr1, n1, arr2, n2); delete[] arr1; delete[] arr2; } }

  17. + = N/2 + N/2 MS(N/2) MS(N/2) + = 4*N/4 N/4 N/4 N/4 N/4 + = 8*N/8 N/8 N/8 N/8 N/8 N/8 N/8 N/8 N/8 ... Each level contributes N Merge sort analysis = N MS(N)

  18. K levels … N/2K Merge sort analysis MS(N) MS(N/2) MS(N/2) N/4 N/4 N/4 N/4 N/8 N/8 N/8 N/8 N/8 N/8 N/8 N/8 N/2K = 1 N = 2K lg N = K lg N levels * N per level= O(NlgN)

  19. In clock time • Compare SelectionSort to MergeSort • 10,000 3 sec .05 sec • 20,000 13 sec .15 sec • 50,000 78 sec .38 sec • 100,000 5 min .81 sec • 200,000 20 min 1.7 sec • 1,000,000 8 hrs (est) 9 sec • O(NlgN) is looking pretty good! But can we do even better?

  20. Can we do even better than MergeSort? • O(N log N) is fastest sort in the general case • So, theoretically, answer is “no” • But, we can come up with a different O(N log N) sort that is practically faster • Want to avoid overhead of creating new arrays (as is done in MergeSort) • Bring on the QuickSort!

  21. 5 3 7 4 8 6 2 1 Quicksort

  22. Recursive Insight 5 3 7 4 8 6 2 1

  23. Recursive Insight 5 3 7 4 8 6 2 1 select “pivot”

  24. Recursive Insight 5 3 7 4 8 6 2 1 • Partition array so: • everything smaller than pivot is on left • everything greater than or equal to pivot is on right • pivot is in-between

  25. Recursive Insight 2 3 1 4 5 6 8 7 • Partition array so: • everything smaller than pivot is on left • everything greater than or equal to pivot is on right • pivot is in-between

  26. Recursive Insight 2 3 1 4 5 6 8 7 Now recursive sort “red” sub-array

  27. Recursive Insight 1 2 3 4 5 6 8 7 Now recursive sort “red” sub-array

  28. Recursive Insight 1 2 3 4 5 6 8 7 Now recursive sort “red” sub-array Then, recursive sort “blue” sub-array

  29. Recursive Insight 1 2 3 4 5 6 7 8 Now recursive sort “red” sub-array Then, recursive sort “blue” sub-array

  30. Recursive Insight 1 2 3 4 5 6 7 8 Everything is sorted!

  31. void Quicksort(int arr[], int n) { if (n < 2) return; int boundary = Partition(arr, n); // Sort subarray up to pivot Quicksort(arr, boundary); // Sort subarray after pivot to end Quicksort(arr+boundary+1,n–boundary-1); } “boundary” is the index of the pivot This is equal to the number of elements before pivot

  32. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; }

  33. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1

  34. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1 pivot lh rh

  35. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1 pivot lh rh

  36. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1 pivot lh rh

  37. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1 pivot lh rh

  38. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1 pivot lh rh

  39. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 7 4 8 6 2 1 pivot lh rh

  40. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  41. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  42. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  43. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  44. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  45. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  46. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  47. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  48. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 8 6 2 7 pivot lh rh

  49. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 2 6 8 7 pivot lh rh

  50. int Partition(int arr[], int n) { int lh = 1, rh = n - 1; int pivot = arr[0]; while (true) { while (lh < rh && arr[rh] >= pivot) rh--; while (lh < rh && arr[lh] < pivot) lh++; if (lh == rh) break; Swap(arr[lh], arr[rh]); } if (arr[lh] >= pivot) return 0; Swap(arr[0], arr[lh]); return lh; } 5 3 1 4 2 6 8 7 pivot lh rh

More Related