Sorting Algorithms and Average Case Time Complexity

1. Sorting Algorithms and Average Case Time Complexity • Simple Sorts O(n2) • Insertion Sort • Selection Sort • Bubble Sort • More Complex Sorts O(nlgn) • Heap Sort • Quick Sort • Merge Sort

2. Heap Sort • Remember the heap data structure . . . • Binary heap = • a binary tree that is • Complete • satisfies the heap property

3. Heap Sort (cont’d) • Given an array of integers: • First, use MakeHeap to convert the array into a max-heap. • Then, repeat the following steps until there are no more unsorted elements: • Take the root (maximum) element off the heap by swapping it into its correct place in the array at the end of the unsorted elements • Heapify the remaining unsorted elements (This puts the next-largest element into the root position)

4. Heap Sort (cont’d) HeapSort(A, n) MakeHeap(A) for i = n downto 2 exchange A[1] with A[i] n = n-1 Heapify(A,1)

5. Heapsort Example • 2 8 6 1 10 15 12 11

6. Heap Sort (cont’d) • Time complexity? • First, MakeHeap(A) • Easy analysis: O(nlgn) since Heapify is O(lgn) • More exact analysis: O(n) • Then: • n-1 swaps = O(n) • n-1 calls to Heapify = O(nlgn) •  O(n) + O(n) + O(nlgn) = O(nlgn)

7. Other Sorting Ideas • Divide and Conquer!

8. Quicksort • Recursive in nature. • First Partition the array, and recursively quicksort the subarray separately until the whole array is sorted • This process of partitioning is carried down until there are only one-cell arrays that do not need to be sorted at all

9. Quicksort Algorithm • Given an array of integers: If array only contains one element, return Else Pick one element to use as the pivot Partition elements into 2 subarrays: Elements less than or equal to the pivot Elements greater than or equal to the pivot Quicksort the two subarrays

10. Quicksort Example

11. Quicksort Example • There are a number of ways to pick the pivot element. Here, we will use the first element in the array

12. Quicksort Partition • Given a pivot, partition the elements of the array such that the resulting array consists of: • One sub-array that contains elements <= pivot • Another sub-array that contains elements >= pivot • The sub-arrays are stored in the original array • Partitioning loops through, swapping elements below/above pivot

13. Quicksort Example: Partition Result:

14. QuickSort template<class T> void quicksort(T data[], int first, int last) { //partition such that left <= pivot, right >= pivot int lower = first+1, upper = last; swap(data[first],data[(first+last)/2]); T pivot = data[first]; while (lower <= upper) { //find the position of the first > pivot element on the left while (data[lower] < pivot) lower++; //find the position of the first < pivot element on the right while (pivot < data[upper]) upper--; if (lower < upper) swap(data[lower++],data[upper--]); else lower++; } swap(data[upper],data[first]); if (first < upper-1) quicksort (data,first,upper-1); //quicksort left if (upper+1 < last) quicksort (data,upper+1,last); //quicksort right }

15. Quicksort - Recursion • Recursion: Quicksort sub-arrays

16. Quicksort: Worst Case Complexity • Worst case running time? • Assume first element is chosen as pivot. • Assume array is already in order: < – Recursion: 1. Partition splits array in two sub-arrays: • one sub-array of size 0 • the other sub-array of size n-1 2. Quicksort each sub-array – Depth of recursion tree? O(n) – Number of accesses per partition? O(n)

17. Quicksort: Best Case Complexity • Assume that keys are random, uniformly distributed. • Best case running time: O(n log2n)

18. Quicksort : worst case complexity in the worst case is O(n2) because it is difficult to control the partitioning process The partitioning depends on the choice of pivot, no guarantee that partitioning will result in arrays of approximately equal size Another strategy is to make partitioning as simple as possible and concentrate on merging sorted (sub)arrays Merge Sort

19. Also recursive in nature Given an array of integers, apply the divide-and-conquer paradigm Divide the n-element array into two subarrays of n/2 elements each Sort the two subarrays recursively using Merge Sort Merge the two subarrays to produce the sorted array The recursion stops when the array to be sorted has length 1 Merge Sort

20. Merge Sort Divide array into two halves Recursively sort each half Merge two halves to make sorted whole Merge Sort

21. Merging: Combine two pre-sorted lists into a sorted whole How to merge efficiently? Use temporary array Need index 1 (for array 1), index 2 (for array 2), 3 (for array tmp) Merging 1 3 19 25 5 7 8 9 29 3 5 7 1

22. Merge(A, left, middle, right) Create temporary array temp, same size as A i1 = left; i2 = middle; i3 = 0; while ((i1 != middle) && (i2 != right)) if (A[i1] < A[i2]) temp[i3++] = A[i1++] else temp[i3++] = A[i2++] copy into temp remaining elements of A copy into A the contents of temp Merging

23. Time Complexity Analysis T(n) = c1 if n =1 T(n) = 2 T(n/2) + c2n Merging divide O(1) sort 2T(n/2) merge O(n)

24. MergeSort(A, left, right) if (left < right) { middle= (left + right)/2 MergeSort(A, left, middle) MergeSort(A, middle+1, right) Merge(A, left, middle+1, right) } Merge Sort