Sorting

1. Sorting Joe Meehean

2. Sorting • Problem • Arrange comparable items in list into sorted order • Most sorting algorithms involve comparing item values • We assume items define • < operator • > operator • == operator

3. Sorting in STL • void sort( Iterator begin, Iterator end) • data items must override < operator • void sort( Iterator begin, Iterator end, Comparator cmp) • Comparator is comparison functor • cmp(a,b) returns true if a should go before b in the sorted list • Often implemented using quick sort

4. Sorting • Aspects we care about • run time • memory cost • Types of algorithms • comparison vs. non comparison • Examples of important sorting algorithms • Google Search: real world example

5. Run-time complexity • Obvious algorithms are O(N2) • Clever ones are O(NlogN) • special purpose sorts can go even faster • Additional considerations • does algorithm always take worst-case time? • what is the average case? • what happens when the list is already sorted?

6. Memory Cost • In-place • sorts list with constant extra memory • e.g., temporary variables • Not in-place • requires additional memory in relation to input size • e.g., another parallel list

7. 2 Kinds of Sorting Algorithms • Comparison sort • compare items in the list • place smaller items near the front • fastest worst case: O(NlogN) • Non-comparison sort • sort using special properties of items • use/extrapolate additional information • e.g., non-comparison sort O(Range+N)

8. Heap Sort • Sorting algorithm based on heaps • Idea • insert items from unsorted list into heap • use heap::removeMin to get items out of heap in sorted order • put items back into list in sorted order

9. Heap Sort • Problems with this approach • Complexity not ideal • inserting N items into heap is O(NlogN) • removing N items from heap is O(NlogN) • it would be better if would could do the whole thing in O(NlogN) • Memory cost • not in-place • need original list + a heap

10. Heap Sort: Improved Complexity • Can “heapify” a vector/array in O(N) • convert unsorted vector into a min heap • For each parent node (N/2 to 0) • make sure its larger than its children • if its not, swap parent with largest child • shiftDown(intpos, K val) • Minor complication • vector starts at 0 • not 1 like a normal heap

11. Heapify P L 1 7 0 9 4 3 6 5 8 2 6 0 8 9 7 3 1 2 4 5

12. Heapify P L 1 7 0 9 4 3 6 5 8 2 6 0 8 9 7 3 1 2 4 5 swap

13. Heapify P L 1 7 2 9 4 3 6 5 8 0 6 0 8 9 7 3 1 2 4 5

14. Heapify P L R 1 7 2 9 4 3 6 5 8 0 6 0 8 9 7 3 1 2 4 5

15. Heapify P L R 1 7 2 9 4 3 6 5 8 0 6 0 8 9 7 3 1 2 4 5 swap

16. Heapify P L R 1 7 2 3 4 9 6 5 8 0 6 0 8 9 7 3 1 2 4 5

17. Heapify P L R 1 7 2 3 4 9 6 5 8 0 6 0 8 9 7 3 1 2 4 5

18. Heapify P L R 1 7 2 3 4 9 6 5 8 0 6 0 8 9 7 3 1 2 4 5 swap

19. Heapify P L R 1 8 2 3 4 9 6 5 7 0 6 0 8 9 7 3 1 2 4 5

20. Heapify P L R 1 8 2 3 4 9 6 5 7 0 6 0 8 9 7 3 1 2 4 5

21. Heapify P L R 1 8 2 3 4 9 6 5 7 0 6 0 8 9 7 3 1 2 4 5 swap

22. Heapify P L R 1 8 2 3 9 4 6 5 7 0 6 0 8 9 7 3 1 2 4 5 swap R P L

23. Heapify P L R 1 8 2 3 9 5 6 4 7 0 6 0 8 9 7 3 1 2 4 5 R P L

24. Heapify P L R 1 8 2 3 9 5 6 4 7 0 6 0 8 9 7 3 1 2 4 5

25. Heapify P L R 1 8 2 3 9 5 6 4 7 0 6 0 8 9 7 3 1 2 4 5 swap

26. Heapify P L R 9 8 2 3 1 5 6 4 7 0 6 0 8 9 7 3 1 2 4 5 R P L

27. Heapify P L R And So on… 9 8 2 3 1 5 6 4 7 0 6 0 8 9 7 3 1 2 4 5 R P L

28. Heapify 9 8 2 3 5 4 6 1 7 0 6 0 8 9 7 3 1 2 4 5

29. Heapify Complexity • O(N) • proof is somewhat complex • see Weiss 6.4.3 if interested • Intuitively • it is faster because we only need to shiftdown ½ the nodes • plus starting at bottom reduces number of shift downs • inserting each node into a heap shifts down for each insert (all the nodes)

30. Heap Sort: In-place • Removing an item from the heap creates a space at the end • This space is where the largest item should go in the finished array • Why don’t we just put it there • recall in heap::removeMax we return h[first] and replace h[first] with h[last] • instead lets swap h[first] with h[last]

31. Heap Sort: In-place swap 9 8 2 3 5 4 6 1 7 0 6 0 8 9 7 3 1 2 4 5 = Sorted Vector = Heap

32. Heap Sort: In-place 0 8 2 3 5 4 6 1 7 9 6 0 8 9 7 3 1 2 4 5 = Sorted Vector = Heap

33. Heap Sort: In-place P L R 0 8 2 3 5 4 6 1 7 9 6 0 8 9 7 3 1 2 4 5 shift down = Sorted Vector = Heap

34. Heap Sort: In-place P L R 8 0 2 3 5 4 6 1 7 9 6 0 8 9 7 3 1 2 4 5 shift down = Sorted Vector = Heap

35. Heap Sort: In-place P 8 7 2 3 5 4 6 1 0 9 6 0 8 9 7 3 1 2 4 5 shift down = Sorted Vector = Heap

36. Heap Sort: In-place swap 8 7 2 3 5 4 6 1 0 9 6 0 8 9 7 3 1 2 4 5 = Sorted Vector = Heap

37. Heap Sort: In-place P L R 3 7 2 8 5 4 6 1 0 9 6 0 8 9 7 3 1 2 4 5 shift down = Sorted Vector = Heap

38. Heap Sort: In-place P L R And So on… 3 7 2 8 5 4 6 1 0 9 6 0 8 9 7 3 1 2 4 5 shift down = Sorted Vector = Heap

39. Heap Sort: In-place 0 2 4 8 1 3 5 7 6 9 6 0 8 9 7 3 1 2 4 5 = Sorted Vector = Heap

40. Heap Sort Item value Position in the Array

41. Heap Sort Complexity • Heapify • O(N) • In-place conversion of heap into sorted array • O(NlogN) • O(N) + O(NlogN) = O(NlogN) • Costs the same if array was sorted to begin with

42. Questions?

43. Quick Sort B A 0 2 4 8 1 3 5 7 6 9 0 3 1 2 4 0 3 1 2 4 A 0 2 4 8 1 3 5 7 6 9 0 6 8 9 3 7 1 2 4 5 • Fundamental Idea • if all values in sorted array A are less than all values in sorted array B • we can easily combine them • an array of size 1 is sorted

44. Quick Sort Algorithm • if number of items in A is one or zero, return • Choose a value from A to be the pivot • Partition A into sub-lists • all values ≤ pivot into left part • all values ≥ pivot into the right part • Return quicksort(L-part) + pivot + quicksort(R-part)

45. Quick Sort: On the way down 2 6 3 5 4 0 7 1

46. Quick Sort: On the way down 2 6 3 5 4 0 7 1 4

47. Quick Sort: On the way down 2 6 3 5 4 0 7 1 2 3 0 1 4 5 6 7

48. Quick Sort: On the way down 2 6 3 5 4 0 7 1 2 3 0 1 4 5 6 7 1

49. Quick Sort: On the way down 2 6 3 5 4 0 7 1 2 3 0 1 4 5 6 7 0 1 2 3

50. Quick Sort: On the way down 2 6 3 5 4 0 7 1 2 3 0 1 4 5 6 7 0 1 2 3 3