Sorting

1. Sorting CS-240 & CS-341 Dick Steflik

2. Exchange Sorting • Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1 compares) • O(n2)

3. Exchange Sorting Basic Algorithm ( no improvements) int ExchangeSort ( int & num[ ] , const int & numel ) { int i,j,k, moves = 0; for ( i = 0 ; i < (numel - 1) ; i++ ) for ( j = 0 ; j < (numel - 1) ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; } return moves; }

4. Exchange Sort Improvement #1 : don’t revist places that are already in the correct place int ExchangeSort ( int & num[ ] , const int & numel ) { int i,j,k, moves = 0; for ( i = 0 ; i < (numel - 1) ; i++ ) for ( j = i ; j < numel ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; } return moves; }

5. Exchange Sort int ExchangeSort ( int & num[ ] , const int & numel ) { int i,j,k, swap,moves = 0; swap = 1; for ( i = 0 ; i < (numel - 1) ; i++ ) if ( swap != 0 ) { swap = 0; for ( j = 0 ; j < numel ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; swap = 1; } } return moves; } // Improvement # 2: // if a pass through the inner loop occurs // that doesn’t product a swap, you’re done. // both of the improvements improve the // overall performance but to not improve // O(n2)

6. Exchange Sort 3 7 5 2 4 compare 3 and 7 ; 7 is > 3 so advance 3 5 7 2 4 compare 7 and 5, 7 > 5 so swap them 3 5 2 7 4 compare 7 and 2, 7 >4 so swap them 3 5 2 4 7 compare 7 and 4, 7 >4 so swap them End of pass 1; notice that 7 is in the right place

7. Exchange Sort - pass 2 3 5 2 4 7 3 5 2 4 7 compare 3 and 5 ; 3 is >5 so advance 3 2 5 4 7 compare 5 and 2, 5 < 2 so swap them 3 2 4 5 7 compare 5 and 4, 5 > 4 so swap them 3 2 4 5 7 compare 5 and 7, 5 < 7 so pass 2 done End of pass 2; notice that 5 and 7 are in the right places

8. Exchange Sort - pass 3 3 2 4 5 7 2 3 4 5 7 compare 3 and 2 ; 3 is >2 so swap them 2 3 4 5 7 compare 3 and 4, 3 <42 so advance 2 3 4 5 7 compare 4 and 5, 4 < 5 so advance 2 3 4 5 7 compare 5 and 7, 5 < 7 so pass 3 done End of pass 3; notice that 4, 5 and 7 are in the right places

9. Exchange Sort - pass n-1 (4) 2 3 4 5 7 2 3 4 5 7 compare 2 and 3 ; 2 is < 3 so advance 2 3 4 5 7 compare 3 and 4, 3 <4 so advance 2 3 4 5 7 compare 4 and 5, 4 < 5 so advance 2 3 4 5 7 compare 5 and 7, 5 < 7 so pass n-1 done End of pass 4; notice that everything is where it should be.

10. Exchange Sort Improvements • on each successive pass, do one less compare, because the last item from that pass is in place • if you ever make a pass in which no swap occurs, the sort is complete • These will both improve performance but Big O will remain O(n2)

11. Insertion Sort • Strategy: divide the collection into two lists, one listed with one element (sorted) and the other with the remaining elements. • On successive passes take an item from the unsorted list and insert it into the sorted list so the the sorted list is always sorted • Do this until the unsorted list is empty

12. Insertion Sort sorted unsorted 3 7 5 2 4 take an item from the unsorted list (7) and insert into the sorted list sorted unsorted 3 7 5 2 4 take next item from the unsorted list (5) and insert into the sorted list sorted unsorted 3 5 7 2 4 take next item from the unsorted list (2) and insert into the sorted list sorted unsorted 2 3 5 7 4 take next item from the unsorted list (4) and insert into the sorted list sorted unsorted 2 3 4 5 7

13. Insertion Sort Void InsertionSort ( int A[ ] , int n ) { int i , j; int temp; for ( i = i < n , i++ ) { // scan down list looking for correct place to put new element j = i ; temp = A[ i ]; while ( j < 0 && temp < A[ j-1 ]) { // shift the list 1 element to the right to make room for new element A[ j ] = A[ j+1 ]; j--; } A [ j ] = temp; } }

14. Insertion Sort • Note that each insertion could be O(n-1) and there are n-1 insertions being done therefore Big O is O(n2) • This is very much like building an ordered linked list except there is more data movement

15. Selection Sort • Strategy: make a pass across the data looking for the largest item, swap the largest with the last item in the array. • On successive passes (n-1) assume the array is one smaller (the last item is in the correct place) and repeat previous step

16. Selection Sort int SelectionSort ( int num [ ] , int numelem ) { int i , j , min minidx , temp , moves = 0; for ( i = 0 ; i < numelem ; i++) { min = num [ i ] ; minidx = i ; for ( j = i + 1 ; j < numelem ; j++ ) { min = num[ j ] ; minidx = j ; } if ( min < num[ i ] ) { temp = num[ i ] ; num[ i ] = min ; num[ minidx ]; moves++; } }

17. Selection Sort biggest last 3 7 5 2 4 3 4 5 2 7 biggest last 3 4 5 2 7 3 4 2 5 7 biggest last 3 4 2 5 7 3 2 4 5 7 3 2 4 5 7 2 3 4 5 7

18. Selection Sort • Notice that in selection sort, there is the least possible data movement • There are still n-1 compares on sublists that become one item smaller on each pass so, Big O is still O(n2) • This method has the best overall performance of the O(n2) algorithms because of the limited amount of data movement

19. Heap Sort • A heap is a tree structure in which the key at the root is max(keys) and the key of every parent is greater than the key of either of its children • Visualize your array as a complete binary tree • Arrange the tree into a heap by recursively reapplying the heap definition

20. Heap Sort • repeat the following n-1 times • swap the root with the last element in the tree • starting at the root reinforce the heap definition • decrement the index of the last element

21. A Sample Heap 8 Notice: largest key is at root 5 7 Notice: Keys of children are < key of parent 3 2

22. Heap Sort void HeapSort( int A[ ] , int n ) { // this constructor turns the array into a max heap Heap H(A,n); int elt; for (int i = n-1 ; i >= 1 ; i--) { // delete smallest element from heap and place it in A[ i ] elt = H.Hdelete ( ); // Hdelete deletes the largest element and reenforces the heap property A[ I ] = elt ; } }

23. Visualize array to sort as tree 0 1 2 3 4 0 3 7 5 2 4 3 2 1 7 5 3 4 2 4

24. Make into a heap 0 0 0 3 7 7 1 7 5 1 3 5 1 4 5 3 4 3 4 3 4 2 4 2 4 2 3 STEP 1 STEP 2 STEP 3 Start at root, compare root to the children, swap root with largest child Original Array Repeat on preorder traversal path. At this point we have a heap Note: Larger arrays will take more steps

25. Now the sort starts 0 0 0 7 3 5 2 2 2 3 1 4 5 1 4 5 1 4 3 3 4 3 4 3 4 2 3 2 7 2 7 Swap the root and the last node Notice what is left is no longer a heap Re heapize it by swaping the root with its largets child

26. Sorting... 0 0 2 4 2 2 1 1 4 3 2 3 3 4 3 4 5 7 5 7 Reenforce the heap property on the remaining tree Swap the root and the last node Notice what is left is no longer a heap

27. Still Sorting... 0 0 0 3 3 2 2 2 2 1 2 4 1 1 2 4 3 4 3 4 3 4 3 4 5 7 5 7 5 7 Swap the root with the last element Reenforce the heap property if necessary Swap the root with the last element

28. And what we have left is... 0 2 2 1 3 4 3 4 5 7 0 1 2 3 4 2 3 4 5 7

29. The Merge Principle • Assume there exist two sorted lists (with queue behavior) 7 5 3 1 8 6 4 2 Compare the items at the front of the list and move the smaller to the back of a third list 7 5 3 1 8 6 4 2 Do it again 7 5 3 2 1 8 6 4

30. The Merge Principle And again 7 5 3 2 1 8 6 4 And again, and again, and again…until 8 7 6 5 4 3 2 1 We have a list whose length is the sum of the lengths and contains all of the elements of both lists, and this list is also ordered

31. Merge Sort void MergeSort ( int data[ ] , int n ) { int n1 , n2 ; // size of first subarray and second subarrays respectively if (n > 1 ) { n1 = n / 2 ; n2 = n - n1; MergeSort ( data , n1); // sort from data[0] to data[n1-1] MergeSort ((data + n1) , n2); // sort from data[n1] to end // merge the two sorted halves Merge (data , n1 , n2 ); } }

32. Merge Sort 75 17 5 12 19 24 4 34 11 3 33 14 23 8 27 43 Picture the given list as a collection of n, 1 element sorted lists (i.e. 75 is a sorted 1 element list, as is 17. Now merge the adjacent lists of length 1 into lists of length 2…. 17 75 5 12 19 24 4 43 11 34 3 33 14 23 8 27 ...now merge the lists of length 2 into lists of length 4 5 17 12 75 4 19 24 43 3 11 33 34 8 14 23 27 ...now merge the lists of length 4 into lists of length 8 4 5 12 17 19 24 43 75 3 8 11 14 23 27 33 34 ...now merge the lists of length 8 into lists of length 16 3 4 5 8 11 12 14 17 19 23 23 27 33 34 43 75 …and there you have it merge sort

33. Quick Sort • This sorting method by far outshines all of the others for flat out speed • Big O is log2n • there are problems, worst case performance is when data is already in sorted order or is almost in sorted order (we’ll analyze this separately) • and there are solutions to the problems • and there is an improvement to make it faster still

34. Quick Sort Pick the leftmost element as the pivot (23). Now , start two cursors (one at either end) going towards the middle and swap values that are > pivot (found with left cursor) with values < pivot (found with right cursor) 23 17 5 12 19 24 4 34 11 3 33 14 26 8 27 43 swap 23 17 5 12 19 8 4 34 11 3 33 14 26 24 27 43 swap 23 17 5 12 19 8 4 34 11 3 33 26 24 27 14 43 swap 23 17 5 12 19 8 4 3 11 34 33 26 24 27 14 43 swap Finally, swap the pivot and the value where the cursors passed each other 11 17 5 12 19 8 4 3 23 34 33 26 24 27 14 43 Note : 23 is now in the right place and everything to its left is < 23 and everything to its left is > 23

35. Quick Sort Now, repeat the process for the right partition 11 17 5 12 19 8 4 3 23 34 33 26 24 27 14 43 swap 11 17 5 12 19 8 4 3 14 swap 11 3 5 4 19 8 12 14 17 swap 11 3 5 4 8 19 12 14 17 swap 8 3 5 4 11 19 12 14 17 Note: the 11 is now in the right place, and the left partition is all < pivot and the right partition is all > pivot

36. Quick Sort Now, repeat the process with the right partition 8 3 5 4 11 19 12 14 17 8 3 5 4 Notice that there is nothing to swap , so swap the pivot and the 4, now the 8 is on the right place 4 3 5 8 Repeat the process on the leftmost partition again 4 3 5 The 4 is now in the right place and the left and right partitions are both of size one so they must also be in the right place 3 4 5

37. Quick Sort Now that we’ve exhausted the left partitions, back up and do the right partition 3 4 5 8 11 19 12 14 17 23 17 12 14 19 swap 14 12 17 swap Since the left partition is just two items, just compare and swap if necessary 12 14 3 4 5 8 11 12 14 17 19 23 Now back up and do the remaining right partition

38. Quick Sort 3 4 5 8 11 12 14 17 19 23 34 33 26 24 27 43 swap 34 33 27 26 24 43 swap 24 33 27 26 34 43 swap 24 26 27 33 24 26 27 33 26 27 33 3 4 5 8 11 12 14 17 19 23 24 26 27 33 34 43 All done

39. Quick Sort (worst case) • If the data is already sorted watch what happens to the partitions 3 4 5 8 11 12 14 17 19 23 24 26 27 33 34 43 There is nothing to swap 4 5 8 11 12 14 17 19 23 24 26 27 33 34 43 Again, nothing to swap….. The partitions are always the maximum size and the performance degrades to O(n2)

40. Quick Sort (worst case solution) • The problem comes from the way we picked the pivot • Pick the pivot a different way • median-of-three • instead of always picking the leftmost value of the partition use the median of the first, the middle and the last value in the partition.

41. Quick Sort Improvement • Since Quick sort is a recursive algorithm, a lot of time gets eaten up in the recursion involved with sorting the small partitions • Solution - use a different algorithm for the small partitions • remember for small collections of data the n2 algorithms perform better than the log2n algorithms • this is one place where exchange sort excels • if the partition size is 15 or less use exchange (or selection sort)

42. Radix Sort