Sorting

1. Sorting

2. Objectives • Become familiar with the following sorting methods: • Insertion Sort • Shell Sort • Selection Sort • Bubble Sort • Quick Sort • Heap Sort • Merge Sort • … more

3. Introduction • One of the most common applications in computer science is sorting, the process through which data are arranged according to their values. • If data were not ordered in some way, we would spend an incredible amount of time trying to find the correct information.

4. Introduction • To appreciate this, imagine trying to find someone’s number in the telephone book if the names were not sorted in some way!

5. General Sorting Concepts • Sorts are generally classified as either internal or external. • An internal sort is a sort in which all of the data is held in primary memory during the sorting process. • An external sort uses primary memory for the data currently being sorted and secondary storage for any data that will not fit in primary memory.

6. General Sorting Concepts • For example, a file of 20,000 records may be sorted using an array that holds only 1000 records. • Therefore only 1000 records are in primary memory at any given time. • The other 19,000 records are stored in secondary storage.

7. Sort Order • Data may be sorted in either ascending or descending order. • The sort order identifies the sequence of sorted data, ascending or descending. • If the order of the sort is not specified, it is assumed to be ascending.

8. Sort Stability • Sort stability is an attribute of a sort indicating that data elements with equal keys maintain their relative input order in the output. • Consider the following example.

9. Sort Stability • Note the unsorted data in (a). • If we use a stable sort, items with equal keys are guaranteed to be sorted in the same order as they appear in the unsorted data.

10. Sort Stability • If, however, we use an unstable sort, data with equal keys can be sorted in any order (i.e., not necessarily in the same order as they appeared in the unsorted data).

11. Sort Efficiency • Sort efficiency is a measure of the relative efficiency of a sort. • It is usually an estimate of the number of comparisons and moves required to order an unordered list.

12. Passes • During the sorting process, the data is traversed many times. • Each traversal of the data is referred to as a sort pass. • Depending on the algorithm, the sort pass may traverse the whole list or just a section of the list. • The sort pass may also include the placement of one or more elements into the sorted list.

13. Types of Sorts • We now discuss several sorting methods: • Insertion Sort • Shell Sort • Selection Sort • Bubble Sort • Quick Sort • Heap Sort • Merge Sort

14. Insertion Sort • In each pass of an insertion sort, one or more pieces of data are inserted into their correct location in an ordered list (just as a card player picks up cards and places them in his hand in order).

15. Insertion Sort • In the insertion sort, the list is divided into 2 parts: • Sorted • Unsorted • In each pass, the first element of the unsorted sublist is transferred to the sorted sublist by inserting it at the appropriate place. • If we have a list of n elements, it will take, at most, n-1, passes to sort the data.

16. Insertion Sort • We can visualize this type of sort with the above figure. • The first part of the list is the sorted portion which is separated by a “conceptual” wall from the unsorted portion of the list.

17. Insertion Sort Here we start with an unsorted list. We leave the first data element alone and will start with the 2nd element of the list (the 1st element of the unsorted list). On our first pass, we look at the 2nd element of the list (thefirst element of the unsorted list) and place it in our sorted list in order. We continue in this fashion until the entire list has beensorted. On our next pass, we look at the 3rd element of the list (the1st element of the unsorted list) and place it in the sorted list in order.

18. Insertion Sort Example In our first pass, we compare the first 2 values. Because theyare out of order, we swap them. Now we look at the next 2 values. Again, they are out oforder so we swap them. Since we have swapped those values, we need to comparethe previous 2 values to make sure that they are still in order. Since they are out of order, we swap them and then continueon with the next 2 data values. These 2 values are out of order, so we swap them and look atthe previous 2 values, etc.

19. Shell Sort • Named after its creator, Donald Shell, the shell sort is an improved version of the insertion sort. • In the shell sort, a list of N elements is divided into K segments where K is known as the increment. • What this means is that instead of comparing adjacent values, we will compare values that are a distance K apart. • We will shrink K as we run through our algorithm.

20. Shell Sort Just as in the straight insertion sort, we compare 2 values andswap them if they are out of order. However, in the shell sort we compare values that are a distance K apart. Once we have completed going through the elements in our list with K=5, we decrease K and continue the process.

21. Shell Sort Here we have reduced K to 2. Just as in the insertion sort, if we swap 2 values, we have to go back and compare theprevious 2 values to make sure they are still in order.

22. Shell Sort All shell sorts will terminate by running an insertion sort(i.e., K=1). However, using the larger values of K first has helped to sort our list so that the straight insertion sort will runfaster.

23. Shell Sort • There are many schools of thought on what the increment should be in the shell sort. • Also note that just because an increment is optimal on one list, it might not be optimal for another list.

24. Insertion Sort vs. Shell Sort • Comparing the Big-O notation (for the average case) we find that: • Insertion: O(n2) • Shell: O(n1.25) //empirically determined • Although this doesn’t seem like much of a gain, it makes a big difference as n gets large. • Note that in the worst case, the Shell sort has an efficiency of O(n2) . • However, using a special incrementing technique, this worst case can be reduced to O(n1.5)

25. Shell Sort h := 1; repeat h := 3*h+1 until h>n; repeat h := h div 3; for i := h+1 to n do begin v := a[i]; j:= i; while j>h & a[j-h]>v do begin a[j] := a[j-h]; j := j - h; end; a[j] := v; end; until h = 1; • Shellsort is a simple extension of insertion sort, which gains speeds by allowing exchange of elements that are far apart. • Idea: rearrange list into h-sorted (for any sequence of values of h that ends in 1.) • Shellsort never does more than n1.5 comparisons (for the h = 1, 4, 13, 40, ...). • The analysis of this algorithm is hard. Two conjectures of the complexity are n(log n)2 and n1.25

26. Insertion Sort vs. Shell Sort(Average Case)

27. Selection Sort • Imagine some data that you can examine all at once. • To sort it, you could select the smallest element and put it in its place, select the next smallest and put it in its place, etc. • For a card player, this process is analogous to looking at an entire hand of cards and ordering them by selecting cards one at a time and placing them in their proper order.

28. Selection Sort • The selection sort follows this idea. • Given a list of data to be sorted, we simply select the smallest item and place it in a sorted list. • We then repeat these steps until the list is sorted.

29. Selection Sort • In the selection sort, the list at any moment is divided into 2 sublists, sorted and unsorted, separated by a “conceptual” wall. • We select the smallest element from the unsorted sublist and exchange it with the element at the beginning of the unsorted data. • After each selection and exchange, the wall between the 2 sublists moves – increasing the number of sorted elements and decreasing the number of unsorted elements.

30. Selection Sort We start with an unsorted list. We search this list for thesmallest element. We then exchange the smallest element (8)with the first element in the unsorted list (23). Again, we search the unsorted list for the smallest element. We then exchange the smallest element (23) with the first element in the unsorted list (78). This process continues until the list is fully sorted.

31. Bubble Sort • In the bubble sort, the list at any moment is divided into 2 sublists, sorted and unsorted. • The smallest element is “bubbled” from the unsorted sublist to the sorted sublist.

32. 1 Pass of the Bubble Sort 23 78 45 8 56 32 Bubble Sort We step down one element andcompare 45 and 8. They areout of sequence, so we swap them and step down again. 8 8 23 8 78 45 32 56 We start with 32 and compareit with 56. Because 32 is lessthan 56, we swap the twoand step down one element. We step down again and compare 8 with 78. These two elements are swapped. We then compare 32 and 8.Because 32 is not less than 8,we do not swap these elements. Finally, 8 is compared with 23and swapped. We thencontinue this process back with56 …

33. Quick Sort • In the bubble sort, consecutive items are compared and possibly exchanged on each pass through the list. • This means that many exchanges may be needed to move an element to its correct position. • Quick sort is more efficient than bubble sort because a typical exchange involves elements that are far apart, so fewer exchanges are required to correctly position an element.

34. Idea of Quick Sort 1) Select: pick an element 2) Divide: rearrange elements so that x goes to its final position E 3) Recurse and Conquer: recursively sort

35. Quick Sort • Each iteration of the quick sort selects an element, known as the pivot, and divides the list into 3 groups: • Elements whose keys are less than (or equal to) the pivot’s key. • The pivot element • Elements whose keys are greater than (or equal to) the pivot’s key.

36. Quick Sort • The sorting then continues by quick sorting the left partition followed by quick sorting the right partition. • The basic algorithm is as follows:

37. Quick Sort • Partitioning Step: Take an element in the unsorted array and determine its final location in the sorted array. This occurs when all values to the left of the element in the array are less than (or equal to) the element, and all values to the right of the element are greater than (or equal to) the element. We now have 1 element in its proper location and two unsorted subarrays. • Recursive Step: Perform step 1 on each unsorted subarray.

38. Quick Sort • Each time step 1 is performed on a subarray, another element is placed in its final location of the sorted array, and two unsorted subarrays are created. • When a subarray consists of one element, that subarray is sorted. • Therefore that element is in its final location.

39. Quick Sort • There are several partitioning strategies used in practice (i.e., several “versions” of quick sort), but the one we are about to describe is known to work well. • For simplicity we will choose the last element to be the pivot element. • We could also chose a different pivot element and swap it with the last element in the array.

40. Quick Sort • Below is the array we would like to sort:

41. Quick Sort • The index left starts at the first element and right starts at the next-to-last element. • We want to move all the elements smaller than the pivot to the left part of the array and all the elements larger than the pivot to the right part.

42. Quick Sort • We move left to the right, skipping over elements that are smaller than the pivot.

43. Quick Sort • We then move right to the left, skipping over elements that are greater than the pivot. • When left and right have stopped, left is on an element greater than (or equal to) the pivot and right is on an element smaller than (or equal to) the pivot.

44. Quick Sort • If left is to the left of right (or if left = right), those elements are swapped.

45. Quick Sort • The effect is to push a large element to the right and a small element to the left. • We then repeat the process until left and right cross.

46. Quick Sort

47. Quick Sort

48. Quick Sort • At this point, left and right have crossed so no swap is performed. • The final part of the partitioning is to swap the pivot element with left.

49. Quick Sort • Note that all elements to the left of the pivot are less than (or equal to) the pivot and all elements to the right of the pivot are greater than (or equal to) the pivot. • Hence, the pivot element has been placed in its final sorted position.

50. Quick Sort • We now repeat the process using the sub-arrays to the left and right of the pivot.