1 / 69

Efficient Sorting Algorithms Overview

Learn about various sorting algorithms, their performance, and when to use them for different data sets. Explore Big-O notation and essential algorithms like selection, bubble, insertion, and more.

hkellogg
Download Presentation

Efficient Sorting Algorithms Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cairo University Faculty of Computers and Information Data Structures CS 214 2nd Term 2012-2013 Sorting I http://en.wikipedia.org/wiki/Sorting_algorithm

  2. CS214 – Data StructuresLecture 04&5: Quadratic Sorting Slides by Mohamed El-Ramly, PhD

  3. Agenda 0. Why Sorting? Quadratic Sorting algorithms Selection Sort Insertion Sort Bubble Sort Included but read yourself Theoretical Boundaries of Sorting Problem STL support for sorting Sub-quadratic Sorting Merge Sort Shell Sort Quick Sort Heap Sort

  4. Readings Compulsory Weiss, Chap 7 Optional Adam Drozdek, Chap 2 up to 2.8 Extra http://www.sorting-algorithms.com/ http://math.hws.edu/eck/jsdemo/sortlab.html

  5. Chapter Objectives • To learn how to use the standard sorting algorithms in STL • To learn how to implement the following sorting algorithms: selection sort, bubble sort, insertion sort, shell sort, merge sort, heap sort and quick. • To understand the difference in performance of these algorithms, and which to use for small, medium and large sets of data and for what types of data. Chapter 9: Sorting

  6. بينما رجل يسافر فى الصحراء إذا بأسد يطلع عليه يريد أن يهاجمه ، فأسرع الرجل يفر من الأسد و الأسد خلفه ، فاشتد الرجل فى الهرب بكل ما أوتى من قوة و الأسد فى أثره لا يتركه و لكنه أبصر بئراً فأسرع إليها و أمسك بالحبل الذي يتدلى داخل البئر لإحضار الماء و قفز فيالبئر فإذا هي عميقة جداً ، فتعلق الرجل بالحبل بقوة و هو يخشى السقوط و نظر الرجل أسفل البئر فإذا هو مظلم شديد السواد و إذا بحية ضخمة في قعر البئر تنظر إليه و هي تتحفز و نظر الرجلإلى أعلى فإذا بالأسد ينتظره و هو يتلمظ ، ثم فوجئ الرجل بفأرين أحدهما أبيض و الآخر أسود يخرجان من شق فى جدار البئر و يقفزان على الحبل و يشرعان فى قرضه فأصابه هم شديد و إضطراب و أخذ الحبل يتأرجح به و يصدمه بجانب البئر يمنة و يسرة ، و بينما هو كذلك إذ إصطدم بخلية نحل بجانب البئر و إذا بعسل يسيل منها فمد إصبعه و أخذ غمسة من العسل فإذا هو لذيذ جدا فأخذ يمديده للعسل و يغمس أصابعه فيه و يأكل و إنهمك في الأكل حتى نسى الأسد الذي يطلبه و الحية التي تنتظره و الفأرين الذين يقرضان الحبل ونسى كل شئ" Chapter 9: Sorting

  7. Why Sorting? • The efficiency of data handling can be substantially increased if the data is sorted. • Imagine an unsorted phone directory or a dictionary! • Often, it is required to sort data before processing. • First, we need to decide on the sorting criterion • Second we need to decide on the sorting algorithm Chapter 9: Sorting

  8. Which Criterion? • First, we need to decide how items are compared. • Often, they have some natural order: 1,2,3, etc or a, b, c, d, etc. • In other cases, it is not directly clear how to order data. How would you order: • Cars • Cats • Books Chapter 9: Sorting

  9. Which Sorting Algorithms? • To compare algorithm efficiency, we need a machine-independent criterion for evaluating their performance. • Two factors determine the algorithm performance: • The number of comparisons • The number of movements • We use Big-O notation for this purpose. Chapter 9: Sorting

  10. What Affects Performance? • Initial Order of the Data • Does the algorithm recognize already ordered data? • How much time is spent in on ordering alread-ordered data? • Algorithm’s intelligence • Best / Worst / Average cases • Comparisons / Movements: simple or complex? • Do we compare int values or arrays? • Do we move int values or structures? Chapter 9: Sorting

  11. Chapter 9: Sorting

  12. Priority queue sorting Selection sort Heap sort The family of sorting methods Main sorting themes Address- -based sorting Comparison-based sorting RadixSort Proxmap Sort Transposition sorting BubbleSort Divide and conquer Diminishing increment sorting Insert and keep sorted MergeSort QuickSort Insertion sort Tree sort ShellSort

  13. Insertion Sort • Based on the technique used by card players to arrange a hand of cards • Player keeps the cards that have been picked up so far in sorted order • When the player picks up a new card, he makes room for the new card and then inserts it in its proper place Chapter 9: Sorting

  14. Insertion Sort Algorithm • For each array element from the second to the last (nextPos = 1) • Insert the element at nextPos where it belongs in the array, increasing the length of the sorted subarray by 1 Chapter 9: Sorting

  15. next to be inserted sorted 3 4 7 12 14 33 14 20 21 38 10 55 9 23 28 16 temp less than 10 10 12 21 14 14 20 33 38 sorted 3 4 7 10 55 9 23 28 16 10 12 14 33 14 20 21 38 One step of insertion sort Chapter 9: Sorting

  16. Psuedocode for Insertion Sort • insertionSort (data [], n) for (i = 1; i < n; i++) move all elements data[j]>data [i]by1position; place data [i]in its proper position; Chapter 9: Sorting

  17. Java Code for Insertion Sort • public static void insertionSort(int[] array) { int inner, outer; for (outer = 1; outer < array.length; outer++) { int temp = array[outer]; inner = outer; while (inner > 0 && array[inner - 1] >= temp) { array[inner] = array[inner - 1]; inner--; } array[inner] = temp;// Invariant:For all i < outer, // j < outer, if i < j then a[i] <= a[j] }} Chapter 9: Sorting

  18. C++ Code for Insertion Sort • template <class T> • void insertionSort (T data[], int n) { for (int i = 1, j; i < n; i++) T tmp = data [i]; for (j = i; j > 0 && tmp < data [j-1]; j--) data [j] = data [j – 1]; data [j] = tmp; } Chapter 9: Sorting

  19. Analysis of insertion sort • Insertion sort sorts array when really needed. • If the array is already sorted, moves are very few. • On the other hand, shifting can be time consuming. Chapter 9: Sorting

  20. Best Case • Outer loop n – 1 • T tmp = data [i]; 1 move x n – 1 times • data [j] = tmp; 1 move x n – 1 times • tmp < data [j-1] 1 comparison x n – 1 • Moves: 2 (n - 1) • Comparisons: n – 1 • O (n) Chapter 9: Sorting

  21. Worst Case • Happens when the data is in reverse order • Outer loop n – 1 • T tmp = data [i]; 1 move x n – 1 times • data [j] = tmp; 1 move x n – 1 times • Inner loop executes, 1, 2, …, n – 1 • For each inner loop, there is 1 comparison • For each inner loop, there is 1 move Chapter 9: Sorting

  22. Worst Case • Comparisons: (1 + 2 + …. + (n - 1)) = n (n – 1) / 2  O(n2) • Moves: (1 + 2 + …. + (n - 1)) + 2 (n – 1) = n (n – 1) / 2 + 2 (n – 1) = n2/2 + 3n/2 – 2  O(n2) Chapter 9: Sorting

  23. Average Case • This is an approximate analysis simpler than the book • Assume the data is randomly ordered • Outer loop n – 1 • T tmp = data [i]; 1 move x n – 1 times • data [j] = tmp; 1 move x n – 1 times • Inner loop executes, 1, 2, …, n – 1 • For each inner loop, there is on average i/2 comparison • For each inner loop, there is on average i/2 moves Chapter 9: Sorting

  24. Average Case • Comparisons: (1 + 2 + …. + (n - 1))/2 = n (n – 1) / 4  O(n2) • Moves: (1 + 2 + …. + (n - 1))/2 + 2 (n – 1) = n (n – 1) / 4 + 2 (n – 1) = n2/4 + 7n/4 – 2  O(n2) Chapter 9: Sorting

  25. Analysis of Insertion Sort • Maximum number of comparisons is O(n*n) • In the best case, number of comparisons is O(n) • The number of shifts performed during an insertion is one less than the number of comparisons or, when the new value is the smallest so far, the same as the number of comparisons • A shift in an insertion sort requires the movement of only one item whereas in a bubble or selection sort an exchange involves a temporary item and requires the movement of three items Chapter 9: Sorting

  26. Chapter 9: Sorting

  27. Selection Sort • Selection sort is a relatively easy to understand algorithm • Sorts an array by making several passes through the array, selecting the next smallest item in the array each time and placing it where it belongs in the array • It attempts to localize exchanges by putting the item directly in its final place. • Efficiency is O(n*n) Chapter 9: Sorting

  28. SelectionSort • Given an array of length n, • Search elements0 through n-1 and select the smallest • Swap it with the element in location 0 • Search elements 1 through n-1 and select the smallest • Swap it with the element in location 1 • Search elements 2 through n-1 and select the smallest • Swap it with the element in location 2 • Search elements 3 through n-1 and select the smallest • Swap it with the element in location 3 • Continue in this fashion until there’s nothing left to search Chapter 9: Sorting

  29. 2 2 7 2 2 4 2 4 7 4 5 8 5 8 8 7 5 5 8 5 7 8 4 4 7 Example and analysis of Selection Sort • The Selection Sort might swap an array element with itself--this is harmless, and not worth checking for • Analysis: • The outer loop executes n-1 times • The inner loop executes about n/2 times on average (from n to 2 times) • Work done in the inner loop is constant (swap two array elements) • Time required is roughly (n-1)*(n/2) • You should recognize this asO(n2) Chapter 9: Sorting

  30. Psuedocode for Selection Sort • selectionSort (data [], n) for (i = 1; i < n-1; i++) select smallest element from data[i],…,data [n-1]; swap it with data [i]; Chapter 9: Sorting

  31. Java Code for Selection Sort public static void selectionSort(int[] a) { int outer, inner, min; for (outer = 0; outer < a.length - 1; outer++) { // outer counts down min = outer; for (inner = outer + 1; inner < a.length; inner++) { if (a[inner] < a[min]) { min = inner; } } // a[min] is least among a[outer]..a[a.length - 1] int temp = a[outer]; a[outer] = a[min]; a[min] = temp; }} Chapter 9: Sorting

  32. C++ Code of Selection Sort • template <class T> • void selectionSort(T data[], int n) { for (int i = 0, j, least; i < n-1; i++) { for (j = i+1, least = i; j < n; j++) if (data [j] < data [least]) least = j; swap (data [least], data [i]); } } Chapter 9: Sorting

  33. Analysis of Selection Sort • template <class T> • void selectionSort(T data[], int n) { for (int i = 0, j, least; i < n-1; i++) { for (j = i+1, least = i; j < n; j++) if (data [j] < data [least]) least = j; swap (data [least], data [i]); } } n - 1 n-1, n-2,…,1 1 comparison 1 swap Chapter 9: Sorting

  34. Best / Worst / Average Case • It does not recognize order of data • Comparisons: 1 + 2 + . . . + (n - 1) = n (n - 1) / 2  O (n2) • Swaps: n – 1  O (n) • Moves: 3 (n – 1)  O (n) Chapter 9: Sorting

  35. Analysis of Selection Sort • What conclusion can we make about Selection Sort? • How intelligent is Selection Sort? • What do you notice about the number of swaps? • Modify the algorithm to further reduce the number of swaps. Chapter 9: Sorting

  36. Chapter 9: Sorting

  37. Bubble Sort • Compares adjacent array elements and exchanges their values if they are out of order • Smaller values bubble up to the top of the array and larger values sink to the bottom Chapter 9: Sorting

  38. 2 2 2 7 2 2 2 2 2 2 2 2 2 7 4 2 5 4 7 4 7 7 5 5 7 7 5 8 8 8 5 5 5 5 4 7 5 4 5 4 4 4 7 7 8 7 5 5 4 7 5 7 4 8 4 4 4 8 8 8 8 8 8 8 8 2 5 4 7 8 Example of Bubble Sort (done) دعا إعرابى الله أن ينزل الغيث فقال: ”اللهم ربنا وإلهنا ومولانا اسقنا غيثا مريعا مغيثا مجلجلا مستنفرا سحا سفوحا غدقا مثعنجرا“ ، فقال رجل سمعه : يا خليفة نوح هذا الطوفان ورب الكعبة أمهلني حتى أوي إلي جبل يعصمني من الماء Chapter 9: Sorting

  39. Psuedocode for Bottom-up Bubble Sort • bubbleSort (data [], n) for (i = 1; i < n-1; i++) for (j = n - 1; j > i; --j) swap items j and j – 1 if out of order; Chapter 9: Sorting

  40. Java Code for Bubble Sort • public static void bubbleSort(int[] a) { int outer, inner; for (outer = a.length - 1; outer > 0; outer--) {// counting down for (inner = 0; inner < outer; inner++) {// bubbling up if (a[inner] > a[inner + 1]) {// if out of order... int temp = a[inner]; // ...then swapa[inner] = a[inner + 1]; a[inner + 1] = temp; } } }} Chapter 9: Sorting

  41. C++ Code of Bottom-up Bubble Sort • template <class T> • void bubbleSort(T data[], int n) { for (int i = 0; i < n - 1; i++) { for (int j = n - 1; j > i; --j) if (data [j] < data [j-1]) swap (data [j], data [j - 1]); } Chapter 9: Sorting

  42. Analysis of Bubble Sort • template <class T> • void bubbleSort(T data[], int n) { for (int i = 0; i < n - 1; i++) { for (int j = n - 1; j > i; --j) if (data [j] < data [j-1]) swap (data [j], data [j - 1]); } n - 1 n-1,…,1 1 comparison 1 swap Chapter 9: Sorting

  43. Worst Case • Reverse ordered data • Comparisons: 1 + 2 + . . . + (n - 1) = n (n - 1) / 2  O (n2) • Swaps: 1 + 2 + . . . + (n - 1) = n (n - 1) / 2  O (n2) • Moves: 3 n (n – 1) / 2  O (n2) Chapter 9: Sorting

  44. Best Case • When array is ordered • Comparisons: 1 + 2 + . . . + (n - 1) = n (n - 1) / 2  O (n2)  Can it be improved to O(n) • Swaps: none  O (1) • Moves: none  O (1) Chapter 9: Sorting

  45. Average Case • Comparisons: 1 + 2 + . . . + (n - 1) = n (n - 1) / 2  O (n2) • Swaps: 1/2 + 2/2 + . . . + (n - 1)/2 = n (n - 1) / 4  O (n2) • Moves: 3 n (n – 1) / 4  O (n2) Chapter 9: Sorting

  46. Analysis of Bubble Sort • Provides excellent performance in some cases and very poor performances in other cases • Main problem: it moves data up, one step at a time • Works best when array is nearly sorted to begin with • Worst case number of comparisons is O(n2) • Worst case number of exchanges is O(n2) • Best case occurs when the array is already sorted • O(n) comparisons • O(1) exchanges Chapter 9: Sorting

  47. Chapter 9: Sorting

  48. Comparison of Quadratic Sorts • None of the algorithms are particularly good for large arrays Chapter 9: Sorting

  49. Comparison of Quadratic Sorts • Bubble Sort, Selection Sort, and Insertion Sort are all O(n2) • As we will see later, we can do much better than this with somewhat more complicated sorting algorithms • WithinO(n2), • Bubble Sort is very slow, and should probably never be used for anything • Selection Sort is intermediate in speed • Insertion Sort is usually the fastest of the three--in fact, for small arrays (say, 10 or 15 elements), insertion sort is faster than more complicated sorting algorithms • Selection Sort and Insertion Sort are “good enough” for small arrays Chapter 9: Sorting

  50. Theoretical Analysis of Sorting Problem • Quadratic algorithms are not efficient ? • Can we obtain better efficiency ? • We need a lower bound or best performance. • We focus on comparisons not moves because in ideal cases, you can avoid any moves (e.g., bubble sort) • What is the best estimate of the number of item comparisons if the array is randomly ordered? Chapter 9: Sorting

More Related