1 / 75

Design & Analysis of Algorithms CSc 4520/6520

Design & Analysis of Algorithms CSc 4520/6520. Sorting and Asymptotic Notations Fall 2013 -- GSU Anu Bourgeois*. Sorting. Insertion sort Design approach: Sorts in place: Best case: Worst case: Bubble Sort Design approach: Sorts in place: Running time:. incremental. Yes.  (n).

duer
Download Presentation

Design & Analysis of Algorithms CSc 4520/6520

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design & Analysis of AlgorithmsCSc 4520/6520 Sorting and Asymptotic Notations Fall 2013 -- GSU Anu Bourgeois*

  2. Sorting • Insertion sort • Design approach: • Sorts in place: • Best case: • Worst case: • Bubble Sort • Design approach: • Sorts in place: • Running time: incremental Yes (n) (n2) incremental Yes (n2)

  3. Insertion Sort • while some elements unsorted: • Using linear search, find the location in the sorted portion where the 1st element of the unsorted portion should be inserted • Move all the elements after the insertion location up one position to make space for the new element 45 38 60 45 60 66 45 66 79 47 13 74 36 21 94 22 57 16 29 81 the fourth iteration of this loop is shown here

  4. 1 2 3 4 5 6 77 5 101 42 35 12 1 2 3 4 5 6 5 77 101 35 12 42 1 2 3 4 5 6 42 77 101 12 35 5 N - 1 1 2 3 4 5 6 42 77 101 12 5 35 1 2 3 4 5 6 42 77 101 5 12 35 “Bubbling” All the Elements

  5. Sorting • Selection sort • Design approach: • Sorts in place: • Running time: • Merge Sort • Design approach: • Sorts in place: • Running time: incremental Yes (n2) divide and conquer No Let’s see!!

  6. Selection Sort Selection Sort • Start with the 1st element, scan the entire list to find its smallest element and exchange it with the 1st element • Start with the 2nd element, scan the remaining list to find the the smallest among the last (N-1) elements and exchange it with the 2nd element • … Example: 89 45 68 90 29 34 17 17 | 45 68 90 29 34 89 29 | 68 90 45 34 89 34 | 90 45 68 89 45 | 90 68 89 68 | 90 89 89 | 90 90

  7. Divide-and-Conquer • Divide the problem into a number of sub-problems • Similar sub-problems of smaller size • Conquer the sub-problems • Solve the sub-problems recursively • Sub-problem size small enough  solve the problems in straightforward manner • Combine the solutions of the sub-problems • Obtain the solution for the original problem

  8. Merge Sort Approach • To sort an array A[p . . r]: • Divide • Divide the n-element sequence to be sorted into two subsequences of n/2 elements each • Conquer • Sort the subsequences recursively using merge sort • When the size of the sequences is 1 there is nothing more to do • Combine • Merge the two sorted subsequences

  9. Merge Sort r p q Alg.: MERGE-SORT(A, p, r) if p < rCheck for base case then q ← (p + r)/2Divide MERGE-SORT(A, p, q)Conquer MERGE-SORT(A, q + 1, r) Conquer MERGE(A, p, q, r)Combine • Initial call:MERGE-SORT(A, 1, n) 1 2 3 4 5 6 7 8 5 2 4 7 1 3 2 6

  10. 1 2 3 4 5 6 7 8 q = 4 5 2 4 7 1 3 2 6 1 2 3 4 5 6 7 8 5 2 4 7 1 3 2 6 1 2 3 4 5 6 7 8 5 2 4 7 1 3 2 6 5 1 2 3 4 6 7 8 5 2 4 7 1 3 2 6 Example – n Power of 2 Divide

  11. 1 2 3 4 5 6 7 8 1 2 2 3 4 5 6 7 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 1 2 3 4 5 6 7 8 2 5 4 7 1 3 2 6 5 1 2 3 4 6 7 8 5 2 4 7 1 3 2 6 Example – n Power of 2 Conquer and Merge

  12. 1 2 3 4 5 6 7 8 9 10 11 q = 6 4 7 2 6 1 4 7 3 5 2 6 1 2 3 4 5 6 7 8 9 10 11 q = 3 4 7 2 6 1 4 7 3 5 2 6 q = 9 1 2 3 4 5 6 7 8 9 10 11 4 7 2 6 1 4 7 3 5 2 6 1 2 3 4 5 6 7 8 9 10 11 4 7 2 6 1 4 7 3 5 2 6 1 2 4 5 7 8 4 7 6 1 7 3 Example – n Not a Power of 2 Divide

  13. 1 2 3 4 5 6 7 8 9 10 11 1 2 2 3 4 4 5 6 6 7 7 1 2 3 4 5 6 7 8 9 10 11 1 2 4 4 6 7 2 3 5 6 7 1 2 3 4 5 6 7 8 9 10 11 2 4 7 1 4 6 3 5 7 2 6 1 2 3 4 5 6 7 8 9 10 11 4 7 2 1 6 4 3 7 5 2 6 1 2 4 5 7 8 4 7 6 1 7 3 Example – n Not a Power of 2 Conquer and Merge

  14. r p q 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 Merging • Input: Array Aand indices p, q, rsuch that p ≤ q < r • Subarrays A[p . . q] and A[q + 1 . . r] are sorted • Output: One single sorted subarray A[p . . r]

  15. r p q 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 Merging • Idea for merging: • Two piles of sorted cards • Choose the smaller of the two top cards • Remove it and place it in the output pile • Repeat the process until one pile is empty • Take the remaining input pile and place it face-down onto the output pile A1 A[p, q] A[p, r] A2 A[q+1, r]

  16. p q r Example: MERGE(A, 9, 12, 16)

  17. Example: MERGE(A, 9, 12, 16)

  18. Example (cont.)

  19. Example (cont.)

  20. Example (cont.) Done!

  21. p q 2 4 5 7  L q + 1 r r p q 1 2 3 6  R 1 2 3 4 5 6 7 8 2 4 5 7 1 2 3 6 n2 n1 Merge - Pseudocode Alg.:MERGE(A, p, q, r) • Compute n1and n2 • Copy the first n1 elements into L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1] • L[n1 + 1] ← ;R[n2 + 1] ←  • i ← 1; j ← 1 • for k ← pto r • do if L[ i ] ≤ R[ j ] • then A[k] ← L[ i ] • i ←i + 1 • else A[k] ← R[ j ] • j ← j + 1

  22. Running Time of Merge(assume last for loop) • Initialization (copying into temporary arrays): • (n1 + n2) = (n) • Adding the elements to the final array: - n iterations, each taking constant time  (n) • Total time for Merge: • (n)

  23. Analyzing Divide-and Conquer Algorithms • The recurrence is based on the three steps of the paradigm: • T(n) – running time on a problem of size n • Divide the problem into a subproblems, each of size n/b: takes D(n) • Conquer (solve) the subproblems aT(n/b) • Combine the solutions C(n) (1) if n ≤ c T(n) = aT(n/b) + D(n) + C(n) otherwise

  24. MERGE-SORT Running Time • Divide: • compute qas the average of pand r:D(n) = (1) • Conquer: • recursively solve 2 subproblems, each of size n/2  2T (n/2) • Combine: • MERGE on an n-element subarray takes (n) time  C(n) = (n) (1) if n =1 T(n) = 2T(n/2) + (n) if n > 1

  25. Solve the Recurrence T(n) = c if n = 1 2T(n/2) + cn if n > 1 Use Master’s Theorem: Compare n with f(n) = cn Case 2: T(n) = Θ(nlgn)

  26. Asymptotic Notations • BIG O: O • f = O(g) if f is no faster then g • f / g < some constant • BIG OMEGA:  • f = (g) if f is no slower then g • f / g > some constant • BIG Theta:  • f = (g) if f has the same growth rate as g • some constant < f / g < some constant

  27. Analysis of Algorithms • An algorithm is a finite set of precise instructions for performing a computation or for solving a problem. • What is the goal of analysis of algorithms? • To compare algorithms mainly in terms of running time but also in terms of other factors (e.g., memory requirements,programmer's effort etc.) • What do we mean by running time analysis? • Determine how running time increases as the size of the problem increases.

  28. Input Size • Input size (number of elements in the input) • size of an array • polynomial degree • # of elements in a matrix • # of bits in the binary representation of the input • vertices and edges in a graph

  29. Types of Analysis • Worst case • Provides an upper bound on running time • An absolute guarantee that the algorithm would not run longer, no matter what the inputs are • Best case • Provides a lower bound on running time • Input is the one for which the algorithm runs the fastest • Average case • Provides a prediction about the running time • Assumes that the input is random

  30. How do we compare algorithms? • We need to define a number of objective measures. (1) Compare execution times? Not good: times are specific to a particular computer !! (2) Count the number of statements executed? Not good: number of statements vary with the programming language as well as the style of the individual programmer.

  31. Ideal Solution • Express running time as a function of the input size n (i.e., f(n)). • Compare different functions corresponding to running times. • Such an analysis is independent of machine time, programming style, etc.

  32. Asymptotic Analysis • To compare two algorithms with running times f(n) and g(n), we need a rough measure that characterizes how fast each function grows. • Hint: use rate of growth • Compare functions in the limit, that is, asymptotically! (i.e., for large values of n)

  33. Rate of Growth • Consider the example of buying elephants and goldfish: Cost: cost_of_elephants + cost_of_goldfish Cost ~ cost_of_elephants (approximation) • The low order terms in a function are relatively insignificant for largen n4 + 100n2 + 10n + 50 ~ n4 i.e., we say thatn4 + 100n2 + 10n + 50 and n4 have the same rate of growth

  34. Asymptotic Notation • O notation: asymptotic “less than”: • f(n)=O(g(n)) implies: f(n) “≤” g(n) •  notation: asymptotic “greater than”: • f(n)=  (g(n)) implies: f(n) “≥” g(n) •  notation: asymptotic “equality”: • f(n)=  (g(n)) implies: f(n) “=” g(n)

  35. Big-O Notation • We say fA(n)=30n+8is order n, or O (n) It is, at most, roughly proportional to n. • fB(n)=n2+1 is order n2, or O(n2). It is, at most, roughly proportional to n2. • In general, any O(n2) function is faster- growing than any O(n) function.

  36. Big-O Visualization O(g(n)) is the set of functions with smaller or same order of growth as g(n)

  37. Asymptotic notations • O-notation

  38. cn =31n n>n0=8  Big-O example, graphically • Note 30n+8 isn’tless than nanywhere (n>0). • It isn’t evenless than 31neverywhere. • But it is less than31neverywhere tothe right of n=8. 30n+8 30n+8O(n) Value of function  n Increasing n 

  39. No Uniqueness • There is no unique set of values for n0 and c in proving the asymptotic bounds • Prove that 100n + 5 = O(n2) • 100n + 5 ≤ 100n + n = 101n ≤ 101n2 for all n ≥ 5 n0 = 5 and c = 101is a solution • 100n + 5 ≤ 100n + 5n = 105n ≤ 105n2for all n ≥ 1 n0 = 1 and c = 105is also a solution Must findSOMEconstants c and n0 that satisfy the asymptotic notation relation

  40. Asymptotic notations (cont.) •  - notation (g(n)) is the set of functions with larger or same order of growth as g(n)

  41. Examples • 5n2 = (n) • 100n + 5 ≠(n2) • n = (2n), n3 = (n2), n = (logn)  cn  5n2  c = 1 and n0 = 1 •  c, n0such that: 0  cn  5n2  c, n0 such that: 0  cn2  100n + 5 100n + 5  100n + 5n ( n  1) = 105n cn2  105n  n(cn – 105)  0  n  105/c Since n is positive  cn – 105  0  contradiction: n cannot be smaller than a constant

  42. Asymptotic notations (cont.) • -notation • (g(n)) is the set of functions with the same order of growth as g(n)

  43. Examples • n2/2 –n/2 = (n2) • ½ n2 - ½ n ≤ ½ n2n ≥ 0  c2= ½ • ½ n2 - ½ n ≥ ½ n2 - ½ n * ½ n ( n ≥ 2 ) = ¼ n2 c1= ¼ • n ≠ (n2): c1 n2≤ n ≤ c2 n2  only holds for: n ≤ 1/c1

  44. Examples • 6n3 ≠ (n2): c1 n2≤ 6n3 ≤ c2 n2  only holds for: n ≤ c2 /6 • n ≠ (logn): c1logn≤ n ≤ c2 logn  c2 ≥ n/logn,  n≥ n0 – impossible

  45. Relations Between Different Sets • Subset relations between order-of-growth sets. RR ( f ) O( f ) • f ( f )

  46. Asymptotic Notations

  47. More Examples • For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) = Θ(g(n)). Determine which relationship is correct. • f(n) = log n2; g(n) = log n + 5 • f(n) = n; g(n) = log n2 • f(n) = log log n; g(n) = log n • f(n) = n; g(n) = log2 n • f(n) = n log n + n; g(n) = log n • f(n) = 10; g(n) = log 10 • f(n) = 2n; g(n) = 10n2 • f(n) = 2n; g(n) = 3n f(n) =  (g(n)) f(n) = (g(n)) f(n) = O(g(n)) f(n) = (g(n)) f(n) = (g(n)) f(n) = (g(n)) f(n) = (g(n)) f(n) = O(g(n))

  48. Master Theorem

  49. Solve the Recurrence T(n) = c if n = 1 2T(n/2) + cn if n > 1 Use Master’s Theorem: Compare n with f(n) = cn Case 2: T(n) = Θ(nlgn)

  50. Merge Sort - Discussion • Running time insensitive of the input • Advantages: • Guaranteed to run in (nlgn) • Disadvantage • Requires extra space N

More Related