1 / 60

CS421 - Course Information

CS421 - Course Information. Website Syllabus Schedule The Book:. What is an algorithm?. Informally, an algorithm is any well-defined computational procedure that takes some value(s) as input and produces some value(s) as output. Goals for an algorithm: Correct Terminates Efficient.

xena-cabrera
Download Presentation

CS421 - Course Information

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS421 - Course Information • Website • Syllabus • Schedule • The Book:

  2. What is an algorithm? Informally, an algorithm is any well-defined computational procedure that takes some value(s) as input and produces some value(s) as output. Goals for an algorithm: • Correct • Terminates • Efficient

  3. What is CS421 about? We will engage in the theoretical study of the design and analysis of computer algorithms. • Analysis:predict the cost of an algorithm in terms of performance and resources used • Design:design algorithms which minimize such costs

  4. Machine Model Assumptions.. Random Access Machine (RAM) Model: • Any memory cell can be accessed in 1 step. • Memory is not limited (unbounded). • Arbitrarily large integers can be stored in each memory cell. • Operations are executed sequentially. • Operators include: • primitive arithmetic (+, -, *, /, modulo, etc..) • logic (if..then, and, or, etc..) • comparators (<, >, =, etc..) and • function calls. • Each operation has a unit cost of 1.

  5. An Example.. Sorting Input: A sequence of n numbers,áa1, a2, …, anñ Output: A permutation (reordering),áa'1, a'2, …, a'nñ, of the input sequence such thata'1£ a'2£ …£ a'n. Example: Input:<13, 7, 42, 3, 6, 41> Output:<3, 6, 7, 13, 41, 42>

  6. Insertion Sort Pseudocode: INSERTION-SORT (A) 1 forj←2tolength[A] 2 dokey←A[j] 3 i←j–1 4 while i>0 and A[i]>key 5 doA[i+1]←A[i] 6 i←i –1 7 A[i+1]=key

  7. Example of insertion sort 8 2 4 9 3 6

  8. 8 2 4 9 3 6 Example of insertion sort

  9. 8 2 4 9 3 6 Example of insertion sort 2 8 4 9 3 6

  10. 8 2 4 9 3 6 2 8 4 9 3 6 Example of insertion sort

  11. 8 2 4 9 3 6 2 8 4 9 3 6 Example of insertion sort 2 4 8 9 3 6

  12. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 Example of insertion sort

  13. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 Example of insertion sort 2 4 8 9 3 6

  14. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 Example of insertion sort 2 4 8 9 3 6

  15. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 Example of insertion sort 2 4 8 9 3 6 2 3 4 8 9 6

  16. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 Example of insertion sort 2 4 8 9 3 6 2 3 4 8 9 6

  17. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 Example of insertion sort 2 4 8 9 3 6 2 3 4 8 9 6 2 3 4 6 8 9 end

  18. How “Good” is Insertion Sort? Recall our goals for an algorithm: • Correct • Terminates • Efficient

  19. Correctness • Informally: At each step the “current card”, j, is inserted into an already sorted subarray A[1..j-1]. • More formally: The loop invariant(a condition that does not change if correct) is that at the start of each for-loop, the subarray A[1..j-1] consists of the elements in A[1..j-1] but in sorted order.

  20. Correctness These properties must hold for a loop invariant: • Initialization: It is true prior to the first iteration. • Maintenance: It is true before an iteration of the loop and remains true before the next iteration. • Termination: When the loop terminates, the invariant yields a useful property which helps show the algorithm is correct.

  21. Correctness In the case of insertion sort: • Initialization: If j=2 at initialization, A[1..j-1] consists of a single element which is by definition sorted. • Maintenance: Informally, the for-loop works by moving A[j-1], A[j-2], etc. one position to the right until the correct position for A[j] is found. Since A[1..j] is now sorted, when j is incremented, A[1..j-1] is sorted. • Termination: The loop terminates when j=n+1. From the Maintenance property we know A[1..j-1] is now sorted. That is A[1..n] is now sorted. Hence the algorithm is correct.

  22. Termination • As we will see later in the semester determining if an arbitrary program terminates or halts is undecidable for Turing machines. • In the specific instance of Insertion-Sort, it is easy to see that the two loops iterate at most n times each and thus the algorithm does not run indefinitely and does terminate.

  23. Efficiency • Running time is a measure of how many steps or primitive operations were performed. • We have already stated that in the RAM machine model each operator has cost 1, but lets assume instead each operator has cost ci, where i is a line in our algorithm.

  24. Kinds of Analysis • Worst-case:T(n) = maximum time of algorithm on any input of size n. • Average-case:T(n) = expected time of algorithm over all inputs of size n. • Requires assumption of statistical distribution of inputs. • Best-case:T(n) = minimum time of algorithm on any input of size n. • Problematic because a generally slow algorithm may works fast on some input.

  25. Running Time Lets analyze our algorithm once more.. INSERTION-SORT (A)costtimes 1 forj←2tolength[A] c1 n 2 dokey←A[j] c2 n-1 3 i←j–1 c3 n-1 4 while i>0 and A[i]>key c4 ∑j=2..ntj 5 doA[i+1]←A[i] c5∑j=2..n(tj-1) 6 i←i –1 c6∑j=2..n(tj-1) 7 A[i+1]=key c7n-1

  26. Best Case INSERTION-SORT (A)costtimes 1 forj←2tolength[A] c1 n 2 dokey←A[j] c2 n-1 3 i←j–1 c3 n-1 4 while i>0 and A[i]>key c4 ∑j=2..ntj 5 doA[i+1]←A[i] c5∑j=2..n(tj-1) 6 i←i –1 c6∑j=2..n(tj-1) 7 A[i+1]=key c7n-1 T(n) = (c1+c2+c3+c4+c7)n – (c2+c3+c4+c7)

  27. Worst Case INSERTION-SORT (A)costtimes 1 forj←2tolength[A] c1 n 2 dokey←A[j] c2 n-1 3 i←j–1 c3 n-1 4 while i>0 and A[i]>key c4 ∑j=2..ntj 5 doA[i+1]←A[i] c5∑j=2..n(tj-1) 6 i←i –1 c6∑j=2..n(tj-1) 7 A[i+1]=key c7n-1 T(n) = (c4+c5+c6)n2/2 + (c1+c2+c3+c4/2-c5/2-c6/2c7)n – (c2 + c3 + c4 + c7)

  28. Average Case INSERTION-SORT (A)costtimes 1 forj←2tolength[A] c1 n 2 dokey←A[j] c2 n-1 3 i←j–1 c3 n-1 4 while i>0 and A[i]>key c4 ∑j=2..ntj 5 doA[i+1]←A[i] c5∑j=2..n(tj-1) 6 i←i –1 c6∑j=2..n(tj-1) 7 A[i+1]=key c7n-1 tj = j/2 .. Only out of order half the time.. T(n) = (c4+c5+c6)n2/4 + (c1+c2+c3+c4/4-c5/4-c6/4c7)n – (c2 + c3 + c4 + c7)

  29. Machine Independent Analysis As the size of the input becomes large, the constants ci don’t matter as much as the exponents and log factors. The constants also make machine independent analysis impossible. • Ignore the constants. • Examine growthof T(n) as n → ∞. • Asymptotic Analysis

  30. Order of Growth • The rate of growth is of primary interest, so we consider only the leading term and ignore all constants (e.g. n^2) • Thus, the worst case running time of Insertion Sort is Θ(n2).Quadratic time. • We will define this more precisely later.

  31. Design Approach: Divide and Conquer • Divide the problem into a number of subproblems. • Conquer the subproblems recursively. • Combine the subproblem solutions into the solution for the original problem. • Recursion: when an algorithm calls itself.

  32. Merge Sort • Divide: Divide an n-element array into two subsequences of n/2 elements each. • Conquer: Sort the two subsequences recursively with merge sort. • Combine: Merge the two sorted arrays to produce the sorted sequence. • Special Case: If the sequence has only one element the recursion “bottoms out” as the sequence is sorted by definition.

  33. MERGE-SORT (A[1 . . n]) • If n = 1, return A. • L = A[ 1 . . n/2 ] • R = A[ n/2+1 . . n ] • L = Merge-Sort(L) • R = Merge-Sort(R) • Return Merge(L, R) Merge Sort

  34. Merging two sorted arrays 20 13 7 2 12 11 9 1

  35. Merging two sorted arrays 20 13 7 2 12 11 9 1 1

  36. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 1

  37. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 1 2

  38. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 1 2

  39. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 1 2 7

  40. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 1 2 7

  41. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 1 2 7 9

  42. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 20 13 12 11 1 2 7 9

  43. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 20 13 12 11 1 2 7 9 11

  44. Merging two sorted arrays 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 20 13 12 11 20 13 12 1 2 7 9 11

  45. 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 20 13 12 11 20 13 12 1 2 7 9 11 12 Merging two sorted arrays

  46. 20 13 7 2 12 11 9 1 20 13 7 2 12 11 9 20 13 7 12 11 9 20 13 12 11 9 20 13 12 11 20 13 12 1 2 7 9 11 12 Merging two sorted arrays Time = Q(n) to merge a total of n elements (linear time).

  47. Analyzing merge sort T(n) Q(1) 2T(n/2) Q(n) MERGE-SORTA[1 . . n] • If n = 1, done. • Recursively sort A[ 1 . . n/2 ] and A[ n/2+1 . . n ] . • “Merge” the 2 sorted lists Sloppiness: Should be T( n/2 ) + T( n/2) , but it turns out not to matter asymptotically.

  48. T(n) = Q(1) ifn = 1; 2T(n/2) + Q(n) ifn > 1. Recurrence for merge sort • We shall usually omit stating the base case when T(n) = Q(1) for sufficiently small n, but only when it has no effect on the asymptotic solution to the recurrence.

  49. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

  50. Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. T(n)

More Related