CS 221

1. CS 221 Analysis of Algorithms Instructor: Don McLaughlin

2. Algorithms • What are they and • Why do we care?

3. Algorithms • “A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output” • “A sequence of computational steps that transforms input to the output” • From Cormen, Leiserson, Rivest and Stein

4. Algorithms • “Algorithms + Data Structures = Programs” • Niklaus Wirth, Swiss Computer Scientist • Or, perhaps- • Programs = Programming_language(Algoritms + Data Structures)

5. Algorithms • So you might think of an algorithm as the equation behind a function • y=f(x) • Just as an equation is a computational procedure for creating a certain output from a certain input

6. Algorithms • “A sequence of unambiguous instructions for solving a problem, i.e. for obtaining a required output for any legitimate input in a finite amount of time” • From Levitin, 2007

7. Algorithms • So, why do we want to study them? • Because there may be many algorithms intended to solve a problem… • …, some are correct and some are not correct • …and of the correct ones some are better than others (as you will see)

8. Algorithms • We are largely going to be concerned about the efficiency of algorithms • What do we mean by efficiency? • Memory • Time

9. Algorithms • Mostly we are going to explore the efficiency of algorithms in terms of run time (T) • In particular we are going to study the growth of algorithms in relation to the size of the problem

10. Algorithms • But, how? • Theoretically, mathematically • Empirically • Visually (algorithm visualization)

11. Theoretical Analysis of Algorithms • To do this we need a few things- • A computational model (what is the computer) • A language for expressing the algorithm • A metric and methodology for measuring the performance and efficiency of an algorithm

12. Theoretical Analysis of Algorithms • RAM – Random Access Machine Model • essentially Von Neumann architecture • A CPU… • …with a fixed and finite set of primitive instructions • …which run in a fixed amount of time • …connected to a bank of memory… • And the CPU can read or write any memory location in one primitive operation

13. Theoretical Analysis of Algorithms • RAM – Random Access Machine Model • Primitive operations might be • Assign a value to variable (memory location) • Call a method or subroutine • Arithmetic operation • Compare two values • Access elements of an array via index • Return from method • Return from algorithm

14. Theoretical Analysis of Algorithms • Language for expressing an algorithm • Programming Language • C, VB, Java,… • Flowcharts

15. Theoretical Analysis of Algorithms • Language for expressing an algorithm • Pseudocode • Natural language expression of algorithm • Really mix of natural language and language from programming language constructs • Intended for human consumption, not computers

16. Theoretical Analysis of Algorithms • Pseudo-code – has some rules • Algorithm declaration – must name algorithm and its parameters • Must define range of input, and domain of output • Expressions – use <- for assignment, = for logical comparison • Scope denoted by indentation • Decision structures • If – Then – Else • While – Do loops • Repeat – until loops • For loops • Method calls • Method returns (including return from algorithm)

17. Theoretical Analysis of Algorithms • Pseudo-code – an example • arrayMax – find the largest value in an array of values Algorithm arrayMax(A, n) Input:An array A storing n >= 1 integers) Output: the maximum value in the array A currentMax <- A[0] for i <-1 to n – 1 do if currentMax < A[i] then currentMax <- A[i] return currentMax

18. Theoretical Analysis of Algorithms • Insertion-Sort Algorithm Insertion-sort(A, n) Input:An array A storing n >= 1 integers) Output: the array A sorted such that A0 < A1 < A2 < …An for j <- 2 to n do key <- A[j] #insert A[j] into the sorted sequence i <- j – 1 while i > 0 and A[j] > key do A[j +1] <- A[j] i <- i -1 A[i + 1] <- key return A

19. Theoretical Analysis of Algorithms • So now the question is - how do we determine the run-time T of these algorithms? • Keep in mind that we need to understand the run-time of algorithms in the context of the size of the problem • …which we will express as n • So we are looking for T(n) … for any given algorithm

20. Theoretical Analysis of Algorithms • So how are we going to do this? • Counting

21. Theoretical Analysis of Algorithms

22. Theoretical Analysis of Algorithms

23. Theoretical Analysis of Algorithms • So T(n) for arrayMax is T(n) = 2 + 1 + n + 4(n-1) + 1 = 5n T(n) = 2 + 1 + n + 6(n-1) + 1 = 7n-2

24. Theoretical Analysis of Algorithms • So T(10) for arrayMax is • At least 5(10) = 50 • Or • No worse than 7(10)-2 = 68

25. Theoretical Analysis of Algorithms

26. Theoretical Analysis of Algorithms

27. Theoretical Analysis of Algorithms

28. Theoretical Analysis of Algorithms • So, what is the run-time T(n) for the algorithm arrayMax? • T(n) T(n) = c1 + c2(n-1) + c3(n-1) + c4(n-1) + 1 T(n) = c1 + (n-1)(c2 + c3 +c4) + 1

29. Theoretical Analysis of Algorithms • Was that every-case, best-case, worst-case? • Why?

30. Theoretical Analysis of Algorithms • So, what would T(n) be for best-case? T(n) = c1 + c2(n-1) + c3(n-1) + 1c5 T(n) = c1 + (n-1)(c2 + c3) + 1c5

31. Theoretical Analysis of Algorithms • Insertion-Sort Algorithm Insertion-sort(A, n) Input:An array A storing n >= 1 integers) Output: the array A sorted such that A0 < A1 < A2 < …An for j <- 2 to n do key <- A[j] #insert A[j] into the sorted sequence i <- j – 1 while i > 0 and A[j] > key do A[j +1] <- A[j] i <- i -1 A[I + 1] <- key return A

32. Theoretical Analysis of Algorithms

33. Theoretical Analysis of Algorithms

34. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1) So… what is the best-case scenario (shortest run-time)?

35. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort Ok, that would mean what in terms of the execution of the algorithm? … that tj = 1 for all j …therefore… … Σ(tj) = n – 1 …and c6 and c7 drop out

36. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort • So, T(n) best-case is T(n) = c1n + c2(n-1) + c4(n-1) + c5(n-1) + c8(n-1) = c1n + (c2 + c4 +c5 +c8)(n-1) = (c1+c2+ c4+c5+c8)n-(c2+c4+c5+c8)

37. Theoretical Analysis of Algorithms

38. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1) So… what is the worst-case scenario (longest run-time)?

39. Theoretical Analysis of Algorithms • By the way - rules of thumb (for now) • Σnj=2j = n(n+1)/2 - 1 • Σnj=2(j-1) = n(n-1)/2

40. Theoretical Analysis of Algorithms • So, the worst-case T(n) = c1n + c2(n-1) + c4(n-1) + C5(n(n+1)/2-1) + c6(n(n-1)/2) + c7(n(n-1)/2) + c8(n-1)