Algorithm

1. Algorithm • An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Design and Analysis of Algorithms - Chapter 1

2. Features of Algorithm • “Besides merely being a finite set of rules that gives a sequence of operations for solving a specific type of problem, an algorithm has the five important features” [Knuth1] • finiteness (otherwise: computational method) • termination • definiteness • precise definition of each step • input (zero or more) • output (one or more) • effectiveness • “Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper” [Knuth1] Design and Analysis of Algorithms - Chapter 1

3. Historical Perspective • Muhammad ibn Musa al-Khwarizmi – 9th century mathematician www.lib.virginia.edu/science/parshall/khwariz.html • Euclid’s algorithm for finding the greatest common divisor Design and Analysis of Algorithms - Chapter 1

4. Greatest Common Divisor (GCD) Algorithm 1 • Step 1 Assign the value of min{m,n} to t • Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, to Step 4 • Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4 • Step 4 Decrease the value of t by 1. Go to Step 2 • Note: m and n are positive integers Design and Analysis of Algorithms - Chapter 1

5. Correctness • The algorithm terminates, because t is decreased by 1 each time we go through step 4, 1 divides any integer, and t eventually will reach 1 unless the algorithm stops earlier. • The algorithm is partially correct, because when it returns t as the answer, t is the minimum value that divides both m and n Design and Analysis of Algorithms - Chapter 1

6. GCD Procedure 2 • Step 1 Find the prime factors of m • Step 2 Find the prime factors of n • Step 3 Identify all the common factors in the two prime expansions found in Steps 1 and 2. If p is a common factor repeated i times in m and j times in n, assign to p the multiplicity min{i,j} • Step 4 Compute the product of all the common factors with their multiplicities and return it as the GCD of m and n • Note: as written, the algorithm requires than m and n be integers greater than 1, since 1 is not a prime Design and Analysis of Algorithms - Chapter 1

7. Procedure 2 or Algorithm 2? • Procedure 2 is not an algorithm unless we can provide an effective way to find prime factors of a number • The sieve of Eratosthenes is an algorithm that provides such an effective procedure Design and Analysis of Algorithms - Chapter 1

8. Euclid’s Algorithm • E0 [Ensure m geq n] If m lt n, exchange m with n • E1 [Find Remainder] Divide m by n and let r be the remainder (We will have 0 leq r lt n) • E2 [Is it zero?] If r=0, the algorithm terminates; n is the answer • E3 [Reduce] Set m to n, n to r, and go back to E1 Design and Analysis of Algorithms - Chapter 1

9. Termination of Euclid’s Algorithm • The second number of the pair gets smaller with each iteration and cannot become negative: indeed, the new value of n is r = m mod n, which is always smaller than n. Eventually, r becomes zero, and the algorithms stops. Design and Analysis of Algorithms - Chapter 1

10. Correctness of Euclid’s Algorithm • After step E1, we have m = qn + r, for some integer q. If r = 0, then m is a multiple of n, and clearly in such a case n is the GCD of m and n. If r !=0, note that any number that divides both m and n must divide m – qn = r, and any number that divides both n and r must divide qn + r = m; so the set of divisors of {m,n} is the same as the set of divisors of {n,r}. In particular, the GCD of {m,n} is the same as the GCD of {n,r}. Therefore, E3 does not change the answer to the original problem. [Knuth1] Design and Analysis of Algorithms - Chapter 1

11. Algorithm Design Techniques • Euclid’s algorithm is an example of a “Decrease and Conquer” algorithm: it works by replacing its instance by a simpler one (in step E3) • It can be shown (exercise 5.6b) that the instance size, when measured by the second parameter, n, is always reduced by at least a factor of 2 after two successive iterations of Euclid’s algorithm • In Euclid’s original, the remainder m mod n is computed by repeated subtraction of n from m • Step E0 is not necessary for correctness; it improves time efficiency Design and Analysis of Algorithms - Chapter 1

12. Notion of algorithm problem algorithm “computer” input output Algorithmic solution Design and Analysis of Algorithms - Chapter 1

13. Example of computational problem: sorting • Statement of problem: • Input: A sequence of n numbers <a1, a2, …, an> • Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤ a´j whenever i < j • Instance: The sequence <5, 3, 2, 8, 3> • Algorithms: • Selection sort • Insertion sort • Merge sort • (many others) Design and Analysis of Algorithms - Chapter 1

14. Selection Sort • Input: array a[1],…,a[n] • Output: array a sorted in non-decreasing order • Algorithm: • for i=1 to n swap a[i] with smallest of a[i+1],…a[n] • see also pseudocode, section 3.1 Design and Analysis of Algorithms - Chapter 1

15. Sorting: Some Terms • Key • Stable sorting • In place sorting Design and Analysis of Algorithms - Chapter 1

16. Graphs: Some Terms • Graph • Vertices, edges • Undirected, directed • Loops • Complete, dense, sparse • Paths: sequences of vertices or of edges? • Path length: number of edges in path • Walk: repeated vertices and edges allowed • Path: repeated vertices are allowed, repeated edges are not allowed • Simple path: neither repeated vertices nor repeated edges are allowed • Cycle: simple path that starts and ends at the same vertex Design and Analysis of Algorithms - Chapter 1

17. Some Well-known Computational Problems • Sorting • Searching • Shortest paths in a graph • Minimum spanning tree • Primality testing • Traveling salesman problem • Knapsack problem • Chess • Towers of Hanoi • Program termination Design and Analysis of Algorithms - Chapter 1

18. Basic Issues Related to Algorithms • How to design algorithms • How to express algorithms • Proving correctness • Efficiency • Theoretical analysis • Empirical analysis • Optimality Design and Analysis of Algorithms - Chapter 1

19. Brute force Divide and conquer Decrease and conquer Transform and conquer Algorithm design strategies See http://www.aw-bc.com/info/levitin/taxonomy.html This is not a unique taxonomy. • Greedy approach • Dynamic programming • Backtracking and Branch and bound • Space and time tradeoffs Design and Analysis of Algorithms - Chapter 1

20. Analysis of Algorithms • How good is the algorithm? • Correctness • Time efficiency • Space efficiency • Does there exist a better algorithm? • Lower bounds • Optimality Design and Analysis of Algorithms - Chapter 1

21. What is an algorithm? • Recipe, process, method, technique, procedure, routine,… with the following requirements: • Finiteness • terminates after a finite number of steps • Definiteness • rigorously and unambiguously specified • Input • valid inputs are clearly specified • Output • can be proved to produce the correct output given a valid input • Effectiveness • steps are sufficiently simple and basic Design and Analysis of Algorithms - Chapter 1

22. Why study algorithms? • Theoretical importance • the core of computer science • Practical importance • A practitioner’s toolkit of known algorithms • Framework for designing and analyzing algorithms for new problems Design and Analysis of Algorithms - Chapter 1

23. Correctness • Termination • Well-founded sets: find a quantity that is never negative and that always decreases as the algorithm is executed • Partial Correctness • For recursive algorithms: induction • For iterative algorithms: axiomatic semantics, loop invariants Design and Analysis of Algorithms - Chapter 1

24. Complexity • Space complexity • Time complexity • For iterative algorithms: sums • For recursive algorithms: recurrence relations Design and Analysis of Algorithms - Chapter 1