ALGORITHMS THIRD YEAR

1. ALGORITHMSTHIRD YEAR BANHA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATIC Lecture five Dr. Hamdy M. Mousa

2. Recurrences

3. Recurrence • A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs. • For Examples, • The worst-case running time T (n) of the MERGE-SORT procedure could be described by the recurrence T (n) = (1) if n = 1 , T(n) = 2T (n/2) + (n) if n > 1 , • solution T (n) = (n lg n).

4. Recurrence Solution Methods • Three methods for solving recurrences - that is, for obtaining asymptotic “” or “O” bounds on the solution. • The substitution method, we guess a bound and then use mathematical induction to prove our guess correct. • The recursion-tree methodconverts the recurrence into a tree whose nodes represent the costs incurred at various levels of the recursion; we use techniques for bounding summations to solve the recurrence. • The master methodprovides bounds for recurrences of the form T (n) = aT (n/b) + f (n), where a ≥ 1, b > 1, and f (n) is a given function.

5. Running time • The running time T (n)of an algorithm is only defined when nis an integer, since for most algorithms, the size of the input is always an integer. • For example, the recurrence describing the worst-case running time of MERGE-SORT

6. Running time • Example: T (n) = 2T (n/2) + (n), with solution T (n) = (n lg n). • Since the running time of an algorithm on a constant-sized input is a constant, • The boundary conditions are usually expressed as “T (n) = O(1) for sufficiently small n.” • When we desire an exact, rather than an asymptotic, solution, we need to deal with boundary conditions. • In practice, we just use asymptotics most of the time, and we ignore boundary conditions.

7. Substitution method • The substitution method for solving recurrences entails two steps: 1. Guess the solution. 2. Use induction to find the constants and show that the solution works. Example:

8. Substitution method • Guess: T (n) = n lg n + n. [Here, we have a recurrence with an exact function, rather than asymptotic notation, and the solution is also exact rather than asymptotic. We’ll have to check boundary conditions and the base case.] 2.Induction: Basis: n = 1  n lg n + n = 1 = T (n)

9. Substitution method Inductive step: • Inductive hypothesis is that T (k) = k lg k + k for all k < n. • We’ll use this inductive hypothesis for T (n/2).

10. Substitution method For the substitution method: • Name the constant in the additive term. • Show the upper (O) and lower () bounds separately. Might need to use different constants for each. Example: T (n) = 2T (n/2)+(n). If we want to show an upper bound of T (n) = 2T (n/2) + O(n), we write T (n) ≤ 2T (n/2) + cnfor some positive constant c.

11. Substitution method 1. Upper bound: Guess: T (n) ≤ dn lg n for some positive constant d. We are given cin the recurrence, and we get to choose das any positive constant. It’s OK for d to depend on c. Therefore, T (n) = O(n lg n).

12. Substitution method 2.Lower bound: Write T (n) ≥ 2T (n/2) + cn for some positive constant c. Guess:T (n) ≥ dn lg n for some positive constant d. Substitution:

13. Avoiding pitfalls It is easy to err in the use of asymptotic notation. • For example, in the recurrence T (n) = 2T (n/2) + n we can falsely “prove” T (n) = O(n) by guessing T (n) ≤ cn and then arguing T (n) ≤ 2(c n/2) + n ≤ cn + n = O(n) , ⇐ wrong!! • since c is a constant. The error is that we haven’t proved the exact form of the inductive hypothesis, that is, that T (n) ≤ cn.

14. Changing variables • Sometimes, a little algebraic manipulation can make an unknown recurrence similar to one you have seen before. As an example, consider the recurrence T (n) = 2T (√n) + lg n , • We can simplify this recurrence, though, with a change of variables. Renaming m = lg nyields T (2m) = 2T (2m/2) + m . • rename S(m) = T (2m)to produce the new recurrence S(m) = 2S(m/2) + m , • which is very much like recurrence (4.4). Indeed, this new recurrence has the same solution: S(m) = O(m lgm). Changing back from S(m) to T (n), we obtain T (n) = T (2m) = S(m) = O(m lg m) = O(lg n lg lg n).

15. The recursion-tree method • In a recursion tree, each node represents the cost of a single subproblem somewhere in the set of recursive function invocations. We sum the costs within each level of the tree to obtain a set of per-level costs, and then we sum all the per-level costs to determine the total cost of all levels of the recursion. • Recursion tree use to generate a guess. Then verify by substitution method.

16. The recursion-tree method Example: T (n) = T (n/3)+T (2n/3)+(n). For upper bound, rewrite as T (n) ≤ T (n/3) + T (2n/3) + cn; For lower bound, as T (n) ≥ T (n/3) + T (2n/3) + cn. • By summing across each level, the recursion tree shows the cost at each level of recursion (minus the costs of recursive calls, which appear in subtrees):

17. The recursion-tree method There are log3n full levels, and after log3/2n levels, the problem size is down to 1. • Each level contributes ≤ cn. • Lower bound guess: ≥ dn log3n = (n lg n)for some positive constant d. • Upper bound guess: ≤ dn log3/2n = O(n lg n)for some positive constant d. • Then prove by substitution.

18. The recursion-tree method Therefore, T (n) = O(n lg n).

19. The recursion-tree method 2. Lower bound: Guess: T (n) ≥ dn lg n. Substitution: Same as for the upper bound, but replacing ≤ by ≥. End up needing Therefore, T (n) = (n lg n). Since T (n) = O(n lg n)and T (n) =  (n lg n), we conclude that T (n) = (n lg n).