An Example in the Design and Analysis of an Algorithm

1. An Example in the Design and Analysis of an Algorithm • Demonstrates: -- Recurrence Relations -- Design Techniques -- The fact that analysis provide useful information • Problem to be solved: BUNNY PROBLEM When a pair of rabbits is less than one month old, it cannot reproduce, but during the second and each succeeding month of life, each pair of rabbits gives rise to one new pair of rabbits. Given that you (Fibonacci) just purchased a pair of new born rabbits, how many pairs will you have n months from now?

2. Illustration

3. Observation

4. The Recurrence Relation • i  0 1 2 3 4 5 6 7 8 9 ... f(i)  1 1 2 3 5 8 13 21 34 55 ...  • f(i) satisfies the following recurrence relation f(i) = f(i-1) + f(i-2) for I ≥ 2, f(0) = f(1) = 1 B(n+1) = B(n) + S(n) = f(n) S(n+1) = B(n) = f(n-1) f(n+1) = B(n+1) + S(n+1) = f(n) + f(n-1)

5. Solution #1 • Function f(n: integer): integer; • begin • if n ≤ 1 then return(1) • else return(f(n-1)+f(n-2)) • end. • Analysis: Two factors -- # of function calls and # of additions. Let FC(n) = # of function calls needed to evaluate f(n) ADD(n) = # of additions needed

6. Analysis • Consider a tree of function calls (showing the value of n at each node).

7. Derivation • Thus, we have the recurrence relations FC(n) = FC(n-1) + FC(n-2) + 1, if n≥2 FC(0) = FC(1) = 1 ADD(n) = ADD(n-1) + ADD(n-2) + 1, if n≥2 ADD(0) = ADD(1) = 0 • Later, we will learn how to solve such equations. For now f(n) = FC(n) = 2f(n)-1, ADD(n) = f(n)-1

8. Asymptotic Performance • Since   • Hence, f(n) = = ADD(n) and FC(n) = O(1.618n) Exponential!!

9. Avoid Redundant Computations • We can use the dynamic programming technique. Idea: Solving the problem (calculating f(n)) consists of solving two smaller problems (calculating f(n-1) and f(n-2)) and "merging" (i.e., adding) the two smaller solutions to obtain a solution to the original problem. Also, we will have to solve the same problem more than once. When a problem has these characteristics, build up a TABLE of SOLUTIONS to the smaller problems, starting with the smallest, and proceed upwards to the largest (original) problem.

10. Solution #2 • Program FIB • var f: array[0..n] of integer; i: integer; • begin • f[0] := 1; • f[1] := 1; • for i := 2 to n do f[i] := f[i-1] + f[i-2]; • end. • The answer is stored in f[n].

11. Analysis • Clearly n-1 operations (additions) are required -- time is O(n). Space required is also O(n). (Space can be reduce to O(1).) • This method usually trades space for time (though not in this case). LINEAR!! a vast improvement.

12. Modified Solution #2 • Program FIB • var a, b, i: integer; • begin • a := 1; • b := 1; • for i := 2 to n do • begin b:= a + b; a := b – a end; • end. • The answer is stored in the variable “b”.

13. Interlude •  This still doesn’t seem very good. Do we need to calculate all the f(i)? Can't we skip over some? • Try substituting recurrence into itself. f(n) = f(n-1) + f(n-2) = f(n-2) + f(n-3) + f(n-2) = 2f(n-2) + f(n-3) = 3f(n-3) + 2f(n-4) = 5f(n-4) + 3f(n-5) = 8f(n-5) + 5f(n-6) (these #s look familiar ) Conjecture: f(n) = f(a)f(n-a) + f(a-1)f(n-a-1) where n≥2; n-1 ≥ a ≥1.

14. Solution #3 • Function f(x : integer) : integer; • var i, j: integer; • begin • if x ≤1 then return(1) • else begin • j :=x/2; • i :=x/2 ; • return(f(i)*f(j)+f(i-1)*f(j-1)); • end; • end.

15. Analysis • Why did we choose i and j  x/2? Another design heuristic -- BALANCING: divide problem sizes as evenly as possible. "Divide and Conquer". • Let n = 100 • Look at "tree" of function calls (notice that FC(n) = 4FC(n/2)+1, FC(1)=1).

16. Analysis

17. What Happened? • But obviously the above can be improved. f(100) calls f(50) twice! • Another important technique -- (subtree isomorphism) Eliminate REDUNDANT Computation

18. A Possible Improvement • Consider 2 cases: • x is even then i = j we want to return f(i)2+f(i-1)2. • x is odd then i-1 = j we want to return f(i-1)*(f(i)+f(i-2)). • What does the tree look like now? • An even node has 1 even son and 1 odd son. • Some odd nodes have 2 odd and 1 even sons (5, 9, 13, ..., i.e. 1 mod 4). • The rest have 1 odd and 2 even sons ( 3 mod 4). • The tree has between O(n1.271) and O(n1.385) nodes. • Using the relation f(i) = f(i-1) + f(i-2), we have f(i-2) = f(i) - f(i-1), the "odd" return value can be further simplified: f(i-1)(f(i)+f(i-2)) = f(i-1)(2f(i)-f(i-1))

19. Solution #4 • Function f(x : integer) : integer; • var i, FI, FIM1: integer; • begin • if x ≤1 then return(1) • else begin • i := x/2; • FI := f(i); • FIM1 := f(i-1); • if x is even • then return(FI**2+FIM1**2) • else return(FIM1*(2*FI-FIM1)) • end; • end.

20. Analysis • Tree of calls is now completely binary. Take n = 32, • Height of tree is about log n, hence it has about n nodes, i.e., FC(n) = O(n).

21. Are You Satisfied? • Obviously there still is some redundant computation. Can all the calls at one level be done only once? Then we would have:

22. Solution #5 Procedure calc(x : integer; var fx, fxm1: integer); var i, fi, fim1, fim2: integer; begin if x ≤ 1 then begin fx := 1; fxm1 := 1 end else begin i := x/2; calc(i, fi, fim1) if x is even then begin fx := fi**2+fim1**2; fxm1 := fim1*(2*fi-fim1)) end else begin fim2 := fi-fim1; fx := fim1*(2*fi-fim1)); fxm1 := fim1**2+fim2**2 end; end; end. Analysis: Tree obviously has Q(log n) nodes!

23. Solution #6

24. Repeated Squaring • X(n) = AX(n-1) = A (A  X(n-2)) = … = An-1  X(1). • Use this relation, we can design aQ(log n) algorithm for calculating Fibonacci numbers. • Hint: Calculate A, A2, A4, …, A2log n, then compute An-1 using these items. Questions: Is there a better way for calculating An-1?

25. Exercise • Implement the above solutions 1, 2 and 6, and compare their running time for calculating f(50), f(100), and f(150). • Can your programs calculate f(500)? • How about f(1000)? • Now, you should realize that real implementation sometimes take a lot of efforts even you understand the underlying algorithm.