Recursion and Cost: Orders of Growth

1. Recursion and Cost:Orders of Growth CMSC 11500 Introduction to Computer Programming October 9, 2002

2. Roadmap • Recap:Structural and Generative Recursion Recursive and Iterative Processes • Recursion (continued) • Tree recursion • Recursive and Iterative solutions • Order of Growth • Analyzing resource requirements • Example: Exponentiation

3. Recap: Structural &Generative Recursion • Lists: self-referential data structures • List manipulation: Structural recursion • Lists as input and input • Lists of lists: hierarchical structure • Generative recursion • Recursive problem solutions • Termination arguments; reduction & combination • (Linear) Recursive processes • Iterative processes

4. Tree Recursive Processes • Fibonacci: • Originally developed to model rabbit reproduction rates • Fib(n)= 0 if n=0; Fib(n) = 1 if n=1 • O.w. Fib(n) = Fib(n-1)+Fib(n-2) • 0,1,1,2,3,5,8,13,21,34,55,89,... (define (fib n) (cond ((= n 0) 0) ((= n 1) 1) (else (+ (fib (- n 1)) (fib (- n 2))))))

5. Recursive Pattern Fib 5 Fib 3 Fib 4 Fib 1 Fib2 Fib 2 Fib 3 1 Fib 1 Fib 0 Fib 0 Fib1 Fib 1 Fib 2 1 0 0 1 1 Fib 0 Fib1 0 1

6. Fibonacci Analysis • Note: Lots of computation • Increases exponentially in size of n • Number of leaves is Fib(n+1) • Integer closest to where • Note: Lots of wasted computation • Entire computation of Fib(3) is repeated • Iterative implementation • Successive additions • Increases only linearly in n

7. Iterative Fibonacci (define (fib n) (fib-iter 1 0 n)) (define (fib-iter a b max) (if (= count 0) b (fib-iter (+ a b) a (- count 1)))))

8. Orders of Growth • How much resource does computation use? • Time: how many steps? • Space: how much memory? • How do resource needs change with input size? • If input doubles in size, do requirements • Stay the same? - constant - e.g. fact-iter space • Double? - linear - e.g. fact-iter time • Quadruple? - quadratic (n^2) - naïve sort • Go up exponentially? - Fibonacci tree

9. Defining Order of Growth • n: Measure of size of input • e.g count, elts in list, nodes/edges in graph, • R(n):amount of resource used solve problem of size n

10. Order of Growth • “Constant”: Theta(1): Fact-iter space • “Linear”: Theta(n): Factorial time • “Exponential”: Theta(phi^n): Fibonacci tree • “Logarithmic”: Theta(log n): Fast exponent

11. Example:Exponents • Linear recursive (define (power number exponent) (cond ((= exponent 0) 1) ((= exponent 1) number) (else (* number (power number (- exponent 1))))))

12. Example: Exponents: Iterative (define (power number exponent) (power-iter number 1 exponent)) (define (power-iter value product counter) (if (= counter 0) product (power-iter value (* value product) (- counter 1))))

13. Fast Exponentiation • Key insight: To compute b^8, • b^2=b*b; b^4=b^2*b^2; b^8=b^4*b^4 • Only 3 multiplications to compute b^8 • Logarithmic growth: log 8 = 3 • Detail: Odd exponent: b^7=b*b^6 (define (fast-exp b n) (cond ((= n 0) 1) ((even? n) (square (fast-exp b (/ n 2)))) (else (* b (fast-exp b (- n 1))))))

14. Summary • Recursive and Iterative Processes • Tree recursion • Computation and Resource Consumption • Time & Space • Increase in resource needs with size of input • Characteristic of implementation, not of problem • Fibonacci: exponential; Fib-iter: linear

15. Next Time • Procedures as first class objects • Procedures as input and output of procedures • Scope with let and lambda