Analyzing Programs: Order of Growth

1. Analyzing Programs:Order of Growth CMSC 11500 Introduction to Computer Programming October 11, 2002

2. Roadmap • Motivation: Max • Order of Growth • Analyzing resource requirements • Example 1: Tree Recursion: Fibonacci • Recursive and Iterative solutions • Example 2: Exponentiation • Recursive • Iterative • Fast

3. Orders of Growth • How much resource does computation use? • Time: how many steps? • Space: how much memory? • How do resource needs change with input size? • If input doubles in size, do requirements • Stay the same? - constant - e.g. fact-iter space • Double? - linear - e.g. fact-iter time • Quadruple? - quadratic (n^2) - naïve sort • Go up exponentially? - Fibonacci tree

4. Defining Order of Growth • n: Measure of size of input • e.g count, elts in list, nodes/edges in graph, • R(n):amount of resource used solve problem of size n

5. Order of Growth • “Constant”: Theta(1): Fact-iter space • “Linear”: Theta(n): Factorial time • “Exponential”: Theta(phi^n): Fibonacci tree • “Logarithmic”: Theta(log n): Fast exponent

6. Tree Recursive Processes • Fibonacci: • Originally developed to model rabbit reproduction rates • Fib(n)= 0 if n=0; Fib(n) = 1 if n=1 • O.w. Fib(n) = Fib(n-1)+Fib(n-2) • 0,1,1,2,3,5,8,13,21,34,55,89,... (define (fib n) (cond ((= n 0) 0) ((= n 1) 1) (else (+ (fib (- n 1)) (fib (- n 2))))))

7. Recursive Pattern Fib 5 Fib 3 Fib 4 Fib 1 Fib2 Fib 2 Fib 3 1 Fib 1 Fib 0 Fib 0 Fib1 Fib 1 Fib 2 1 0 0 1 1 Fib 0 Fib1 0 1

8. Fibonacci Analysis • Note: Lots of computation • Increases exponentially in size of n • Number of leaves is Fib(n+1) • Integer closest to where • Note: Lots of wasted computation • Entire computation of Fib(3) is repeated • Iterative implementation • Successive additions • Increases only linearly in n

9. Iterative Fibonacci (define (fib n) (fib-iter 1 0 n)) (define (fib-iter a b count) (if (= count 0) b (fib-iter (+ a b) a (- count 1)))))

10. Example:Exponents • Linear recursive (define (power number exponent) (cond ((= exponent 0) 1) ((= exponent 1) number) (else (* number (power number (- exponent 1))))))

11. Example: Exponents: Iterative (define (power number exponent) (power-iter number 1 exponent)) (define (power-iter value product counter) (if (= counter 0) product (power-iter value (* value product) (- counter 1))))

12. Fast Exponentiation • Key insight: To compute b^8, • b^2=b*b; b^4=b^2*b^2; b^8=b^4*b^4 • Only 3 multiplications to compute b^8 • Logarithmic growth: log 8 = 3 • Detail: Odd exponent: b^7=b*b^6 (define (fast-exp b n) (cond ((= n 0) 1) ((even? n) (square (fast-exp b (/ n 2)))) (else (* b (fast-exp b (- n 1))))))

13. Summary • Recursive and Iterative Processes • Tree recursion • Computation and Resource Consumption • Time & Space • Increase in resource needs with size of input • Characteristic of implementation, not of problem • Fibonacci: exponential; Fib-iter: linear

14. Next Time • Procedures as first class objects • Procedures as input and output of procedures • Scope with let and lambda