Orders of Growth: Recap, Fast Exponentiation, Hanoi Towers, Perfect Numbers

1. Lecture 4 Recap: Order of Growth Fun with recursion Fast exponentiation Hanoi towers Perfect numbers (if time permits) מבוא מורחב - שיעור 4

2. The worst-case over allinputs of size n Orders of Growth • Suppose n is a parameter that measures the size of a problem (the size of its input) • R(n)measures the amount of resources needed to solve a problem of size n. • Two common resources are space, measured by the number of deferred operations, and time, measured by the number of primitive steps. מבוא מורחב - שיעור 4

3. Orders of Growth • Want to estimate the “order of growth” of R(n): R1(n) = 100n2 R2(n) = 2n2+10n+2 R3(n) = n2 Are all the same in the sense that if we multiply the input by a factor of 2, the resource consumption increases by a factor of `about’ 4 Order of growth is proportional to n2 מבוא מורחב - שיעור 4

4. Orders of Growth n log n n n2 n3 2n מבוא מורחב - שיעור 4

5. Orders of Growth • We say that R(n)Q(f(n))if there are constants c1,c2> 0 such that for all n≥1 c1 f(n) ≤R(n) ≤c2 f(n) • We say that R(n)O(f(n))if there is a constant c>0such that for all n≥1 • R(n) ≤cf(n) • We say that R(n)(f(n))if there is a constant c>0such that for all n≥1 • R(n) ≥cf(n) מבוא מורחב - שיעור 4

6. R(n)O(f(n)) R(n) does not grow faster than f(n) R(n)(f(n)) R(n) does not grow slower than f(n) R(n)Q(f(n)) R(n) and f(n) grow at the same rate The intuition of this notation… R(n)O(f(n))  f(n)(R(n)) R(n)Q(f(n))  f(n)Q(R(n)) מבוא מורחב - שיעור 4

7. f(n) = O(g(n)) 1 מבוא מורחב - שיעור 4

8. f(n) = Θ (g(n)) 1 מבוא מורחב - שיעור 4

9. Orders of Growth True or False? f 100n2 O(n) t 100n2 (n) 100n2 Q(n) f 100n2 Q(n2) t True or False? f 2100 (n) t 2100n  O(n2) 2n  Q(n) f 210 Q(1) t מבוא מורחב - שיעור 4

10. The conditional form (cond (<test-1> <consequent-1>) (<test-2> <consequent-2>) …. (<test-n> <consequent-n>) (else <consequent-else>)) (define (abs x) (cond ((> x 0) x) ((= x 0) 0) ((< x 0) (- x)))) (else (- x)))) מבוא מורחב - שיעור 4

11. computing ab (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1)))))) מבוא מורחב - שיעור 4

12. Iterative version (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1)) מבוא מורחב - שיעור 4

13. Another algorithm for computing ab • If b is even, then ab = (a2)(b/2) • If b is odd, then ab = a*a(b-1) Problem size reduced to half in at most two steps. (define (exp-fast1 a b) ; computes ab (cond ((= b 0) 1) ((even? b) (exp-fast1 (* a a) (/ b 2))) (else (* a (exp-fast1 a (- b 1))))))) מבוא מורחב - שיעור 4

14. (exp-fast 3 56) (define (exp-fast1 a b) (cond ((= b 0) 1) ((even? b) (exp-fast1 (* a a) (/ b 2))) (else (* a (exp-fast1 a (- b 1))))))) ; compute 3^56 (exp-fast1 3 56) ; b=1110002 (exp-fast1 9 28) ; b= 111002 (exp-fast1 81 14) ; b= 11102 (exp-fast1 6561 7) ; b= 1112 6561 * (exp-fast1 6561 6) ; b= 1102 6561 * (exp-fast1 43046721 3) ; b= 112 6561 * 43046721 * (exp-fast1 43046721 2) ; b= 102 6561 * 43046721 * (exp-fast1 1853020188851841 1) ; b= 12 6561 * 43046721 * 1853020188851841 * (exp-fast1 .. 0) ; b=0 6561 * 43046721 * 1853020188851841 523347633027360537213511521 Note: scheme allows the use of very large numbers מבוא מורחב - שיעור 4

15. How much time does exp-fast take? If b is even: T(b) = T(b/2)+c and if b is odd then: T(b) = T(b-1)+c = T((b-1)/2)+2c = … OK, but what is it? In theta notation, that is! מבוא מורחב - שיעור 4

16. Let’s start with a simpler case • Say that T(b) = T(b/2)+c always • For what values of b does this happen? • T(b) = T(b/2)+c = T(b/4)+c+c = T(b/8)+c+c+c = …. = T(b/(2^k))+ k*c • When does b/(2^k) = 1? • k = log(b) => T(b) = T(1)+ log(b)*c = Q (1)+ Q ( log(b)) = Q(log(b)) Note: logbase1 x = k logbase2 x for some constant k > 0 מבוא מורחב - שיעור 4

17. Counting “odd” steps • We also take “odd” steps (where b becomes odd) • But how often? • Claim: At most one “odd” step per each “even” step • So we make no more than twice the number of steps! • Still Q(log(b)) • Space complexity is Q(log(b)) as well מבוא מורחב - שיעור 4

18. Fast exponentiation (Iterative Approach) product  1, base = a, exp = b Init: Loop: Even? b: base  base2 exp  exp/2 Odd? b: product  product*a exp  exp-1 מבוא מורחב - שיעור 4

19. Scheme code (Iterative Approach) (define (exp-fast2 a b) (define (exp-fast-iter a b product) (cond ((= b 0) product) ((even? b) (exp-fast-iter (* a a) (/ b 2) product)) (else (exp-fast-iter a (- b 1) (* a product))))) (exp-fast-iter a b 1)) מבוא מורחב - שיעור 4

20. Comparing the three exponentiation procedures exp-fast1and exp-fast2 are exponentially faster than exp-1 and exp-2 מבוא מורחב - שיעור 4

21. Towers of Hanoi • Three posts, and a set of disks of different sizes. • A disk can be placed only on a larger disk (or on bottom). • At the beginning all the disks are on the left post. The goal is to move the disks one at a time, while preserving these conditions, until the entire stack has moved from the left post to another You are allowed to move only the topmost disk at a step מבוא מורחב - שיעור 4

22. Use our paradigm • Wishful thinking: • Smaller problem: A problem with one disk less • How do we use it ? To move n disks from post A to post C (using B as aux): Move top n-1 disks from post A to post B (using C as aux) Move the largest disk from post A to post C Move n-1 disks from post B to post C (using A as aux) We solve 2 smaller problems ! מבוא מורחב - שיעור 4

23. Towers of Hanoi (define (move-tower size from to aux) (cond ((= size 1) (one-move from to)) (else (move-tower (- size 1) from aux to) (one-move from to) (move-tower (- size 1) aux to from)))) (define (one-move from to) (display "Move top disk from ") (display from) (display " To ") (display to) (newline)) מבוא מורחב - שיעור 4

24. (move-tower 2 1 3 2) 3 2 1 3 2 1 3 2 1 (move-tower 2 3 2 1) 3 2 1 3 2 1 3 2 1 3 a 2 1 3 2 1 Towers of Hanoi -- trace (move-tower 3 1 2 3) Move top disk from 1 to 2 Move top disk from 1 to 3 Move top disk from 2 to 3 Move top disk from 1 to 2 Move top disk from 3 to 1 Move top disk from 3 to 2 Move top disk from 1 to 2 מבוא מורחב - שיעור 4

25. Orders of growth for towers of Hanoi Denote by T(n) the number of steps that we need to take to solve the case for n disks. T(n) = 2T(n-1) + 1 T(1) = 1 This solves to: T(n) = 2n-1= Q (2n) What does that mean ? מבוא מורחב - שיעור 4

26. Hanoi Towers Say we want to solve the problem for 400 disks. Say it takes a second to move a disk . We need about 2400 seconds. That’s about 2373 years. That’s about 2363 millenniums. Might be longer then the age of the universe …. Infeasible !!!! מבוא מורחב - שיעור 4

27. Let’s buy a fast computerand make it feasible. Our new computer can move giga billion (260 ) disks a second. Absolutely the last word in the field of computing. We need about 2340 seconds. That’s about 2313 years. That’s about 2303 millenniums. Does not help much. Infeasible !!!! An algorithm with exponential time complexityis not scalable מבוא מורחב - שיעור 4

28. What about Space complexity? מבוא מורחב - שיעור 4

29. Towers of Hanoi (define (move-tower size from to aux) (cond ((= size 1) (one-move from to)) (else (move-tower (- size 1) from aux to) (one-move from to) (move-tower (- size 1) aux to from)))) Pending operations (define (one-move from to) (display "Move top disk from ") (display from) (display " To ") (display to) (newline)) מבוא מורחב - שיעור 4

30. (mt 2 2 3 1) (mt 2 3 1 2) (mt 1 2 1 3) (mt 1 2 1 3) (mt 1 1 3 2) (mt 1 3 2 1) (one-move 2 1) (one-move 2 3) (one-move 3 1) (one-move 2 1) (one-move 1 3) (one-move 3 2) (one-move 2 1) Tree Recursion (mt 3 2 1 3) מבוא מורחב - שיעור 4

31. Towers of Hanoi Space complexity The number of pending operations is the height of the recursion tree. So the space complexity is S(n) = Q(n) Note that the second recursive call is treated as tail recursion, and forms an iteration (no pending operation for these calls). מבוא מורחב - שיעור 4

32. Summary • Orders of growth analysis allows us to understand and compare properties of algorithms independently of the machines they run on • Exponential time is usually infeasable • So is polynomial time O(nk) for large k’s in most cases… • We always try to find low complexity algorithms or heuristics, shortcuts that work most of the time but not always… מבוא מורחב - שיעור 4