Lecture 3

1. Lecture 3 Material in the textbook Sections 1.1.5 to 1.2.1 מבוא מורחב

2. Today Continue Lecture 2 (sqrt) Refine our model Applicative vs. Normal Order Understand how it captures the nature of processes Recursive vs. Iterative processes Orders of growth מבוא מורחב 2

3. X = 2 G = 1 X/G = 2 G = ½ (1+ 2) = 1.5 X/G = 4/3 G = ½ (3/2 + 4/3) = 17/12 = 1.416666 X/G = 24/17 G = ½ (17/12 + 24/17) = 577/408 = 1.4142156 Computing SQRT: A Numeric Algorithm • To find an approximation of square root of x, use the following recipe: • Make a guess G • Improve the guess by averaging G and x/G • Keep improving the guess until it is good enough

4. (define initial-guess 1.0) (define precision 0.0001) (define (sqrt-iter guess x) (if (good-enough? guess x) guess (sqrt-iter (improve guess x) x))) (define (good-enough? guess x) (< (abs (- (square guess) x)) precision)) (define (improve guess x) (average guess (/ x guess))) (define (sqrt x) (sqrt-iter initial-guess x))

5. Good programming Style • 1. Divide the task to well-defined, natural, and • simple sub-tasks. • E.g: good-enough? and improve. • Rule of thumb : If you can easily name it, it does a well-defined task. 2. Use parameters. E.g.: precision, initial-guess. 3. Use meaningful names.

6. Procedural abstraction • It is better to: • Export only what is needed • Hide internal details. The procedure SQRT is of interest for the user. The procedure improve-guessis an internal detail. • Exporting only what is needed leads to: • A clear interface • Avoids confusion

7. Rewriting SQRT (Block structure) (define (sqrt x) (define (sqrt-iter guess x) (if (good-enough? guess x) guess (sqrt-iter (improve guess x) x))) (define (good-enough? guess x) (< (abs (- (square guess) x)) precision)) (define (improve guess x) (average guess (/ x guess))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess x))

8. Further improving sqrt Note that in every application of sqrt we substitute for x the same value in all subsequent applications of compound procedures ! Therefore wedo not have to explicitly pass x as a formal variable to all procedures. Instead, can leave it unbounded (“free”).

9. SQRT again, taking advantage of the refined substitution model (define (sqrt x) (define (good-enough? guess) (< (abs (- (square guess) x)) precision)) (define (improve guess) (average guess (/ x guess))) (define (sqrt-iter guess) (if (good-enough? guess) guess (sqrt-iter (improve guess)))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess))

10. SQRT (cont.) ==>(sqrt 2) (define (good-enough? guess) (< (abs (- (square guess) 2)) precision)) (define (improve guess) (average guess (/ 2 guess))) (define (sqrt-iter guess) (if (good-enough? guess) guess (sqrt-iter (improve guess)))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess))

11. Lexical Scoping - again The lexical scoping rules means that the value of a variable which is unbounded (free) in a procedure f is taken from the procedure in which f was defined. It is also called static scoping

12. Proc1.x Proc3.x Another example for lexical scope (define (proc1 x) (define (proc2 y) (+ x y)) (define (proc3 x) (proc2 x)) (proc3 (* 2 x))) (proc1 4) proc1.x = 4 (proc3 8) proc3.x = 8 (proc2 8) proc2.y = 8 proc2.x=proc1.x=4 12

13. Applicative order vs. Normal OrderEvaluation To Evaluate a combination: (other than special form) Evaluate all of the sub-expressions in any order Applythe procedure that is the value of the leftmost sub-expression to the arguments (the values of the other sub-expressions) מבוא מורחב

14. Applicative order evaluation rules Combination... (<operator> <operand1> …… <operand n>) • Evaluate <operator> to get the procedure and evaluate <operands> to get the arguments • If <operator> is primitive: do whatever magic it does • If <operator> is compound: evaluate body with formal parameters replaced by arguments מבוא מורחב

15. Normal order evaluation Combination … (<operator> <operand1> …… <operand n>) • Evaluate <operator> to get the procedure and evaluate <operands> to get the arguments • If <operator> is primitive: do whatever magic it does • If <operator> is compound: evaluate body with formal parameters replaced by arguments מבוא מורחב

16. The Difference Normal ((lambda (x) (+ x x)) (* 3 4)) (+ (* 3 4) (* 3 4)) (+ 12 12) 24 Applicative ((lambda (x) (+ x x)) (* 3 4)) (+ 12 12) 24 This may matter in some cases: ((lambda (x y) (+ x 2)) 3 (/ 1 0)) Scheme is an Applicative Order Language! מבוא מורחב

17. Compute ab (Recursive Approach) • wishful thinking : • base case: ab=a * a(b-1) a0 = 1 • (define exp-1 • (lambda (a b) • (if (= b 0) • 1 • (* a (exp-1 a (- b 1)))))) מבוא מורחב

18. ab=a * a * a*…*a • Which is: b ab = a2 *a*…*a= a3 *…*a • Operationally: • Halting condition: product  product * a counter  counter - 1 counter = 0 Compute ab (Iterative Approach) • Another approach: מבוא מורחב

19. Syntactic Recursion Compute ab (Iterative Approach) (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1)) How then, do the two procedures differ? They give rise to different processes – lets use our model to understand how. מבוא מורחב

20. Recursive Process • (define exp-1 • (lambda (a b) • (if (= b 0) • 1 • (* a (exp-1 a (- b 1)))))) • (exp-1 3 4) • (* 3 (exp-1 3 3)) • (* 3 (* 3 (exp-1 3 2))) • (* 3 (* 3 (* 3 (exp-1 3 1)))) • (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) • (* 3 (* 3 (* 3 (* 3 1)))) • (* 3 (* 3 (* 3 3))) • (* 3 (* 3 9)) • (* 3 27) • 81 מבוא מורחב

21. Iterative Process (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1)) (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 מבוא מורחב

22. (exp-1 3 4) • (* 3 (exp-1 3 3)) • (* 3 (* 3 (exp-1 3 2))) • (* 3 (* 3 (* 3 (exp-1 3 1)))) • (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) • (* 3 (* 3 (* 3 (* 3 1)))) • (* 3 (* 3 (* 3 3))) • (* 3 (* 3 9)) • (* 3 27) • 81 (exp-2 3 4) Growing amount of space (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) Constant amount of space (exp-iter 3 1 27) (exp-iter 3 0 81) 81 The Difference מבוא מורחב

23. operation pending no pending operations Why More Space? • Recursive exponentiation: (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1))))) • Iterative exponentiation: • (define (exp-2 a b) • (define (exp-iter a counter product) • (if (= counter 0) • product • (exp-iter a (- counter 1) (* a product)))) • (exp-iter a b 1))

24. Example: Factorial wishful thinking : base case: n! =n * (n-1)! n = 1 (define fact (lambda (n) (if (= n 1) 1 (* n (fact (- n 1)))))) Iterative or Recursive? מבוא מורחב 24

25. Summary • Recursive process num of deferred operations “grows proportional to b” • Iterative process num of deferred operations stays “constant” (actually it’s zero) Can we better quantify these observations? Orders of growth… מבוא מורחב

26. Order of Growth: Recursive Process • (exp-1 3 4) • (* 3 (exp-1 3 3)) • (* 3 (* 3 (exp-1 3 2))) • (* 3 (* 3 (* 3 (exp-1 3 1)))) • (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) • (* 3 (* 3 (* 3 (* 3 1)))) • (* 3 (* 3 (* 3 3))) • (* 3 (* 3 9)) • (* 3 27) • 81  4 • (exp-1 3 5) • (* 3 (exp-1 3 4)) • (* 3 (* 3 (exp-1 3 3))) • (* 3 (* 3 (* 3 (exp-1 3 2)))) Dependent on b • (* 3 (* 3 (* 3 (* 3 (exp-1 3 1))))) • (* 3 (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) • (* 3 (* 3 (* 3 (* 3 (* 3 1)))) • (* 3 (* 3 (* 3 (* 3 3)))) • (* 3 (* 3 (* 3 9))) • (* 3 (* 3 27)) • (* 3 81)  5 • 243

27. Iterative Process (define (exp-2 a b) (define (exp-iter a b product) (if (= b 0) product (exp-iter a (- b 1) (* a product)))) (exp-iter a b 1) (exp-2 3 4) (exp-2 3 5) (exp-iter 3 4 1) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 2 27) (exp-iter 3 0 81) (exp-iter 3 0 81) (exp-iter 3 0 243) 81 243 Some constant, independent of b

28. The worst-case over all inputs of size n Orders of Growth • Suppose n is a parameter that measures the size of a problem (the size of its input) • R(n)measures the amount of resources needed to compute a solution procedure of size n. • Two common resources are space, measured by the number of deferred operations, and time, measured by the number of primitive steps. מבוא מורחב

29. Orders of Growth • Want to estimate the “order of growth” of R(n): R1(n)=100n2 R2(n)=2n2+10n+2 R3(n) = n2 Are all the same in the sense that if we multiply the input by a factor of 2, the resource consumption increases by a factor of 4 Order of growth is proportional to n2 מבוא מורחב

30. Orders of Growth • We say R(n)has order of growth Q(f(n))if there are constants c1 0and c2 0 such that for all n c1f(n)<= R(n)<= c2f(n) • R(n)O(f(n)) if there is a constant c  0such that for all n • R(n) <= cf(n) • R(n)(f(n)) if there is a constant c  0such that for all n • cf(n)<= R(n) מבוא מורחב

31. Orders of Growth True or False? f 100n2 O(n) t 100n2 (n) 100n2 Q(n) f 100n2 Q(n2) t True or False? f 2100 (n) t 2100n  O(n2) 2n  Q(n) f 210 Q(1) t מבוא מורחב

32. (exp-1 3 4) “n”=b=4 • (* 3 (exp-1 3 3)) • (* 3 (* 3 (exp-1 3 2))) • (* 3 (* 3 (* 3 (exp-1 3 1)))) • (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) • (* 3 (* 3 (* 3 (* 3 1)))) • (* 3 (* 3 (* 3 3))) • (* 3 (* 3 9)) • (* 3 27) • 81 Resources Consumed by EXP-1 • Space b <= R(b) <= b which is Q(b) • Time b <= R(b) <= 2b which is Q(b) Linear Recursive Process מבוא מורחב

33. (exp-2 3 4) “n”=b=4 (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 Resources Consumed by EXP-2 • Space Q(1) • Time Q(b) Linear Iterative Process מבוא מורחב

34. Summary • Trying to capture the nature of processes • Quantify various properties of processes: • Number of steps a process takes (Time Complexity) • Amount of Space a process uses (Space Complexity) מבוא מורחב