The environment model evaluator and compiler

1. The env model evaluator Motivation The environment model evaluator and compiler • Saves repeated renaming and substitution: Using new data structures (frame, environment), explicit substitution is replaced by variable bindings. • Can be utilized to gain further improvements that cannot be achieved using the substitution model (coming up next..)

2. The env model evaluator Structure & implementation The environment model evaluator and compiler Abstract syntax parser Core Data structures Special forms Derived expressions • New data structures: • A closure • A binding • A frame • An environment • Additional procedures to handle with these ADTs • Procedure application and • evaluation of expressions is • now done with respect to a • certain environment. • No need for renaming & • substitution procedures. No changes here!

3. + #<primitive:+> The env model evaluator Structure & implementation The environment model evaluator and compiler Data structures package: env A closure: The data structure representing a procedure value includes, in addition to its params and body, also the env in which the procedure has been declared. This env is to be extended when applying the procedure. P:B: A binding: A pair consisting of a variable and its value.

4. 3 #<primitive:+> #<procedure> + x f The env model evaluator Structure & implementation The environment model evaluator and compiler Data structures package: • A frame: • A variable-value mapping. Every variable in a frame is bound to one value only. • Implemented as a pair of lists: a list of variables and a list of values.

5. E2 E1 … E3 x 3 … y 4 + * + Implementation cons * Implementation … cons Implementation … The env model evaluator Structure & implementation The environment model evaluator and compiler Data structures package: • An environment: • A finite sequence of frames in which the last frame is the global environment. • Implemented as a list of frames.

6. The env model evaluator Structure & implementation The environment model evaluator and compiler • “Our” global environment is referenced by a variable in Dr.Scheme! • A finite sequence of frames in which the last frame is the global environment. • Implemented as a list of frames. t-g-e “ Dr.Scheme’s global environment “ + * + Implementation cons * Implementation … cons Implementation …

7. The env model evaluator Structure & implementation The environment model evaluator and compiler env-eval (functional-env-core.scm) (define env-eval (lambda (exp env) (cond ((atomic? exp) (eval-atomic exp env)) ((special-form? exp) (eval-special-form exp env)) ((application? exp) (apply-procedure (env-eval (operator exp) env) (list-of-values (operands exp) env))) (else (error "Unknown expression type -- EVAL" exp)))))

8. The env model evaluator Example 1 The environment model evaluator and compiler > (env-eval '(define f (lambda (x) (lambda (y) (+ x y)))) t-g-e) • ((special-form? exp) (eval-special-form exp env)) (define eval-special-form (lambda (exp env) (cond … ((definition? exp) (if (not (eq? env the-global-environment)) (error "Non global definition" exp) (eval-definition exp))) … • (define eval-definition • (lambda (exp) • (add-binding! (make-binding (definition-variable exp) • (env-eval (definition-value exp) the-global-environment))) • 'ok))

9. The env model evaluator Example 1 The environment model evaluator and compiler > (env-eval '(define f (lambda (x) (lambda (y) (+ x y)))) t-g-e) • ((special-form? exp) (eval-special-form exp env)) (define eval-special-form (lambda (exp env) (cond … ((lambda? exp) (eval-lambda exp env)) … (defineeval-lambda (lambda (exp env) (make-procedure (lambda-parameters exp) (lambda-body exp) env)))

10. The env model evaluator Example 1 The environment model evaluator and compiler > (env-eval '(define f (lambda (x) (lambda (y) (+ x y)))) t-g-e) env (define make-procedure (lambda (parameters body env) (attach-tag (list parameters body env) 'procedure))) Q: Why use list and not just return '(procedure (x) (lambda (y) (+ x y)) t-g-e)? A: If we did, t-g-e wouldhavebeenquoted, thusproducing a symbol instead of the data structurethatrepresentsthe global environmentusedbyourevaluator. P:B:

11. The env model evaluator Example 1 The environment model evaluator and compiler > (env-eval '(define f (lambda (x) (lambda (y) (+ x y)))) t-g-e) • Back to the original expression: Bind f with the created closure in our GE. • (define eval-definition • (lambda (exp) • (add-binding! (make-binding (definition-variable exp) • (env-eval (definition-value exp) the-global-environment))) • 'ok)) (add-binding! (make-binding f (list 'procedure '(x) '(lambda (y) (+ x y)) t-g-e)))

12. The env model evaluator Example 1 The environment model evaluator and compiler > (env-eval '(define f (lambda (x) (lambda (y) (+ x y)))) t-g-e) t-g-e f +Imp. + > t-g-e * • #0=(( (f+ * cons null?) • (procedure (x) ((lambda (y) (+ x y))) #0#) • (primtive #<primitive:+>) • (primitive #<primitive:*>) • (primitive #<primitive:cons>) • (primitive #<primitive:null?>) )) cons … *Imp. … consImp. • The resulting structure is circular. 'procedure x 'lambda 'y '+ 'x 'y

13. The env model evaluator Example 2 The environment model evaluator and compiler > (env-eval ‘f t-g-e) (define eval-atomic (lambda (exp env) (if (or (number? exp) (boolean? exp) (null? exp)) exp (lookup-variable-value exp env)))) • We look up f in the given env. • The result is our representation of a procedure: a tagged list storing the body, the params and the env where the procedure was defined. • The resulting structure is circular. #0=(procedure (x) ((lambda (y) (+ x y))) (((f + * cons null?) #0# (primitive #<primitive:+>) (primitive #<primitive:*>) (primitive #<primitive:cons>) (primitive #<primitive:null?>))))

14. The env model evaluator Example 3 The environment model evaluator and compiler > (env-eval '(define g (f 1) ) t-g-e) ((application? exp) (apply-procedure (env-eval (operator exp) env) (list-of-values (operands exp) env))) (apply-procedure (procedure (x) ((lambda (y) (+ x y))) t-g-e) (1)) Extend the env in which the procedure was defined with a new frame. Within this frame, bind x to 1. (#2) GE (#3) • (procedure • #0=(y) • #1=((+ x y)) • ( (#2=(x) 1) • . #3=(((f + * cons null?) • (procedure #2# ((lambda #0# . #1#)) #3#) • (primitive #<primitive:+>) • (primitive #<primitive:*>) • (primitive #<primitive:cons>) • (pimitive #<primitive:null?>))) )) x: 1

15. The env model evaluator Example 3 The environment model evaluator and compiler t-g-e E1 g f + x 1 * +Imp. … *Imp. consImp. … 'procedure 'procedure ‘y ‘x 'lambda '+ 'x 'y 'y '+ 'x 'y

16. The env model evaluator Adding a let* exp The environment model evaluator and compiler • What is a let* expression? • Similar to a let, only that every defined variable can be used in the following definitions. • Can be regarded as a nested let expression. • The number of variables corresponds to the number of nested let expressions. >(definea 10) Supporting a let* expression >(let* ((a 1) (c (* a 2))) • (+ a c)) 3 >(let ((a 1)) (let ((c (* a 2))) (+ a c))) >((λ (a) • ((λ (c) (+ a c)) • (* a 2)) • 1) >(let ((a 1) (c (* a 2))) (+ a c)) 21

17. The env model evaluator Adding a let* expression The environment model evaluator and compiler • Evaluation rule for let* expressions: The evaluation rule for an expression (let* ((v1 e1) (v2 e2) … (vn en)) b1 b2 … bm), with respect to an environment E0, is defined as follows: E1 E0 * make-frame [(v1),(env-eval e1 E0)] E2 E1 * make-frame [(v2),(env-eval e2 E1)] ... EnEn-1* make-frame [(vn),(env-eval enEn-1)] (env-eval b1 En) ... (env-eval bm-1 En) Return (env-evalbm En) • Q: How would the evaluation rule for let look like? • A: There will be only one frame extending E0 and creating a new environment E1. All of the expressions e1…en will be evaluated with respect to E0.

18. The env model evaluator Adding let* as a special form Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler • 1. Add the required ADT procedures to the ASP ASP Data Structures(+GE) Derived expressions Core (let* ((v1 e1) (v2 e2) … (vn en)) b1 b2 … bm) Special forms (define (let*-first-variable exp) (caar (let*-bindings exp))) (define (let*? exp) (tagged-list? exp 'let*)) (define let*-bindingscadr) (define (let*-first-initial-value exp) (cadar (let*-bindings exp))) (define let*-bodycddr) (define (make-let*-bindingsvarsvals) (map list varsvals)) (define (let*-variables exp) (map car (let*-bindings exp))) (define (let*-initial-values exp) (map cadr (let*-bindings exp))) (define (make-let* bindings body) (append (list 'let* bindings) body))

19. Supporting a let* expression: As a special form The env model evaluator Adding let* as a special form The environment model evaluator and compiler The environment model evaluator and compiler • 2. Add the required changes to identify a let* expression as a special form ASP Data Structures(+GE) Derived expressions Core (let* ((v1 e1) (v2 e2) … (vn en)) b1 b2 … bm) Special forms (define (special-form? exp) (or (quoted? exp) (lambda? exp) (definition? exp) (if? exp) (begin? exp) (let*? exp))) (define (eval-special-form exp env) (cond ((quoted? exp) (text-of-quotation exp)) ((lambda? exp) (eval-lambda expenv)) … ((let*? exp) (eval-let* exp env)))) • How would eval-let* work?

20. Supporting a let* expression: As a special form The env model evaluator Adding let* as a special form The environment model evaluator and compiler The environment model evaluator and compiler • 3. Add the required changes to evaluate a let* expression (let* ((v1 e1) (v2 e2) … (vn en)) b1 b2 … bm) Option A: (define (eval-let* exp env) (let ((body (let*-body exp))) (if (null? (let*-bindings exp)) (eval-sequence body env) (let ( (first-var (let*-first-variable exp)) (first-val (let*-first-initial-value exp)) (rest-bindings (let*-rest-bindings exp)) ) (eval-let* (make-let* rest-bindings body) (extend-env(make-frame (list first-var) (list (env-eval first-valenv)) ) env )))))) Create a new, smaller let* expression and recursively evaluate it with respect to an extended env.

21. Supporting a let* expression: As a special form The env model evaluator Adding let* as a special form The environment model evaluator and compiler The environment model evaluator and compiler • 3. Add the required changes to evaluate a let* expression (let* ((v1 e1) (v2 e2) … (vn en)) b1 b2 … bm) Option B: (define (eval-let* exp env) (let ( (vars (let*-variables exp)) (vals (let*-initial-values exp)) (body (let*-body exp)) ) (letrec ((helper (lambda (vars-lstvals-lstenv) (if (null? vars-lst) (eval-sequence body env) (helper (cdrvars-lst) (cdrvals-lst) (extend-env(make-frame (list (car vars-lst)) (list (env-eval (car vals-lst) env))) env)))))) (helper varsvalsenv)))) Keep extending the given env with new frames. When vars-lst becomes empty, No more extensions are needed. It only remain to evaluate the body. • Q: How is option B different from option A? • A: In option A we create code in run-time (when calling make-let*).

22. Supporting a let* expression: As a special form The env model evaluator Adding let* as a derived exp The environment model evaluator and compiler The environment model evaluator and compiler • 1. Add the required ADT procedures to the ASP: Same as earlier! ASP Data Structures(+GE) Derived expressions Core • 2. Add the required changes to identify a let* expression as a derived expression Special forms (define (derived? exp) (or (cond? exp) (function-definition? exp) (let? exp) (let*? exp))) • 3. Support a let* expression in the shallow-derive procedure (define (shallow-derive exp) (cond ((cond? exp) (cond->if exp)) … ((let*? exp) (let*->nested-let exp)) (else "Unhandled derivision" exp))) • How will let*->nested-let work?

23. Supporting a let* expression: As a special form The env model evaluator Adding let* as a derived exp The environment model evaluator and compiler The environment model evaluator and compiler • 4. Provide with a procedure to derive a let* expression (define (let*->nested-let exp) (let ( (bindings (let*-bindings exp)) (body (let*-body exp)) ) (if (null? (cdr bindings)) (derive (make-let bindings body)) (derive (make-let (make-let-bindings (list (let*-first-variable exp)) (list (let*-first-initial-value exp))) (list (let*->nested-let (make-let* (cdr bindings) body)))))))) • Q1: Why do we use derive in the body of let*->nested-let? • A1: We derive let* into nested let expressions, but let is a derived expression itself. We must derive it so the evaluator can handle it. • Q2: In the alternative clause, we enclose the body parameter for make-let in a list. Why? • A2: Since the body may include a sequence of expressions, make-let expects a list. let*->nested-let returns a single expression, so we must enclose it within a list to be given as an argument to make-let.

24. The env model compilerWhat is it good for? Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler • Env-eval combines expression analysis and evaluation in the same process. • This leads to inefficiency due to repeated analysis (repeated calls for ASP procedures). • We avoid this by separating analysis from evaluation. • This could not be achieved in the substitution model, where every application involves • substitution and analysis. • We analyze our code, producing a procedure that is ready for execution, given an environment argument. The procedure returns the evaluation of the analyzed code with respect to the given environment. No analysis is performed during the evaluation. (define analyze (lambda (exp) (lambda (env) <analyzed -body>))) (define env-eval (lambda (exp env) <body>))

25. The env model compilerWhat is it good for? Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler • Interpreter: • Each time fact is called, the interpreter repeatedly uses ASP procedures to identify the body as an if expression and to get the required components for evaluation. (define fact (lambda (n) (if (= n 0) 1 (* n (fact (- n 1)))))) env-eval:Exp*Env->Scheme-Type • Compiler: • To avoids repeated analysis, Takes an exp, performs analysis in static time and returns • a procedure that awaits an environment argument. • Once the procedure is applied, it evaluates the analyzed exp with respect to the given • environment. No further analysis is performed. compiler:Exp->[Env->Scheme-Type]

26. The env model compilerExample Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler (define (analyze exp) (cond ((atomic? exp) (analyze-atomic exp)) ((special-form? exp) (analyze-special-form exp)) ((application? exp) (analyze-application exp)) (else (error "Unknown expression type -- EVAL“ exp)))) (define (eval-if exp env) (if (true? (env-eval (if-predicate exp) env)) (env-eval (if-consequent exp) env) (env-eval (if-alternative exp) env))) Evaluator code for if-expression (define (analyze-if exp) (let ( (pred (analyze (if-predicate exp))) (consequent (analyze (if-consequent exp))) (alternative (analyze (if-alternative exp)))) (lambda (env) (if (true? (predenv)) (consequent env) (alternative env))))) Compiler code for if-expression The analysis of an if expression is performed only once. It saves us the calls for ASP procedure for the evaluation during run-time. The returned procedure performs evaluation only.

27. The env model compilerAdding a let* expression Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler • What should be done to add let* as a derived expression? • Same thing we did with the evaluator! • How would we support a let* expression as a special form? • 1. Add the required ADT procedures to the ASP • 2. identify a let* expression as a special form • 3. Support a let* expression in the analyze-special-form procedure: (define (analyze-special-form exp) (cond ((quoted? exp) (analyze-quoted exp)) ((lambda? exp) (analyze-lambda exp)) … ((let*? exp) (analyze-let* exp)))) • How will analyze-let*work?

28. The env model compilerAdding a let* expression Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler • 4. Provide with a procedure to analyze a let* expression: • Take an evaluator procedure that evaluates a let* expression. • Curry the env parameter. • inductively apply analyze to all sub expressions. Option A: What’s wrong with it? (define (eval-let* exp env) (let ((body (let*-body exp))) (if (null? (let*-bindings exp)) (eval-sequence body env) (let ( (first-var (let*-first-variable exp)) (first-val (let*-first-initial-value exp)) (rest-bindings (let*-rest-bindings exp)) ) (eval-let* (make-let* rest-bindings body) (extend-env(make-frame (list first-var) (list (env-eval first-valenv)) ) env )))))) • It produces code on run-time, which we cannot analyze. Analysis is performed ONLY on static time, before evaluation!

29. The env model compilerAdding a let* expression Supporting a let* expression: As a special form The environment model evaluator and compiler The environment model evaluator and compiler • The second version for eval-let* (option B) translated into analyze-let*: (define (analyze-let* exp) (let ( (vars (let*-variables exp)) (vals (map analyze (let*-initial-values exp))) (body (analyze-sequence (let*-body exp))) ) (letrec ((helper (lambda (vars-lstvals-lstenv) (if (null? vars-lst) (body env) (helper (cdrvars-lst) (cdrvals-lst) (extend-env (make-frame (list (car vars-lst)) (list ((car vals-lst) env))) env)))))) (lambda (env) ;letrc body– carrying (helper varsvalsenv))))) Note: If we were to support let* as a derived expression, we would have used derive-analyze-eval