A Theory of Hygienic Macros

1. A Theory ofHygienic Macros PhD Thesis Proposal David Herman

2. The power of macros • Derived (user-defined) syntactic constructs • Defined by rewriting to existing constructs • Translated at compile-time (define-macro (increment! a) (set! a (+ a 1)))

3. local bindings, software contracts, custom loop forms, derived datatypes, first-class module systems, object-orientation, test harnesses, data constructors, laziness, short-circuit boolean operators, iteration protocols, coroutines, exceptions, functions, recursion, iteration, primitive datatypes threads, type checkers, static analyses, parser generators, partial evaluation, debuggers, introspection, etc… The power of macros • Abstractions for scope, control, program structure • Powerful meta-programming idioms • Allow for a minimal language core language user language designer

4. The need for hygiene (define-macro (or e1 e2) (with ([t e1]) (if t t e2))) (with ([x 1]) (or #f x)) => (with ([x 1]) (with ([t #f]) (if t t x))) (with ([t 1]) (or #f t)) => (with ([t 1]) (with ([t #f]) (if t t t)))

5. The need for hygiene (define-macro (or e1 e2) (with ([t e1]) (if t t e2))) (with ([x 1]) (or #f x)) => (with ([x 1]) (with ([t #f]) (if t t x))) (with ([t 1]) (or #f t)) => (with ([t 1]) (with ([t′ #f]) (if t′ t′ t)))

6. What are hygienic macros?

7. What are hygienic macros? (define-macro (or e1 e2) (with ([t e1]) (if t t e2))) (with ([x 1]) (or #f x)) => (with ([x 1]) (with ([t #f]) (if t t x))) (with ([t 1]) (or #f t)) => (with ([t 1]) (with ([t #f]) (if t t t)))

8. What are hygienic macros?

9. What are hygienic macros? KFFD ’86: Generated identifiers that become binding instances in the completely expanded program must only bind variables that are generated at the same transcription step. i.e., not provided as the macro’s input i.e., a single rewriting of a macro application

10. What are hygienic macros? Clinger and Rees ’91: • It is impossible to write a macro that inserts a binding that can capture references other than those inserted by the macro. • It is impossible to write a macro that inserts a reference that can be captured by bindings other than those inserted by the macro. again: not provided as the macro’s input

11. Why isn’t this enough?

12. Pathological macros in Scheme Impossible with hygienic macros, right? (loop (begin (when (prime? i) (break i)) (increment! i)))

13. Pathological macros in Scheme Contrast with the explicit version: (loop/explicit break (begin (when (prime? i) (break i)) (increment! i))) provided as input

14. Pathological macros in Scheme Petrofsky extraction: (loop (begin (when (prime? i) (break i)) (increment! i)))

15. Pathological macros in Scheme Dumpster diving: (loop (begin (when (prime? i) (break i)) (increment! i)))

16. Pathological macros in Scheme Dumpster diving: (loop/explicit break (begin (when (prime? i) (break i)) (increment! i)))

17. Pathological macros in Scheme (define-macro (murky a e) (begin (set! a e) (lambda (a) e))) (let ([foo 10]) (murky foo (+ foo 1)))

18. Pathological macros in Scheme (define-macro (indecisive ([a e]) body) (if-zero …complicated computation… ((lambda (a) body) e) ((lambda (a) e) body)))

19. What do we know about hygiene?

20. Hygiene and lexical scope (define-macro (or e1 e2) (with ([t e1]) (if t t e2))) (with ([x 1]) (or #f x)) => (with ([x 1]) (with ([t #f]) (if t t x))) (with ([t 1]) (or #f t)) => (with ([t 1]) (with ([t′ #f]) (if t′ t′ t))) =

21. The essence of hygienic macros Hygienic macro expansion isinsensitive to -conversion. In other words,hygienic = respects !

22. What is the scope of with? (define-macro (with ([a e1]) e2) …) First try: look at results of expansion (with ([x 1]) (or #f x)) => ((lambda (x) ((lambda (t) (if t t x)) #f) 1) (with ([t 1]) (or #f t)) => ((lambda (t) ((lambda (t′) (if t′ t′ t)) #f) 1)

23. Using expansion to define scope • Goal: define hygienic macro expansion in terms of -equivalence. • Strategy: define -equivalence in terms of hygienic macro expansion. Oops.

24. What is the scope of with? (define-macro (with ([a e1]) e2) …) Better idea: provide a binding specification with consumes an argument of shape(with ([a e1]) e2) and produces an expression, where: • a is an identifier • e1 and e2 are expressions • a is bound in e2

25. What is the scope of with? (define-macro (with ([a e1]) e2) …) Better idea: provide a binding specification with : (with ([<a> expr]) expra) →expr

26. Annotated with macro (define-macro (with ([a e1]) e2) : expr [(a : binder) (e1 : expr) (e2 : expr a)] …) • States macro writer’s intention about scope • Admits rigorous -equivalence definition • Provides correctness criterion for hygienicmacro expansion

27. Thesis The key to specifying and verifying hygienic macro expansion is specifying the binding structure of macros.

28. A Theory of Hygienic Macros m, a model of hygienic macros • Macro binding specifications (types) • Theory of -equivalence • High-level expansion semantics • Definition and proof of hygiene

29. A Theory of Hygienic Macros m, a model of hygienic macros • Macro binding specifications (types) • Theory of -equivalence • High-level expansion semantics • Definition and proof of hygiene

30. (with ([x 1]) x) (x.x) 1 m: a model of hygienic macros S-expressions -calculus

31. m: a model of hygienic macros →(with ([x 1]) x) →((lambda (x) x) 1) →((x.x) 1) → (x.x) 1

32. m: a model of hygienic macros let syntax fun = macro ((a)e):((<a>)expra) => a.e with = macro (([ae])body):(([<a> expr])expra) => ((a.body) e) in (with ([f(fun (x) 42)]) (fy.y)) end

33. A Theory of Hygienic Macros m, a model of hygienic macros • Macro binding specifications (types) • Theory of -equivalence • High-level expansion semantics • Definition and proof of hygiene

34. 1. Macro binding specifications • Shape types based on Culpepper and Felleisen ’03: with : (with ([idexpr]) expr) →expr • Types for reasoning about scope inspired by nominal datatypes and nominal logic of Gabbay and Pitts ’01

35. 1. Type checking let syntax m = macro p :  => … in x.e end [m: → expr]⊢ x.e:expr

36. 1. Type checking let syntax m = macro p :  => … in x.e end [m:→ expr, x:expr]⊢ e:expr

37. 1. Meta-level scope let syntax m = macro (e1 e2) : (expr expr) => t.(e1 e2) in … end [t:expr], [e1:expr, e2:expr]⊢ e:expr

38. , ⊢ s :  1. Two dimensions of binding • : program bindings (object-level) • : macro pattern bindings (meta-level)

39. A Theory of Hygienic Macros m, a model of hygienic macros • Macro binding specifications (types) • Theory of -equivalence • High-level expansion semantics • Definition and proof of hygiene

40. 2. Theory of -equivalence • Adapted to macro applications by exploiting shape type annotations • Relies on novel shape-directed -conversion • Based on Gabbay/Pitts “swapping” formulation of-equivalence

41. binding occurrence 2. -equivalence via swapping let val x = x in x (fn x => x) end = let val y = x in y (fn w => w) end

42. free 2. -equivalence via swapping let val z = x in x (fn x => x) end = let val z = x in y (fn w => w) end

43. 2. -equivalence via swapping let val z = x in x (fn x => x) end = let val z = x in y (fn w => w) end

44. 2. -equivalence via swapping let val z = x in z (fn z => z) end = let val z = x in z (fn w => w) end

45. 2. -equivalence via swapping let val z = x in z (fn z′ => z′) end = let val z = x in z (fn z′ => z′) end

46. 2. -equivalence via swapping • For each binder x, pick fresh name z • Rename the binding occurrence of x • In the body, swap all occurrences of x with z • Recur on subterms • Compare results syntactically

47. 2. -equivalence for m (with ([x x]) (x (fun (x) x))) = (with ([y x]) (y (fun (w) w))) ?

48. 2.Shape-directed -conversion (with ([x x]) (x (fun (x) x))) (with ([<a> expr]) expra) (with ([y x]) (y (fun (w) w))) binding occurrence

49. 2.Shape-directed -conversion (with ([z x]) (x (fun (x) x))) (with ([<a> expr]) expra) (with ([z x]) (y (fun (w) w))) not in scope of a; not converted

50. 2.Shape-directed -conversion (with ([zx]) (x (fun (x) x))) (with ([<a> expr]) expra) (with ([zx]) (y (fun (w) w))) entering scope of a; swap all occurrences of x / y with z