1 / 46

Banana Algebra:

Banana Algebra:. Syntactic Language Extension via an Algebra of Languages and Transformations. Jacob Andersen [ jacand@cs.au.dk ] Aarhus University. Claus Brabrand [ brabrand@itu.dk ] IT University of Copenhagen. The Banana Algebra Café. The " Banana Algebra Café " :

conan-trevino
Download Presentation

Banana Algebra:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Banana Algebra: Syntactic Language Extension via an Algebra of Languages and Transformations Jacob Andersen [ jacand@cs.au.dk ] Aarhus University Claus Brabrand [ brabrand@itu.dk ] IT University of Copenhagen

  2. The Banana Algebra Café • The "Banana Algebra Café": • Located in Costa Rica • City of Cahuita (pop. 3,000): Banana Breakfast! Cahuita!

  3. Outline • Introduction: "What is a Banana?" • Bananas for Language Transformation • Language Extension Pattern • Banana Algebra • Examples • Implementation • Related Work • Conclusion

  4. What is a 'Banana' ? • Datatype; "list": • Banana("sum-of-list"): • Implicit recursion on input structure • Separation of recursion and evaluation • bottom-up re-combination of intermediate results list = Num int | Cons int * list listint [Num n]=n [Cons nl]=n+[l] (aka. "Catamorphism" ) (|n.n, (n,l ).n+l|) Another Ex.: "length-of-list"

  5. Language Transformation • Bananas (statically typed): • Source language: 'LS' • Target language: 'LT' • Nonterminal-typing: '' • Reconstructors: 'c'  (| LS -> LT [] c |) LS -> LT list = Num int | Cons int * list tree = Nil | Leaf int | Node tree * tree [list -> tree] [Num n]= Leaf n [Cons nl]= Node (Leaf n) [l] Type-check'able!

  6. "Growing Languages with Metamorphic Syntax Macros"[ Claus Brabrand | Michael Schwartzbach ] ( PEPM 2002 ) "The metafront System: Safe and Extensible Parsing and Transformation"[ Claus Brabrand | Michael Schwartzbach ] ( LDTA 2003, SCP J. 2007 ) Statically reduce: Banana Algebra (term) Banana (const) Banana Properties • Banana properties: • Simple(corresponds to: “simple recursion”) • Safe(syntactically safe + always terminate) • Efficient(linear time in size of input + output) • (Expressive)(…enough for interesting extensions) • Banana Algebra “for free” (16 banana ops): • Modular • Incremental • Simple • Safe • Efficient • (Expressive) Been around for many years We now propose 

  7. Outline • Introduction: "What is a Banana?" • Bananas for Language Transformation • Language Extension Pattern • Banana Algebra • Examples • Implementation • Related Work • Conclusion

  8. Language Extension Pattern Numeral extension: Lambda-Calculus: 'LS' 'LT' Exp : var Id : lam Id * Exp : app Exp * Exp : zero : succ Exp : pred Exp Exp : var Id : lam Id * Exp : app Exp * Exp '' Nonterminal typing: [Exp -> Exp] Reconstructors: 'c' [var V] = var V [lam VE] = lam V[E] [app E1E2] = app [E1][E2] [zero] = lam z (var z) [succ E] = lam s [E] [pred E] = app [E] (lam z (var z)) Catamorphism: (| LS -> LT [] c |) Using very simple numeral encoding

  9. Algebraic Solution ln+ll + lnl (| ln -> l[Exp -> Exp] [zero] = lam z (var z) [succ E] = lam s [E] [pred E] = app [E] ... |) ll idx l ln Exp : var Id : lam Id * Exp : app Exp * Exp Exp : zero : succ Exp : pred Exp

  10. Extending Java java+repeatjava + repeatjava javajava (| repeat -> java[Stm -> Stm, Exp -> Exp] [repeat S E] = do-while [S] (not [E]) |) idx java repeat Java grammar:( 575 lines !!! ) ... Stm : "repeat" Stm "until" "(" Exp ")" ";"

  11. Languages (L): l v L\L L+L src( X ) tgt( X ) letv =LinL letxw =XinL Transformations (X): x w X\L X+X XX idx( L ) letv =LinX letxw =XinX Banana Algebra (|L -> L[] c |) { CFG }

  12. Algebraic Laws • Idempotency of '+': • Commutativity of '+': • Associativity of '+': • Source-identity: • … LL + L L1 + L2L2 + L1 L1 + (L2 + L3)  (L1 + L2) + L3 Target-identity: Ltgt(idx(L)) Lsrc(idx(L))

  13. Outline • Introduction: "What is a Banana?" • Bananas for Language Transformation • Language Extension Pattern • Banana Algebra • Examples • Implementation • Related Work • Conclusion

  14. Example Revisited --- "ln2l.x" --- letl = "l.l" inletln = "ln.l" in idx(l) + (| ln -> l[Exp -> Exp] Exp.zero = '\z.z' ; Exp.succ = '\s.$1' ; Exp.pred = '($1 \z.z)' ; |) --- "l.l" --- --- "ln.l" --- { Id = [a-z] [a-z0-9]* ; Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ; } { Exp.zero : "zero" ; Exp.succ : "succ" "(" Exp ")" ; Exp.pred : "pred" "(" Exp ")" ; }

  15. Numerals + Booleans …with Nums & Bools? l+ln+lbl + …with Nums …with Bools lb+ll ln+ll + + ll lbl lnl idx idx l lb ln l

  16. Java + Repeat --- "java.l" --- 575 lines { Java ... "try" Stm "catch" ... Name.id : Id ; } --- "repeat.l" --- { Stm.repeat : "repeat" Stm "until" "(" Exp ")" ";" ; } --- "repeat2java.x" --- letjava = "java.l" inletrepeat = "repeat.l" in idx(java) + (| repeat -> java[Exp -> Exp, Stm -> Stm] Stm.repeat = 'do $1 while (!($2));' ; |) 7 lines !

  17. Concrete vs. Abstract Syntax Concrete syntax: Stm.repeat = 'do $1 while (!($2));' ; Exp (with explicit assoc./prec.): Abstract syntax: Stm.repeat = Stm.do(<1>, Exp.exp1( Exp1.exp2( Exp2.exp3( Exp3.exp4( Exp4.exp5( Exp5.exp6( Exp6.exp7( Exp7.neg( Exp8.par(<2>) ))))))))) ; Exp.or : Exp1 "||" Exp ; .exp1 : Exp1 ; Exp1.and : Exp2 "&&" Exp1 ; .exp2 : Exp2 ; Exp2.add : Exp3 "+" Exp2 ; .exp3 : Exp3 ;  Exp7.neg : "!" Exp8 ; .exp8 : Exp8 ; Exp8.par : "(" Exp ")" ; .var : Id ; .num : IntConst ; (unambiguous: concrete  abstract) NB: Tool supportsBOTH !

  18. "FUN" Example The "FUN" Language: used for Teaching Functional Programming (at Aarhus University) Fun Basically The Lambda Calculus with…: numerals, booleans, arithmetic, boolean logic, local definitions, pairs, literals,lists, signs, comparisons, dynamic types, fixed-point combinators, … Fun grammar transform Literals Literals→Nums Unsigned arithmetic + booleans + definitions + pairs Nums→λ Bools→λ Defs→λ Pairs→λ + + + Lambda Calculus

  19. "FUN" Example Component re-use Fun Fun + FunSigned Fun grammar transform Fun grammar transform + FunSigned GT Literals Literals→Nums Literals→Nums Signed arith→Nums Unsigned arithmetic + booleans + definitions + pairs Nums→λ Bools→λ Defs→λ Pairs→λ + + + Lambda Calculus

  20. "FUN" Example Fun + FunSigned + FunCompare + FunTypesafe Fun GT + FunSigned GT + FunCompare GT + FunTypesafe GT 245x Banana Algebra ops 4 MB Banana ! Unsigned arithmetic + booleans + definitions + pairs Nums→λ Bools→λ Defs→λ Pairs→λ + + + Lambda Calculus

  21. "FUN" Usage Statistics • Usage statistics (245x operators) in "FUN": • 58x { …cfg… }Constant languages • 51x "file.l"Language inclusions • 28x L + LLanguage additions • 23x vLanguage variables • 17x (|LL[]c|)Constant transformations • 17x X + XTransformation additions • 14x "file.x"Transformation inclusions • 10x let-inLocal definitions • 9x idx(L)Identity transformations • 8x XXCompositions • 4x L \ L Language restriction • 4x wTransformation variables • 2x src(X)Source extractions

  22. EXERCISE Incremental Development --- "li.l" --- --- "l.l" --- { Id = [a-z] [a-z0-9]* ; Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ; } { Exp.id : "id" ; } --- "li2l.x" --- let l = "l.l" in idx(l) + (| "li.l" -> l[Exp -> Exp] Exp.id : '\z.z' ; |) --- "ln.l" --- { Exp.zero : "zero" ; Exp.succ : "succ" Exp ; Exp.pred : "pred" Exp ; } --- "ln2l.x" --- --- "ln2li.x" --- let l = "l.l" in idx(l) + (| "ln.l" -> l[Exp -> Exp] Exp.zero : '\z.z' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 \z.z)' ; |) let l = "l.l" in idx(l) + (| ln -> l+"li.l" [Exp -> Exp] Exp.zero : 'id' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 id)' ; |) --- "ln2l.x" --- "li2l.x" o "ln2li.x"

  23. Example cont'd • Both statically reduce to samecatamorphism: (| Exp.app : Exp.app($1, $2) ; Exp.lam : Exp.lam($1, $2) ; Exp.pred : Exp.app($1, Exp.lam(Id("z"), Exp.var(Id("z")))) ; Exp.succ : Exp.lam(Id("x"), $1) ; Exp.var : Exp.var($1) ; Exp.zero : Exp.lam(Id("z"), Exp.var(Id("z"))) ; |) { Id = [a-z] [0-9a-z]* ; Exp.app : "(" Exp Exp ")" ; Exp.lam : "\" Id "." Exp ; Exp.pred : "pred" Exp ; Exp.succ : "succ" Exp ; Exp.var : Id ; Exp.zero : "zero" ; } { Id = [a-z] [0-9a-z]* ; Exp.app : "(" Exp Exp ")" ; Exp.lam : "\" Id "." Exp ; Exp.var : Id ; } -> [Exp -> Exp, Id->Id]

  24. Other Examples • Self-Application(The tool on itself!): • SQL embedding(in <bigwig>): • My-Java (endless variations): [L1 << L2] = '(L1 \ L2) + L2' [X1 << X2] = '(X1 \ src(X2)) + X2' Stm.select = 'factor (<2>) { if (<3>) return ( # \+ (<1>) ); }' java ( + sql) ( \ loops) o syntaxe_francais

  25. Implementation The 'Banana Algebra' Tool: (3,600 lines of O'Caml) [ http://www.itu.dk/people/brabrand/banana-algebra/ ] Uses(underlying technologies): 'dk.brics.grammar': for parsing, unparsing, and ambiguity analysis ! 'XSugar': for transformation: "concrete syntax  abstract XML syntax" 'XSLT': for transformation: "XML  XML"

  26. Outline • Introduction: "What is a Banana?" • Bananas for Language Transformation • Language Extension Pattern • Banana Algebra • Examples • Implementation • Related Work • Conclusion

  27. Related Work (I/III) • Macro Systems: "Growing Languages with Metamorphic Syntax Macros"[ Claus Brabrand | Michael Schwartzbach ] ( PEPM 2002 ) "The metafront System: Safe and Extensible Parsing and Transformation"[ Claus Brabrand | Michael Schwartzbach ] ( LDTA 2003 , SCP J. 2007 )

  28. Both; compared to bananas: More ambitious (expressivity) No termination guarantees (safety) Transformation "indirect" (simplicity) Related Work (II/III) • Attribute Grammars: • Language transformation (and extension)… • …via computation on AST's (using "inherited" or "synthesized" or … attributes) • E.g., Eli, JastAdd, Silver, … • Rewrite Systems: • Language transformation (and extension)… • …via syntactic rewriting, using encodings…: • gradually rewrite "S-syntax" to "T-syntax" • gradually rewrite "S-syntax" to "T-syntax" • E.g., Elan, TXL, ASF+SDF, Stratego/XT, … ST ST

  29. Related Work (III/III) • Functional Programming: • Catas mimicked by "disciplined style" of fun. programming • …aided by: • Traversal functions (auto-synthesized from datatypes) • Combinator libraries • "Shortcut fusion" (to eliminate ' ' at compile-time) • Category Theory: • A lot of this work can be viewed as Category Theory: Basically ye olde issue: GPL vs. DSL

  30. Statically reduce: Banana Algebra (term) Banana (term) Conclusion • IFbananasare sufficiently: • (Expressive) • THEN you get…: • Banana Algebra “for free” (16 banana ops): • Incremental • Modular • Simple • Safe • Efficient "Niche" 

  31. BONUS SLIDES - Reduction Semantics - If you want all the details: "Syntactic Language Extension via an Algebra of Languages and Transformations"[ Jacob Andersen | Claus Brabrand ] ( ITU Technical Report, Dec. 2008 )

  32. Reduction Semantics • Environments: • Reduction relations: • Abbreviations: • ...as a short-hand for: • ...as a short-hand for: ENVL = VARLEXPL environment of languages ENVX = VARXEXPX environment of transformations ENVLENVXEXPLEXPL 'L' ENVLENVXEXPXEXPX 'X' ,|- L L l (,,L,l)  'L' ,|- X X x (,,X,x)  'X'

  33. Semantics (L) [CONL] [VARL] l wfl ,lLl ,vL (v) ,LLl ,L'Ll' [RESL] ,L \ L'Lll' l ,LLl ,L'Ll' l~l' [ADDL] l ,L + L'Lll' l

  34. Semantics (L) ,XX (| lS -> lT [] c |) [SRCL] ,src (X)LlS ,XX (| lS -> lT [] c |) [TGTL] ,tgt (X)LlT [v=l],L'Ll' ,LLl [LETL] ,letv=LinL'Ll'

  35. Semantics (X) (|lS->lT[]c|) ,LTLlT ,LSLlS wfx [CONX] ,(| LS -> LT [] c |)X (| lS -> lT [] c |) ,XXx ,LLl [VARX] ,wX (w) [RESX] ,X \ LXxl x ,XXx ,X'Xx' x~x' [ADDX] x ,X + X'Xxx' x

  36. Semantics (X) ,XX (| lS -> lT [ ] c |) ,X'X (| lS' -> lT' ['] c' |) lT lS' [COMPX] l ,X'XX (| lS -> lT' [' ] c' c |) ,LLl [IDXX] ,idx (L)X (| l -> l [id(l)] idc(l) |) ,[w=x]X' Xx' ,XXx [LETL] ,letxw=XinX'Xx'

  37. BONUS SLIDES - More Examples -

  38. Numeral & Boolean Extension • Numeral Extension (catamorphism): • Boolean Extension (catamorphism): [var V] = var [V] [lam VE] = lam [V][E] [app E1E2] = app [E1][E2] [zero] = lam z (var z) [succ E] = lam s [E] [pred E] = app [E] (lam z (var z)) Exp : var Id : lam Id * Exp : app Exp * Exp : zero : succ Exp : pred Exp Exp : var Id : lam Id * Exp : app Exp * Exp [var V] = var [V] [lam VE] = lam [V][E] [app E1E2] = app [E1][E2] [true] = lam a (lam b (var a)) [false] = lam a (lam b (var b)) [if E1E2E3] = app (app [E1][E2]) [E3] Exp : var Id : lam Id * Exp : app Exp * Exp : true : false : if Exp Exp Exp Exp : var Id : lam Id * Exp : app Exp * Exp

  39. Lambda with Booleans lb+ll + lbl (| lb -> l[Exp -> Exp] [true] = '\a.\b.a' [false] = '\a.\b.b' [if E1 E2 E3] = '(([E1][E2]) [E3])' |) ll idx l lb Exp : var Id : lam Id * Exp : app Exp * Exp Exp : true : false : if Exp Exp Exp

  40. Incremental Development --- "li.l" --- --- "l.l" --- { Id = [a-z] [a-z0-9]* ; Exp.var : Id ; Exp.lam : "\\" Id "." Exp ; Exp.app : "(" Exp Exp ")" ; } { Exp.id : "id" ; } --- "li2l.x" --- let l = "l.l" in idx(l) + (| "li.l" -> l[Exp -> Exp] Exp.id : '\z.z' ; |) --- "ln.l" --- { Exp.zero : "zero" ; Exp.succ : "succ" Exp ; Exp.pred : "pred" Exp ; } --- "ln2l.x" --- --- "ln2li.x" --- let l = "l.l" in idx(l) + (| "ln.l" -> l[Exp -> Exp] Exp.zero : '\z.z' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 \z.z)' ; |) let l = "l.l" in idx(l) + (| ln -> l+"li.l" [Exp -> Exp] Exp.zero : 'id' ; Exp.succ : '\x.$1' ; Exp.pred : '($1 id)' ; |) --- "ln2l.x" --- "li2l.x" o "ln2li.x"

  41. Usage Scenarios • Programmers: • May extend existing languages (~ syntax macros) • Developers: • May embedDSLs into host languages (SQL in Java) • Developers (and teachers): • May incrementally specify multi-layeredlanguages • Compiler writers: • May rely on tool and implement only a small core • (and then specify the rest externally as extensions)

  42. BONUS SLIDES - Parsing & Error Reporting -

  43. Parsing • Parsing(XSugar): • Variant of Earley's algorithm: O( ||3 ) • Can parse anycontext-free grammar • Closed under union of languages • Support for production priority • Tool easily adapts to other parsing algorithms

  44. . ASTL / ~L L Ambiguity: parsingunparsing • Unparsing: • Canonical whitespace . . ASTL / ~L L . . • Parsing: • Grammar ambiguity  

  45. Ambiguity Analysis • Ambiguity Analysis: • Using implementation ( ) on: • Sourcelanguage; • Target language; and/or • …all intermediate languages (somewhat expensive) • (Note: Ambiguity analysis comes with XSugar tool) "Analyzing Ambiguity of Context-Free Grammars"[ Claus Brabrand | Robert Giegerich | Anders Møller ] ( CIAA 2007 ) "dk.brics.grammar" [ by Anders Møller ]

  46. Error Reporting • Error reporting: • Static parse-error (O'Caml-lex): • Static transformation error (XSugar): • (is actually a parse-error in a cata reconstructor) • Dynamic parse-error (XSugar): • Dynamic transformation error: • impossible :-) Prototype *** In ln2l.x (4,4)-(4,7): Parse error at "Exp" *** Parse error at character 6 (line 1, column 7) in /tmp/shape84e645.txt Could be improved *** Parse error at character 23 (line 1, column 24) in /dev/stdin

More Related