Lecture 12 Implementing Small Languages internal vs. external DSLs, hybrid small DSLs

1. Lecture 12Implementing Small Languagesinternal vs. external DSLs, hybrid small DSLs Ras Bodik Ali and Mangpo Hack Your Language! CS164: Introduction to Programming Languages and Compilers, Spring 2013UC Berkeley

2. Overview Implementation of DSLs External vs. internal DSLs internal == embedded in the host language Deep embedding Shallow embedding Examples

3. Styles of DSL implementations External DSL • own syntax (and parser) • dedicated interpreter Internal DSL: deep embedding in the host language • deep embedding: program exists as data (eg AST) • the host language implements an interpreter InternalDSL: shallow embedding in the host language • shallow embedding: DSL constructs are composed purely of host language constructs • we are using the interpreter of the host language note: instead of interpreter, we can use a compiler

4. External DSLs

5. Architecture of external-DSL implementations External DSL is a standalone language • just like “big” languages, Java, Python, C, … • domain programming abstractions with own syntax Similar interpreter/compiler architecture parser, optional analysis of AST, interpretation/compilation If compiled, DSL need not be translated into code • instead, it can be translated to a data structure which will be used by some other program P • yes, P can be viewed as an interpreter of the DSL, and the data structure is an “AST”

6. Examples Interpreted external DSLs External DSLs compiled to code External DSLs compiled to a data structure

7. Internal DSLs shallow embedding

8. Embed your DSL into a host language Shallow embedding: the new language is a library written in the host language sometimes called a “framework” When DSL is implemented as a library, we often don’t think of it as a language even though it defines own abstractions and operations But the library implementation goes very far

9. Example 1: sockets Sockets are library with own abstractions and rules Socket f = new Socket(mode) f.connect(ipAddress) f.write(buffer) f.close() Programming abstractions: socket, address, buffer Usage rules: • do not write into a closed socket • do not close a socket twice

10. Usage rules (side note) How abstractions can be combined is part of the language. With network sockets, these are enforced at runtime: ex: sock.write() checks that socket has been open This is analogous to dynamic (runtime) type checking ex: in Python, 1 + “a” will fail at runtime, when types of 1 and “a” are checked in the + operation

11. Usage rules (side note) Could we enforce these socket rules at compile time? This would be analogous to static type checking. ex: in Java, inti; String s; i+s; fails at compile time even knowing what specific values i and s will take at runtime.

12. Usage rules (side note) Could we enforce these socket rules at compile time? Yes, use static types (as in say Java) and refine Socket into Socket, OpenSocket, ClosedSocket How do we rewrite this code to use refined types? Socket f = new Socket(mode) f.connect(ipAddress) f.write(buffer) f.close()

13. Example 2: regex matching The library defines: i) functions that DSL programmers use to define a patternex: pattern ("abc"|"de")."x"can be defined as follows: defpatt= seq(alt(prim("abc"),prim("de")),prim("x")) ii) a function matches of a string against a pattern: match(string, patt)

14. Example 3: rfig: formatting DSL embedded into Ruby. see slide 8 in http://cs164fa09.pbworks.com/f/01-rfig-tutorial.pdf …

15. The animation in rfig, a Ruby-based language slide!('Overlays', 'Using overlays, we can place things on top of each other.', 'The pivot specifies the relative positions', 'that should be used to align the objects in the overlay.', overlay('0 = 1', hedge.color(red).thickness(2)).pivot(0, 0), staggeredOverlay(true, # True means that old objects disappear 'the elements', 'in this', 'overlay should be centered', nil).pivot(0, 0), cr, pause, # pivot(x, y): -1 = left, 0 = center, +1 = right staggeredOverlay(true, 'whereas the ones', 'here', 'should be right justified', nil).pivot(1, 0), nil) { |slide| slide.label('overlay').signature(8) }

16. DSL as a framework It may be impossible to hide plumbing in a procedure these are limits to procedural abstraction We can use advanced language features, such as closures, coroutines Framework, a library parameterized with client code • typically, you register a function with the library • library calls this client callback function at a suitable point • ex: an action to perform when a user clicks on DOM node

17. Example 4: jQuery Before jQuery var nodes = document.getElementsByTagName('a'); for (vari = 0; i < nodes.length; i++) { var a = nodes[i]; a.addEventListener('mouseover', function(event) { event.target.style.backgroundColor=‘orange'; }, false ); a.addEventListener('mouseout', function(event) { event.target.style.backgroundColor=‘white'; }, false ); } jQuery abstracts iteration and events jQuery('a').hover( function() { jQuery(this).css('background-color', 'orange'); }, function() { jQuery(this).css('background-color', 'white'); } );

18. Embedding DSL as a language Hard to say where a framework becomes a language not too important to define the boundary precisely Rules I propose: it’s a language if • its abstractions include compile- or run-time checks --- prevents incorrect DSL programs ex: write into a closed socket causes an error • we use syntax of host language to create (an illusion) of a dedicated syntax ex: jQuery uses call chaining to pretend it modifes a single object: jQuery('a').hover( … ).css( …)

19. rake rake: an internal DSL, embedded in Ruby Author: Jim Weirich functionality similar to make • has nice extensions, and flexibility, since it's embedded • ie can use any ruby commands even the syntax is close (perhaps better): • embedded in Ruby, so all syntax is legal Ruby http://martinfowler.com/articles/rake.html

20. Example rake file task :codeGen do # do the code generation end task :compile => :codeGen do # do the compilation end task :dataLoad => :codeGen do # load the test data end task :test => [:compile, :dataLoad] do # run the tests end

21. Ruby syntax rules Ruby procedure call

22. How is rake legal ruby? Deconstructing rake (teaches us a lot about Ruby): task :dataLoad => :codeGen do # load the test data end task :test => [:compile, :dataLoad] do # run the tests end

23. Two kinds of rake tasks File task: dependences between files (as in make) file 'build/dev/rake.html' => 'dev/rake.xml' do |t| require 'paper' maker = PaperMaker.newt.prerequisites[0], t.name maker.run end

24. Two kinds of tasks Rake task: dependences between jobs task :build_refact => [:clean] do target = SITE_DIR + 'refact/' mkdir_p target, QUIET require 'refactoringHome' OutputCapturer.new.run {run_refactoring} end

25. Rake can orthogonalize dependences and rules task :second do #second's body end task :first do #first's body end task :second => :first

26. General rules Sort of like make's %.c : %.o BLIKI = build('bliki/index.html') FileList['bliki/*.xml'].each do |src| file BLIKI => src end file BLIKI do #code to build the bliki end

27. Internal DSLs deep embedding

28. Deep embedding Represent the program as an AST or with some other data structure This AST can be interpreted or compiled

29. Parsing involved: DSL in a GP language GP: general purpose language

30. Parsing involved: GP in a DSL language GP: general purpose language

31. Reading Read the article about the rake DSL

32. Acknowledgements This lecture is based in part on Martin Fowler, “Using the Rake Build Language” Jeff Friedl, “Mastering Regular Expressions”