Building programming languages from components

1. Building programming languages from components Simon Dobsonsimon.dobson@cs.tcd.ie http://www.cs.tcd.ie/

2. Introduction • Changing contexts for computer science • systems distributed at Internet scales - security, robustness, ... • component-based software engineering • Challenges for programming languages • new constructs, new applications • changing programmer demographics • Can see languages themselves as component systems • rapid experimentation • domain-specific languages made easy • My aim: to tell you about some work we’ve been doing in this area Programming languages from components - 2

3. Language challenges • Need to experiment with languages quickly • language - programming in the huge, composition operators, how known features mutate in new circumstances • trust management - security, authentication, webs of trust • tools - component repositories, code selection • the balance between control and expressiveness • A compiler is a big piece of kit • typically much bigger than the average program written using it • hard to get source, hard to understand once you’ve got it • barrier to entry into language design too high for many Programming languages from components - 3

4. A language as a component system • Many (most?) language features are orthogonal • don’t need to know exactly what expressions are available to describe a sequential composition of them • substantial independent re-factoring of features auto Int with [| 1, 2, 3 |] (fun( Int n ) n + 1)(11) “hello” 12 The details of the sub-features aren’t really important in defining the feature itself exp1 ; exp2 • Features tend to be compositional: they often work by combining sub-expressions without inspecting them Programming languages from components - 4

5. What does this mean? • A language is a composition of features • typing and behaviour - and syntax too, to an extent • Specify the features independently, and compose • there will be some dependencies… • …but a lot more independence • Effects • re-use - leverage feature independence • easy experimentation - just a composition exercise • extensibility - present features as syntax, or libraries, or hidden • targeting - exactly the features you want, how you want them • performance - generality brings overheads • complexity - dependencies, novel approach Programming languages from components - 5

6. Vanilla • We’ve been developing an experimental language design system called Vanilla • build interpreters from language fragments (“pods”) • re-use existing fragments • incrementally construct language tools • Standard pods cover a large sub-set of the language design space • imperative, functional, “typeful”, object-oriented • 100% pure Java • The idea is to be able to experiment with (and deploy) new language variants quickly and easily Programming languages from components - 6

7. A framework with core algorithms running over a set of components providing the actual functionality Services Sub-typing Vanillaarchitecture Parser Type checker Record attributes derived from typing for use at run-time Interpreter Attributes Programming languages from components - 7

8. ASTStringType ASTUniversalType ASTFunction Component phases Phase walks the AST Components express an interest in the different node types Try each interested component until one succeeds, one fails, or all decline to commit themselves Having several interested components overloads the abstract syntax Some interesting generalisations - e.g. functions taking types, the “dot” operator, ... Phase Programming languages from components - 8

9. Component/phase interactions interface TypeChecker { public Type typeCheck( ASTExpression x, TypeEnvironment tc, Context c ) throws TypeException; public Type parseType( ASTType x, TypeEnvironment tc, Context c ) throws TypeException;} Recursive calls from components access the complete phase Overall component-based phase walks the AST and calls components interested in each AST node type encountered Programming languages from components - 9

10. What’s in a pod Parser ALL(X <: T) X Concrete syntax to abstract syntax fun(X <: T) … Type-checker new UniversalType(…) Abstract syntax to type Attributes Behaviour may depend on attributes derived from type-checking Abstract syntax to behaviour Interpreter new IClosure(…) Programming languages from components - 10

11. Some standard pods • Core • ground types, simple constructed types, control structures, ... • Functions • higher-order closures with currying • Bounded universals • C++ templates • Automorphic values • dynamic typing • -recursive types • simple self-references • Objects and classes • based on Abadi and Cardelli object calculi with covariant self types The basis for functional programming - no type inference as yet Used for Java/CORBA object types, too Programming languages from components - 11

12. The Vanilla toolset • Framework • a component architecture for language tools • Pods • implement a single language feature - and only that feature • try to work uniformly across sub-expressions (e.g. don’t manipulate them operationally, just evaluate them appropriately) • Tools • batch and interactive interpreters • parser component generator - generates composable fragments of recursive-descent parsers from JavaCC-like grammar files • pod compilers - combine components together into a single class, eliminating run-time class loading • Samples - an implementation of the O-2 language Programming languages from components - 12

13. Languages • A language, in Vanilla terms, is just a set of pods composed together within the framework • language definition files list the classes Grammar fragment Typing and sub-typing // core pod ie.vanilla.pods.core.Core ie.vanilla.pods.core.CoreTypeComponent ie.vanilla.pods.core.CoreSubtypeComponent ie.vanilla.pods.core.CoreInterpreterComponent ie.vanilla.pods.core.CoreInitialValueComponent Interpreter Default initial values • Tools instanciate classes into the framework • …and this is component-based too, of course… • Once loaded, behaves like a “normal” interpreter Programming languages from components - 13

14. Pod re-use • Pod usage • add/change a pod in a language without re-writing the whole thing • change a component of a pod, without changing the others • Same pods in several languages or variants • e.g. implement all Pascal and a lot of Java using the standard pods FOR i := 1 TO 10 DO j := j + f(i)END; for(i = 1; i < 11; i = i + 1) j = j + f(i); ASTFor ASTInteger Basically we just eliminate some of the possibilities syntactically ASTLess ASTAdd ... Programming languages from components - 14

15. Must have Ions • Why do we name the super-class of a class? • just need to know the (minimal) type of possible super-classes • Ions • “class with a free super-class” (function over classes) • bind to a real super-class before instanciating • just change the pod implementing classes - nowhere else Apply the same functionality to a family of legal super-classes Reify ion with a particular super-class before instanciating the resulting class Don’t over-commit to the super-class Programming languages from components - 15

16. Client-side CORBA integration Import IDL definitions directly off the web - no stub compiler import idl "http://www.random.org/Random.idl"; Random r = bind(Random, ior "http://www.random.org/Random.ior"); Int i; Sequence(Int) rs = for(i = 0; i < 10; i = i + 1) r.lrand48(); Binding builds a DII-based stub Call the stub like any other method IDL maps to Vanilla types through a component-based mapping Programming languages from components - 16

17. Conclusion • As long as we program, we’ll need new language variants • composition, security, trust management • high-level, domain-specific • Vanilla provides an infrastructure for experimentation • define language features as fragments, and compose • lots of standard pods to re-use • experiment quickly across the spectrum of language design • Vanilla is open source, and on the web athttp://www.vanilla.ie/ Programming languages from components - 17