Integrating Bigloo into the .NET Platform

1. Integrating Bigloointo the .NET Platform Séminaire du club IN’Tech Grenoble 15/05/2003 Yannis.Bres@sophia.inria.fr Bernard.Serpette@sophia.inria.fr Manuel.Serrano@sophia.inria.fr

2. Introduction Outline Scheme, Functional Programming and Bigloo The .NET Framework Motivations for a Bigloo .NET back-end How do we implement such a different language than C#/J#/… in .NET? Separate compilation A different notion of polymorphism Mapping of types and constructs Closures Tail recursion Continuations Inlining Conclusions

3. A functional language (strict evaluation, side-effects), derived from LISP Scheme First class data types: Numerals, strings, symbols, pairs, vectors (arrays), lambdas High-order language  functions are first-class objects Dynamic type-checking Very powerful macro system + eval at run-time Prefixed notation n-ary functions made coherent (andcond1cond2cond3cond4 …) A standard: R5RS (Revised5 Report on Scheme)

4. Imperative programming: int fac( int n ) { int result= 1; while (n > 0) { result*= n; --n; } return result; } Functional Programming Functional programming: (define (facn) (if (zero? n) 1 (* n (fac (- n 1)))))

5. An optimizing Scheme compiler to C code, JVM bytecode and now .NET bytecode Bigloo Functional programming often thought to have bad performances Bigloo generates C code whose performance is close to human-written C code Bigloo JVM programs are about 2 times slower than Bigloo C programs An extension of Scheme R5RS Separate compilation (modules) Object extensions à la CLOS/Meroon + extensible classes Optional type annotations for compile-time strong typing Fair-Threads … Support for continuations on the C back-end, only No arbitrary-precision integer arithmetic

6. Similar to the Java platform: The .NET Framework A stack-based register-less Virtual Machine (the Common Language Runtime) A portable bytecode (Common Intermediate Language), verified then executed Memory is garbage collected Bytecode is Just-In-Time compiled to native code (can be precompiled in .NET) 3~4 implementations of .NET so far: Microsoft (Commercial + Rotor): de facto reference implementations Ximian (Mono): JIT compiler, runtime almost complete, few compilation tools DotGNU (Portable .NET): interpreter only, missing parts in runtime, many tools Hopes of higher performances: Bytecode is fully linked within assemblies At least, applications start faster: building .NET shell tools is realistic Bytecode is systematically JITed: more efficient than looking for hot-spots? At least, more deterministic run times

7. As for Java: Motivations for a Bigloo .NET back-end "Write Once, Run Everywhere" promise Big sets of features provided by the runtime libraries Lots of services provided by the VM: RTTI, GC, subtyping polymorphism, … .NET is more language-agnostic Not a simple mapping of Java constructs to JVM bytecode Better interoperability with other languages Ability to use the most suitable language for different parts of an application .NET has better performances? Can we get a .NET back-end at low cost once we have a JVM back-end? Bigloo generates C code whose efficiency is close to human-written C code  Can we achieve the same in the .NET Framework?

8. Bigloo and its back-ends Bigloo Application Bigloo Compiler Bigloo Runtime Bigloo JVM App Bigloo CIL App ··· C# App Eiffel# App Bigloo C App Bigloo Runtime Bigloo Runtime Bigloo JVM Runtime Bigloo .NET RT Bigloo Runtime Bigloo C Runtime JVM CLR OS OS OS

9. Compiler and runtime code sizes (lines)

10. Introduction Outline Scheme, Functional Programming and Bigloo The .NET Framework Motivations for a Bigloo .NET back-end How do we implement such a different language than C#/J#/… in .NET? Separate compilation A different notion of polymorphism Mapping of types and constructs Closures Tail recursion Continuations Inlining Conclusions

11. A Bigloo application is made of several modules (one per file), a module may define several classes, the graph of module references can be cyclic Separate compilation Each module is compiled separately into a .il file 2 problems: MS ilasm can’t compile to object files  no C# and IL mixing The Bigloo .NET runtime is a mix of C# sources and Bigloo generated IL MS ilasm wants types to be fully-qualified by the module defining the type Finding the module that defines a type should be the job of the linker Solution: Code is compiled using PNet ilasm, which performs separate compilation Problem: Binaries generated by PNet are not recognized by MS VMs and not signed (yet) Solution: Binaries can be disassembled, reassembled and signed by MS tools

12. In C++/Java/C#/Eiffel/…/CLR object systems: A different notion of polymorphism Classes encapsulate both data and behaviour (methods), different visibility levels Method dispatch based on dynamic or static type of objects In CLOS object system: Classes only encapsulate data, no visibility concept Behaviour is implemented through generic functions Generic function dispatch based on dynamic type of arguments Bigloo performs single dispatch, i.e. dispatch based on first argument only First idea: use .NET object system All generics must be defined on the root class No more separate compilation Very large and sparse vptr tables Our approach: All Bigloo types are indexed, all instances have a type index Dispatch based on the type index, using compressed vptr tables

13. integer int32, possibly boxed Mapping of types and constructs real float64, possibly boxed string byte[] (for mutability) symbol bigloo.symbol pair bigloo.pair Unboxed numerals are passed on the stack Boxed numerals are allocated in the heap, and a reference is passed on the stack Boxing is reduced as much as possible thanks to optional type annotation and type inference system (SUA) module (singleton) class class class method method closure bigloo.procedure function static method or inlined loop

14. <Lambda , optional private environment> used as first-class values Closures Comparable with Java inner anonymous classes with a single method (define (inc x) (lambda (y) (+ x y))) (map (inc 5) '(1 2 3 4))  (6 7 8 9) Most of non-escaping closures (not used as values) are simplified by the compiler: Inlined as loops when called as tail-recursive functions Turned into regular functions otherwise First idea: each closure is a singleton instance of a class Too much of a load on the type system (>4000 real closures in the compiler!) Even worse with JVM, where each class is a file!

15. Current implementation from the Java back-end, which lacks of function pointers: Closures Each closure is an instance of a bigloo.procedure: Object[] environment; int arity, index; publicObject apply( Object argument_list ) { switch( index ) { case 0: call closure body 0 case 1: call closure body 1 … Problems: Cost of virtual function apply Cost of body dispatching depending on closure index Loss of potential type information for environment variables .NET possible improvement? Delegates (<pointer to object , pointer to method>) could simplify the dispatch Delegate invocation seems to be 2x slower than indexed call…

16. Continuation of fac call No continuation for iter call In functional programming, iterations are most of the time written as recursions: Tail Recursion Recursive n!: (define (facn) (if (zero? n) 1 (* n (fac (- n 1))))) 6 (* 3 ) (fac 3) 2 (fac 2) (* 2 ) (fac 1) 1 (* 1 ) 1 (fac 0) Tail-recursive n!: (define (facn) (define (itern acc) (if (zero? n) acc (iter (- n 1) (* acc n)))) (iter n 1)) ● (iter 3 1) (iter 2 3) (iter 1 6) (iter 0 6) 6 (fac 3) R5RS requirement: tail-calls should not add activation blocks

17. Tail recursion not feasible in C and Java Tail Recursion Self-recursive or co-recursive function calls implemented as gotos (define (even?n) (or (zero? n) (odd? (- n 1)))) (define (odd?n) (and (not (zero? n)) (even? (- n 1)))) What about functions called as tail recursion ? In Java: stack consumption In .NET: call instructions can be flagged as tail. Security issue: security grant/denial frames are dropped Probably not a real concern for Bigloo users Significant performance penalty!

18. call/cc (call with current continuation) captures the continuation of a computation Continuations Continuations are generalization of escapes: Abort a computation (exception) + resume a computation (stack restoration) (+ 3 (call/cc (lambda (k) (* 3 4)))) 15 (define kont '?) (+ 3 (call/cc (lambda (k) (set! kont k) (* (/ 5 (k 8)) 4)))) 11 (kont 10) 13 R5RS requirement: continuations are first-class values  Ability to restore the stack state save by a call/cc at any time The stack can only be introspected in .NET! Simplification: continuations can only be called within the call/cc dynamic extent  Continuations become simple computation escapes Perfectly handled by exceptions holding the call/cc return value

19. Scheme/Bigloo programs use a lot of small functions like cons, car, cdr, … Inlining Such functions must be inlined ! Bigloo already performs inlining of Bigloo functions What about Bigloo .NET runtime functions? As for the JVM JIT compiler, inlining doesn’t seem to be performed as expected Inlining must be done "by hand", by injecting IL code instead of function calls

20. A new back-end for Bigloo, to embrace the "Common Language Runtime" initiative Conclusions Enlargement of Functional Programming targets Recette succeeds at 100% on MS platforms The Bigloo compiler has succeeded the bootstrap proof of concept A work still in progress Tail calls not yet investigated Could we handle continuations within the CLR? No benchmark yet available