Cling – Past, Present and Future

1. Cling – Past, Present and Future V. Vassilev, P. Canal, A. Naumann, P. Russo CERN, PH-SFT & FermiLab

2. TPluginManager CINT Reflection Type Info PyROOT ... IO Role of C++ Interpreter in ROOT • Load/Store C++ objects • Runtime Dynamism • TFile::Open(“http://...”) • gDirectory->Get(“hist”) • python runReco.py • Fast Prototyping ROOT TClass Limited C++ language support for reflection and type introspection.

3. LLVM Toolchain As an Alternative • LLVM Core “The LLVM Project is a collection of modular and reusable compiler and toolchain technologies...” • More than 120 active contributorsApple, ARM, Google, Qualcomm, QuIC, NVidia, AMD and more • ~250 commits/week • Clang“The goal of the Clang project is to create a new C, C++, Objective C and Objective C++ front-end for the LLVM compiler.“ • More than 100 active contributorsApple, ARM, AMD and more • ~150 commits/week

4. In The Meanwhile: LLVM LLVM and Clang “The LLVM Project is a collection of modular and reusable compiler and toolchain technologies...”

5. Cling's Dual Personality • An interpreter – looks like an interpreter and behaves like an interpreterCling follows the read-evaluate-print-loop (repl) concept. • More than interpreter – built on top of compiler libraries (Clang and LLVM)Contains interpreter parts and compiler parts. More of an interactive compiler or an interactive compiler interface for clang. No need to compile Cling/ROOT with Clang/LLVM or having clang installed on the OS

6. Cling Is Better Than CINT • Full C++ support incl. C++11 • Stable and informative intermediate representations of the source • Being developed with the vision to be used in multithreaded environment (an interpreter object per thread)

7. Full C++ Support • Templates and STL are not an issue

8. Full C++ Support • Natural path to the new standards C++11/C++* $ cling –std=c++11

9. Full C++ Support • Natural path to the new standards C++11/C++* $ cling –std=c++11

10. Full C++ Support • Boost is not a dream Inline ASM an issue with the current JIT

12. Diagnostics • Column numbers and caret diagnosticsCaretDiagnostics.C:4:13: warning: '.*' specified field precision is missing a matching 'int' argumentprintf("%.*d"); ~~^~ • Range highlightingRangeHighlight.C:14:39: error: invalid operands to binary expression ('int' and 'A')return y + func(y ? ((SomeA.X + 40) + SomeA) / 42 + SomeA.X : SomeA.X); ~~~~~~~~~~~~ ^ ~~~~~~

13. Diagnostics • Pointer vs Referencesinput_line_410:2:6:error:member reference type 'TNamed' is not a pointernRef->GetName();~~~~^input_line_413:2:7:error:member reference type 'TNamed *' is a pointer; maybe you meant to use '->’?nPtr1.GetName();~~~~^ -> • Fix-it hintsFixItHints.C:7:27:warning:use of GNU old-style field designator extensionstruct point origin = { x: 0.0, y: 0.0 };^~.x =FixItHints.C:12:3: error:use of undeclared identifier 'floot'; did you mean 'float’? floot p;^~~~ float

14. Diagnostics • AmbiguitiesAmbiguities.C:20:30:error:return type of virtual function 'Clone' is not covariant with the return type of the function it overrides (ambiguous conversion from derived class 'TeachingAssistant' to base class 'Person': class TeachingAssistant -> class Student -> class Person class TeachingAssistant -> class Teacher -> class Person)virtual TeachingAssistant* Clone() const;^Ambiguities.C:7:19: note:overridden virtual function is herevirtual Person* Clone() const; ^ • Templatesinput_line_401:2:2:error: use of class template LorentzVector requires template arguments LorentzVector v;^Math/GenVector/LorentzVectorfwd.h:28:39: note:template is declared here template<class CoordSystem> class LorentzVector;~~~~~~~~~~~~~~~~~~~~~~~~ ^ Person virtual void Clone() Teacher Student TeachingAssistant virtual void Clone()

15. Diagnostics • Macro expansionsMacroExpansionInformation.C:14:7: error:invalid operands to binary expression ('int' and 'A’)X = MAX(X, *SomeA);^~~~~~~~~~~~~~MacroExpansionInformation.C:5:24: note:expanded from macro 'MAX’#define MAX(A, B) ((A) > (B) ? (A) : (B))~~~ ^ ~~~ • Template instantiationsinput_line_395:2:18: error: no matching constructor for initialization of 'PtEtaPhiEVector' (aka 'LorentzVector<PtEtaPhiE4D<double> >’)PtEtaPhiEVector v2( "v1.Rho()", v1.Eta(), v1.Phi(), v1.E() );^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~Math/GenVector/LorentzVector.h:77:8:note:candidate constructor not viable: no known conversion from 'const char [9]' to 'const Scalar’ (aka 'const double') for 1st argumentLorentzVector(const Scalar & a,^Math/GenVector/LorentzVector.h:88:17: note:candidate constructor template not viable: requires single argument 'v', but 4 arguments were providedexplicit LorentzVector(const LorentzVector<Coords> & v ) :^

17. Data Flow ① Compilation Engine (clang) ②③④⑥ ⑤ Transformation Engine Cling ⑦ ⑧ LLVM Execution Engine ①, ② –Text ③, ④, ⑤, ⑥ – Abstract Syntax Tree ⑦ - LLVM Bit Code ⑧-Target Machine Code Machine Code (x86, Alpha, ...)

18. Compilation Engine Clang Parser Transformation Engine Cling Clang Code Generator Well-defined, descriptive intermediate source representations LLVM Execution Engine

19. Transformation Engine Cling Transaction TransformationEngine • TransactionCling represents the incremental input as a set of AST nodes. • Transaction TransformersCling enables each transaction to be further customized by other clients by implementing a transaction transformer. • Interpreter CallbacksImplements callbacks for the “interesting” events. Transaction

20. Execution Engine LLVM EE-s have complete target info Thus calling into compiled libraries is not an issue. No boundary interpreted/compiled world Possible to derive from compiled classes, proper calculation of offsets and so on. LLVM JIT LLI MCJIT 📞 Compiled libs(.so) Machine Code (x86, Alpha, ...)

21. Challenges • Incompatible concepts like compilation and interpretationMany tasks that are trivial for an interpreter become a nightmare for a compiler. • Make C++ usable at the promptIncorporate the experience we have with CINT. First step: adopt the successful usability extensions from CINT.

22. Extending C++ Language [root]$ sin(12); void wrapper() { sin(12);} We want to be able to run statements void wrapper1() { inti = 12; } ? [root]$ [root]$ int i = 12; sin(i); void wrapper2() { sin(i); }

23. Extending C++ Language • Wrap the input • Look for declarations • Extract the declarations one level up, as global declarations inti = 12; void wrapper1() { inti = 12; printf("%d\n",i); } [root]$ [root]$ inti = 12; printf("%d\n",i); printf("%f\n",sin(i)); void wrapper2() { printf("%f\n", sin(i)); }

24. Streaming Execution Results [root]$ [root]$ inti = 12 (int) 12 sin(i) (double) -5.365729e-01 No semicolon (;) No semicolon (;) Precise type information Precise type information [root]$ [root]$ std::string s = “Hello” (std::string) @0x7fff65ae783cc_str: “Hello” enum e { e1 = 12, e2 = 13, e3 = 13}; e1 (enum e) (e::e1) : (int) 12 Precise location information [root]$ HelloWorld (void (void)) Function @0x108880030 at /tmp/HelloWorld.h:2: void HelloWorld() { printf("HelloWorld!\n"); }

25. Error Recovery • Filled input-by-input (aka Transaction) • Incorrect inputs must be discarded as a whole

26. Implicit auto keyword We meant inti = 5; or in C++11 auto i = 5; TNamed * f = …or in C++11auto f = … i = 5; f = new TNamed(“a”, ”b”) Cling will mark the AST Node as an implicit auto candidate and later on a custom AST pass will do the work.

27. Late Binding • Defined in • the root file if (cond) { TFile* F = 0; if (is_day_of_month_even()) F = TFile::Open("even.root"); else F = TFile::Open("odd.root"); hist->Draw(); } hist->Draw();  Opens a dynamic scope. It tells the compiler that clingwill take over the resolution of possible unknown identifiers • The root file is gone. • Issue an error.

28. Late Binding if (cond) { TFile* F = 0; if (is_day_of_month_even()) F = TFile::Open("even.root"); else F = TFile::Open("odd.root"); gCling->EvaluateT<void>("hist->Draw()", ...); } hist->Draw(); Automatically transformed into valid C++ code on AST level • Tell the compiler the identifier will be resolved at runtime • Wrap it into valid C++ code • Partially recompile at runtime

29. Code Unloading // Calculator.h class Calculator { int Add(int a, int b) { return a - b; } ... }; [root]$ [root]$ [root]$ [root]$ [root]$ [root]$ [root]$ [root]$ .L Calculator.h Calculator calc; calc.Add(3, 1) (int) 2 //WTF!?* .L Calculator.h Calculator calc; calc.Add(3, 1) (int) 4 // // Calculator.h class Calculator { int Add(int a, int b) { return a + b; } ... }; * What's That Function

30. Future Plans • Migrate to MCJIT • Object file emitted to memory • Runtime dynamic linker • Windows 64 Support • Tools • Automatic Differentiation

31. See Tomas Gahr’s demo later at the workshop Thank you!

32. Backup slides

33. Pre-Compiled Headers/Modules Carefully crafted data structures designed to improve translator's performance: • Reduce lexical, syntax and semantic analysis • Loaded “lazily” on demand

34. Pre-Compiled Headers/Modules Design advantages: • Loading is significantly faster than re-parsing • Minimize the cost of reading • Read times don't depend on size • Cost of generating isn't large

35. PCM PCH And thus PCM much more flexible

36. CINT GCCXML Reflection Type Info PCH vs PCM • Where we should get it from? Who knows more about types than the translator? Reflex Not very descriptive