is not a framework… but it makes frameworks better

1. CCA Common Component Architecture is not a framework… but it makes frameworks better Gary Kumfert Center for Applied Scientific Computing This work was performed under the auspices of the U.S. Department of Energy by the University of California, Lawrence Livermore National Laboratory under Contract No. W-7405-Eng-48. UCRL-PRES-205332

2. Outline • Zen of Babel (too short to be an intro) • Establish my “street-cred” (uses & users of Babel) • Change-Oriented Software(birth of a new buzz-word?)

3. These subroutines come directly from the SIDL Some other subroutines are “built in” to every SIDL class/interface F90/Babel “Hello World” Application programhelloclient usegreetings_English implicitnone type(greetings_English_t) :: obj character (len=80) :: msg character (len=20) :: name name=’World’ callnew( obj ) callsetName( obj, name ) callsayIt( obj, msg ) calldeleteRef( obj ) print *, msg end programhelloclient

4. greetings.sidl: A Sample SIDL File  package greetings version1.0 { interface Hello { void setName( in string name ); string sayIt ( ); } class English implements-all Hello { } }   Next several slides will show implementations of this interface in Python & C++. See C & Fortran 90 in our SC’04 Tutorial Slides www.llnl.gov/CASC/components

5. A Python Implementation  class English: def __init__(self, IORself): self.__IORself = IORself # DO-NOT-DELETE splicer.begin(__init__) self.d_name = ’’ # DO-NOT-DELETE splicer.end(__init__) def sayIt(self): # DO-NOT-DELETE splicer.begin(sayIt) return ’Hello ’ + self.d_name + ’!’ # DO-NOT-DELETE splicer.end(sayIt) def setName(self, name): # DO-NOT-DELETE splicer.begin(sayIt) self.d_name = name # DO-NOT-DELETE splicer.end(sayIt)     greetings/English_Impl.py

6. A C++ Implementation  namespace greetings { class English_impl { private: // DO-NOT-DELETE splicer.begin(greetings.English._impl) ::std::string d_name; // DO-NOT-DELETE splicer.end(greetings.English._impl)    greetings_English_Impl.hh ::std::string greetings::English_impl::sayIt() throw () { // DO-NOT-DELETE splicer.begin(greetings.English.sayIt) ::std::string msg(”Hello ”); return msg + d_name + ”!”; // DO-NOT-DELETE splicer.end(greetings.English.sayIt) }    greetings_English_Impl.cc

7. When I say “Language Interoperability” I mean something very different than from what most applications do. Scripting Driver (Python) Physics Models (FORTRAN 90) Simulation Framework (C) Visualization System (Java) Solver Library (C++) Numerical Routines (FORTRAN 77) Suppose your iterative solver isn’t converging, but oscillating in a curious way. Can you pause the simulation, write a Python script to extend the (C++) convergence check and log the pertinent physics in those regions? Logging and Plotting (Python)

8. When I say “Language Interoperability” I mean complete language transparency Scripting Driver (Python) Physics Models (FORTRAN 90) Simulation Framework (C) Visualization System (Java) Solver Library (C++) Numerical Routines (FORTRAN 77) Now suppose you have a regime in your physics models that is of interest. Can you extend Fortran90 modules in some scripts to explore new ideas as the simulation progresses? Adaptive Sampling (Python)

9. Mixing Of Languages is Commonplace, but too often Inflexible & Fragile • Developers are often painfully aware of language boundaries • Usually fixed early on in design process • Frequent source of portability problems. • Usually a great reluctance to add new languages • Often teams will rewrite a package in a language they already use, rather have to add a new one. • Particularly true when several languages are already in play. • Most tools for language interoperability are point-to-point & and have a strong bias for one or the other

10. When Mixing n Languages, Tool usage can grow O(n2) Fortran 77 cfortran.h Chasm COM CORBA JNI Native Platform-Dependent Siloon SWIG Fortran 90 C C++ Python Java

11. Babel is an n-way Language Interoperability Tool Once a library has been “Babelized” it is equally accessible from all supported languages f77 f90 C C++ Python Additional languages added ~1/year Java

12. Babel Supports a Uniform Model Across All Languages Full OOP, Polymorphism, Exception Handling in every language. f77 f90 C C++ Python Can throw an exception from C++, catch it in F77 and have the exception itself be in C Java

13. Quick Review:Connectivity Strategies • File I/O (slowest…even archival) • UNIX Pipes • Messaging Layer (fairly slow, data is packed & unpacked, easiest to do portably) • CORBA, WebServices, SOAP, XMLRPC • Interpreted Middleware (virtual machine) • EJB, .NET, Pyre • In Process (fastest, most complex/arcane, vendor specific) • Microsoft COM • Babel

14. Applying Babel to Legacy Code • Write your SIDL interface • Generate server side in your native language • Edit Implementation (Impls) to dispatch to your code (Do NOT modify the legacy library itself!) • Compile & Link into Library/DLL Stubs mycode.sidl Babel Compiler IORs libmycode.so Skels Impls legacy_library.so

15. Example of Babelized Legacy Code: MPI package mpi version 2.0 { class Comm { int send[Int]( in array<int,1,row-major> data, in int dest, in int tag ); ... } } mpi.sidl API choices made in this example: • Made sends and receives operations on MPI communicator objects • Overloaded methods based on scalar type • Use Babel arrays instead of buffer and count • “row-major” (or “column-major”) also guarantees non-strided, even for 1-D arrays.

16. Example of Babelized Legacy Code (MPI): The *Impl.h /* DO-NOT-DELETE splicer.begin(mpi.Comm._includes) */ #include “mpi.h” /* DO-NOT-DELETE splicer.end(mpi.Comm._includes) */ ... struct mpi_Comm__data { /* DO-NOT-DELETE splicer.begin(mpi.Comm._data) */ MPI_Comm comm; /* DO-NOT-DELETE splicer.end(mpi.Comm._data) */ }; mpi_comm_Impl.h • MPI is a C standard, so implement the Babel wrappers in C. • New Communication Objects have state • For C state is kept in a *_data struct. • Remember to observe splicer blocks

17. Example of Babelized Legacy Code (MPI): The *Impl.c int32_t impl_mpi_Comm_sendInt( mpi_Comm self, SIDL_int__array data, int32_t dest, int32_t tag ) { /* DO-NOT-DELETE splicer.begin(mpi.Comm.sendInt) */ struct mpi_Comm__data *dptr = mpi_Comm__get_data( self ); void * buff = (void*) sidl_int__array_first(data); int count = sidl_int__array_length(data, 0); return mpi_send( buff, count, MPI_INT, dest, tag, dptr->comm); /* DO-NOT-DELETE splicer.end(mpi.Comm.sendInt) */ } • In C, use *_get_data() to extract user-defined state from Babel object. • Since array is 1-D and unstrided, use address of first element as buffer mpi_comm_Impl.c

18. Outline • Zen of Babel (too short to be an intro) • Establish my “street-cred” (uses & users of Babel) • Change-Oriented Software(birth of a new buzz-word?)

19. Use Babel to…Automate Language Wrappings • E.g. Hypre (LLNL) • Scalable linear solvers and preconditioners (Mostly C) • Had 4 partial, non-portable Fortran Bindings • None evolved with changes to their core code • Also useful for shifting burden & blame “Blame Babel!” --Randy Bramley

20. Babel Background • History: • Started 1999 as LDRD • 21 quarterly releases since July 2001 • Code-Generator 1MB Java Jar file • Runtime Library 2MB C library • Make check • Requires 1.5 GB of disk • Takes ~6 hours on ASCI Blue node

21. Use Babel to…Get OOP in non-OO languages • Don’t wait for F2003 for polymorphism • Get basic benefits of OOP, without venturing beyond C (and without hand-coding your own virtual function dispatch like PETSc and X Windows did)

22. Use Babel to…Build Component Frameworks • CCA (SciDAC) • Rob Armstrong already talked about • PSI (LLNL, LDRD) • Multiscale physics – Material science • ALE3D or ALE-AMR at coarse scale • DD or Crystal Plasticity at meso-scale • Extend Pope’s work on in-situ adaptive sampling • MPMD model – Fork&shedule multiple MPI subtasks on petascale machines • DUNE(LLNL) • Granular Flows • Overlord, federated external codes, unifying parallel data store

23. Use Babel to…Establish Domain Standards • TSTT (SciDAC ISIC for Meshing) • TOPS (SciDAC ISIC for Solvers) • CCA (Started with 3 legacy frameworks which were not interoperable) • Others that are “tentative” or “in-the-works” • LAPACK • ScaLAPACK • NetSolve

24. Use Babel to…Connect F77 to F77 ! ;) • NWChem (PNNL) • Integrate a F77 code with its own F77 code • Both link internally against C code • One assumes a trailing underscore • Another insists on –fno-underscore flag • Really use Babel to create a uniform ABI.

25. Outline • Zen of Babel (too short to be an intro) • Establish my “street-cred” (uses & users of Babel) • Change-Oriented Software(birth of a new buzz-word?)

26. How Big is a “Big Code”?(lines of source) • Simulator for Major Systems in a Tokamak? • Simulator for capsule physics in NIF? • Hewlett-Packard Printer Driver? Thanks to Paul F. DuBois, LLNL

27. 300,000 500,000 5,000,000 How Big is a “Big Code”?(lines of source) • Simulator for Major Systems in a Tokamak? • Simulator for capsule physics in NIF? • Hewlett-Packard Printer Driver? Thanks to Paul F. DuBois, LLNL

28. How Big is a “Big Code”?(lines of source) • Simulator for Major Systems in a Tokamak? • Simulator for capsule physics in NIF? • Hewlett-Packard Printer Driver? 300,000 From a software engineering perspective: if the large scale simulations aren’t really that big, why do they seem so difficult? 500,000 5,000,000

29. Scientific Computing Software is Dominated by Change • Scientific programs are changed much more often than programs of similar size in other fields. • A twenty-year-old LLNL program changed substantively 75 times in one year. It was not a period of major new development or a new machine. • The developers are not the only ones who need to change the program – the users do too. • Even the application area may change or expand. Thanks to Paul F. DuBois, LLNL

30. Principles of Change-Oriented Software • Assume Change • Develop an Appropriate Strategy • Constantly Reevaluate

31. Change-Oriented Principles:1. Assume Change • Internal • New Understanding or Interests • New Algorithms • External • New Partnerships • New Funding Directions • New Hardware • New Software Tools, Languages, etc.

32. Change-Oriented Principles:2. Appropriate Strategy • Throw-Away • Internal Only • Targeted External • Framework Specific • Components • Community Code + Planned Longevity - + Upfront Cost - - Pain w/ Unanticp. Change +

33. Change-Oriented Principles:3. Constantly Reevaluate • Changeability is not fairy dust you sprinkle in your code at the end of the project! • Occurrence of painful, unexpected change does not necessarily indicate strategy was flawed

34. All These Software Techniques Try to Support Change • Absorb change without losing correctness • Empower and exploit the creativity of users • Reduce dependency entanglement among developers

35. Current Change-Oriented State of Art: Scripting Python • Python is BIG at Livermore • SciPy.org: • SWIG and PyFort shrink-wrapped codes • Enthought, Inc. provides the consulting services • PyMPI • gives you a interactive session to parallel machine Hydro FFT Graphics

36. Users Like Scripting • Developers aren’t a bottleneck • Users share domain-specific expertise with each other. • Users are much more productive • Users enjoy coding (scripting is fun) • If you expose the “main loop”, they can add physics or modify quantities (e.g., adding noise to boundary conditions or energy deposition).

37. Developers Like Scripting • Developers get built-in graphical debugger • Prototype algorithms in interpreter. • Many facilities get out of compiled code for good • If 90% of runtime is spent in 10% of code, why not script the other 90%? • Can try new uses/configurations for existing pieces without a lot of investment

38. Downside of Scripting • Does your code look like this?

39. Downside of Scripting • Does your code look like this? • Or like this?

40. Components vs. Scripting • Like Babel, Scripting is still imperative programming model using libraries • Components have a “composability model” • Components register themselves to runtime system • Reflection and Introspection • Components declare what they provide as well as what the consume

41. Software Components: Commercial vs. Computational • Industry developed component technology to • increase reuse • control costs • scale to large software systems • Large Scale Simulation needs it for • integration of small systems to large ones • amenability to change • manage correctness in the face of change

42. Ongoing Babel Research • Build (converting source to binary) • Distributed Babel • Nonblocking RMI • Semantics in SIDL • Automatic Component Extraction • Extensibility (Typemaps)

43. More Information • Babel • www.llnl.gov/CASC/components • components@llnl.gov • CCA • www.cca-forum.org • kumfert@llnl.gov • Babel Team • LLNL: Tom Epperly, Tammy Dahlgren, & Jim Leek, alumni, summer students • CCA: Wael Elwasif (ORNL), Boyana Norris (ANL), Steve Parker (Utah), Ben Allan & Rob Armstrong (SNL) • Open Source Community: Adam Powel III (MIT) Debian Maintainer, and others