Feature Modularized Theorems in Software Product Lines

1. Feature Modularized Theorems in Software Product Lines Don BatoryDepartment of Computer SciencesUniversity of Texas at Austin ASM’07

2. Introduction • My research is at intersection of • software product lines • program generators • domain-specific languages • I come from world of • informal software engineering and design • not ASMs, compilers, formal software development, or mathematics • What differentiates my work • start with practice, find a theory that fits practice • I use algebra to explain my ideas • precursor to more formal theories of automated development ASM’07

3. This Presentation (Broadly) • Explain how program synthesis scales • software product lines (families of related programs) • Feature Oriented Programming (FOP) • Show how informal software design concepts (which engineers use) can be placed on a simple, general mathematical foundation • I am advocating • software has a simple algebraic structure • compositional • supports declarative languages • designs can be automatically optimized • ... Something important is missing! ASM’07

4. Product lines are used in many industries automotive, hardware, software reduce development costs improve quality increase variability Ex: Dell Web site declarative feature-based spec Software distinguished from other products by lack of warranty few results on verifying software product lines (SPL) verification is a fundamental piece that is missing from FOP Perspective on Product Lines ASM’07

5. Connection to ASMs • Growing community that believe verification must be intimately integrated with software design & modularity • ASMs and FOP conceived independently • both use features • increments in functionality • as organizational centerpiece • ASMs can contribute to generation of correct software in product lines • presentation brings notions of product lines to ASMs • presentation brings ASM verification techniques to FOP • Ongoing and not yet completed work with E. Börger • status report ASM’07

6. JBook (2001) correctness proof for defining, interpreting, compiling and executing bytecode for the Java 1.0 language using ASMs Discoveries static initializers underspecified (portability) concurrent initializations may deadlock compilers violated initialization semantics through standard optimization techniques + others... Case Study ASM’07

7. This Presentation (Specifically) • Show surprising similarity of features in FOP and JBook • How features modularize structure of • grammars and programs • theorems (statements and proofs) • By composing features, complete grammars, programs, theorems are synthesized • may be different for each program in a product line • brings us closer to verifying programs in software product lines ASM’07

8. JBook a brief review ASM’07

9. grammar interpreter compiler JVMinterpreter ExpI · · · · · · grammar grammar grammar grammar grammar grammar interpreter interpreter interpreter interpreter interpreter interpreter compiler compiler compiler compiler compiler compiler JVMinterpreter JVMinterpreter JVMinterpreter JVMinterpreter JVMinterpreter JVMinterpreter ExpE StmE StmI ExpO StmC ExpC Overview of JBook • JBook presents structured way using ASMs to incrementally develop the Java 1.0 grammar, interpreter, compiler, and bytecode (JVM) interpreter (that includes a bytecode verifier) • incrementally refine sublanguage of imperative expressions ASM’07

10. Overview of JBook • JBook presents structured way using ASMs to incrementally develop the Java 1.0 grammar, interpreter, compiler, and bytecode (JVM) interpreter (that includes a bytecode verifier) • incrementally refine sublanguage of imperative expressions java1.0grammar java1.0 interpreter java1.0 compiler JVM Java1.0 ASM’07

11. bytecode compiler JVM interpreter interpreter JVMRun InterpRun proof Structure of JBook • At this point, various properties are considered • ex: correctness of the compiler • equivalence of interpreter execution of program and the JVM execution of compiled program • JBook not developed with product lines in mind • definition, correctness of single interpreter, compiler of Java1.0 • show how proofs (and not only programs) are incrementally developed • let’s give it a product line description JavaAST JavaProgram parser ASM’07

12. GenVoca a 1992 model of FOP ASM’07

13. constant functions JBook Product Line • GenVoca is metaprogramming model of SPLs • base programs are values (constant functions) • features are functions that map programs to refined programs • model is an algebra of constants, functions • expressions (feature compositions) define different programs in JB product line ASM’07

14. grammar interpreter compiler JVMinterpreter ExpI grammar grammar grammar grammar grammar grammar interpreter interpreter interpreter interpreter interpreter interpreter compiler compiler compiler compiler compiler compiler JVMinterpreter JVMinterpreter JVMinterpreter JVMinterpreter JVMinterpreter JVMinterpreter · · · · · · StmE ExpO StmI StmC ExpE ExpC Java10 = JBook Product Line • JB defines a product line of Java dialects • Product considered in JBook (· denotes function composition) ASM’07

15. JBook Product Line • JBook composed all the features of JB • Possible to omit features of Java1.0 to provide different sublanguages of Java, but these are uninteresting • Better: add features to support • updates to Java 1.1 – 1.5 • state machines • metaprogramming • + others... • to mix-and-match features to yield a SPL of language dialects • exactly what we have done in FOP ASM’07

16. AHEAD Tool Suite show similarity with JBook structure ASM’07

17. AHEAD Tool Suite (ATS) • 250+K LOC Java circa 2001 • Tool suite for building GenVoca product lines • Bootstrapped: ATS is a GenVoca product line of tool suites • specify dialect of Java as composition of langauge features • synthesize grammar + tools (parsers, precompilers, etc.) • Conceptually identical to the JBook organization SoundFamiliar? ASM’07

18. Cultural Enrichment: An Application of ATS that may be of interest to the ASM community ASM’07

19. t1 start one t3 t2 stop Fire Support Simulation • Built ATS for creating a fire support simulator for US Army • distributed application where actions of agents are defined by state machines • Don’t write machines in pure Java • programs are essentially unreadable • nested switch statements, small procedures • must “reengineer” abstractions from code • Added state machines to Java • and ability to refine these machines ASM’07

20. Pure Java void otherwise_Default(M m) { ignore_message(m); } void start_otherwise(M m) { otherwise_Default(m); } void one_otherwise(M m) { otherwise_Default(m); } void stop_otherwise(M m) { otherwise_Default(m); } void exit_start(M m) { } void exit_one(M m) { } void exit_stop(M m) { } void enter_start(M m) { current_state = start; } void enter_one(M m) { current_state = one; } void enter_stop(M m) { current_state = stop; } public void receive_message(M m) { if (current_state == start) { exit_start(m); start_branches(m); return; } if (current_state == one) { exit_one(m); one_branches(m); return; } if (current_state == stop) { exit_stop(m); stop_branches(m); return; } error( -1, m ); } public String getState() { if (current_state == start) return "start"; if (current_state == one) return "one"; if (current_state == stop) return "stop"; System.err.println("unrecognizable state " + current_state); System.exit(1); return /* should never get here */ null; } // boolean booltest() { ... } example() { current_state = start; } } class example { int current_state; final static int start = 1000; final static int one = 1001; final static int stop = 1002; boolean t1_test(M m) { return !booltest(); } boolean t2_test(M m) { return booltest(); } boolean t3_test(M m) { return true; } void t1_action(M m) { } void t2_action(M m) { } void t3_action(M m) { } void start_branches(M m) { if ( t1_test(m)) { t1_action(m); enter_one(m); return; } if ( t2_test(m)) { t2_action(m); enter_stop(m); return; } ; start_otherwise(m); } void one_branches(M m) { if ( t3_test(m)) { t3_action(m); enter_stop(m); return; } ; one_otherwise(m); } void stop_branches(M m) { ; stop_otherwise(m); } ASM’07

21. Error exits Transitiondecls Constructor, methods State Machine Extension to Java state_machine example { event_delivery receive_message(M m); no_transition { error( -1, m ); } otherwise_default { ignore_message(m); } states start, one, stop; transition t1 : start -> one conditions !booltest() do { /* t1 action */ } transition t2 : start -> stop conditions booltest() do { /* t2 action */ } transition t3 : one -> stop conditions true do { /* t3 action */ } // boolean booltest() { ... } example() { current_state = start; } } State decls ASM’07

22. Pure Java void otherwise_Default(M m) { ignore_message(m); } void start_otherwise(M m) { otherwise_Default(m); } void one_otherwise(M m) { otherwise_Default(m); } void stop_otherwise(M m) { otherwise_Default(m); } void exit_start(M m) { } void exit_one(M m) { } void exit_stop(M m) { } void enter_start(M m) { current_state = start; } void enter_one(M m) { current_state = one; } void enter_stop(M m) { current_state = stop; } public void receive_message(M m) { if (current_state == start) { exit_start(m); start_branches(m); return; } if (current_state == one) { exit_one(m); one_branches(m); return; } if (current_state == stop) { exit_stop(m); stop_branches(m); return; } error( -1, m ); } public String getState() { if (current_state == start) return "start"; if (current_state == one) return "one"; if (current_state == stop) return "stop"; System.err.println("unrecognizable state " + current_state); System.exit(1); return /* should never get here */ null; } // boolean booltest() { ... } example() { current_state = start; } } class example { int current_state; final static int start = 1000; final static int one = 1001; final static int stop = 1002; boolean t1_test(M m) { return !booltest(); } boolean t2_test(M m) { return booltest(); } boolean t3_test(M m) { return true; } void t1_action(M m) { } void t2_action(M m) { } void t3_action(M m) { } void start_branches(M m) { if ( t1_test(m)) { t1_action(m); enter_one(m); return; } if ( t2_test(m)) { t2_action(m); enter_stop(m); return; } ; start_otherwise(m); } void one_branches(M m) { if ( t3_test(m)) { t3_action(m); enter_stop(m); return; } ; one_otherwise(m); } void stop_branches(M m) { ; stop_otherwise(m); } ASM’07

23. example t4 two t5 t1 start one t3 t2 stop State Machine Refinements Easy too… refines state_machine example { state two; transition t4 : start -> two conditions … do { /* t4 action */ } transition t5 : two -> stop conditions … do { /* t5 action */ } } ASM’07

24. Reduction of Complexity • Simplified mapping between “white-board” designs of agent protocols and implementation: • direct and invertible specifications • writing state machines matches informal designs domain experts use • requires fewer mental gymnastics to understand code • simplifies maintenance and extensibility • less error-prone product ASM’07

25. ExCIS 2007 • Fire Support Simulator for U.S. Army (since 2001) • 80K LOC in AHEAD • 200 features • 34 distinct state machines • 147 states • 650 transitions • 217 classes (different from state machines) • Now integrated with other simulators • Again, ExCIS is an application that was built using ATS ASM’07

26. Returning to Theme of This Talk • Build versions of ATS by composing language features • Ex: to extend Java with state machines: newJava = SM · Java • Synthesize language grammar, tools for editing, pretty printing, transforming programs by feature composition • Conceptually identical to the JBook organization ASM’07

27. AHEAD a 2002 model of FOP ASM’07

28. AHEAD • AHEAD is generalization of GenVoca that exposes different representations of programs and reveals ability to refine representations by features • governed by two simple facts • Fact#1: every program has multiple representations • source, documentation, makefiles, etc. • Fact#2: when a feature is added to a program, any or all of its representations are refined • refine source, refine documentation, refine makefiles, ... • Program representations are refined lock-step by features ASM’07

29. grammar of ExpI ASM interpreter of ExpI ASM compiler of ExpI ASM JVM interpreter of ExpI ASM of other representation(s) GExpI IExpI CExpI JExpI Constants • GenVoca constant is a vector of representations of a base program • ExpI sublanguage of Java imperative expressions ExpI = TExpI ASM’07

30. grammar refinement of StmI ASM interpreter refinement of StmI ASM compiler refinement of StmI ASM JVM interpreter refinement of StmI ASM refinement other representation(s) DGStmI DIStmI DCStmI DJStmI Functions • GenVoca function is a vector of representation refinements • StmI adds imperative statements to Java StmI = DTStmI ASM’07

31. DGStmI ·GExpI DGStmI GExpI DIStmI ·IExpI DIStmI IExpI = DCStmI ·CExpI DCStmI CExpI DJStmI ·JExpI DJStmI JExpI DTStmI ·TExpI DTStmI TExpI Composing Features • Is vector composition! • ex: Imperative Java JavaI = StmI ExpI ASM’07

32. Theorems • Are another program representation • JBook presents theorems about the programs that it develops, among them correctness of the Java1.0 compiler • TExpI denotes theorem of correctness for ExpI compiler • proof that interpreter runs of an ExpI program are equivalent to the JVM run of the compiled program • DTStmI denotes refinement of this theorem by StmI feature • show how a theorem is refined later... • DTExpI·TExpI synthesizes correctness theorem for JavaI ASM’07

33. What This Means • Claim: features modularize all program representations • grammars, code, and theorems • Composing features synthesizes • grammar, code, and theorems • for a target program in a product line • Different compositions yield the representations of all programs in a product line • show examples shortly • One more idea in AHEAD... ASM’07

34. Recursion and Nested Vectors • Elements of a vector can themselves be a vector • representation can have sub-representations, recursively • express as refinable vector • composition operator (·) recursively composes nested vectors • essence of AHEAD • Theorems have vector structure T= [ S, P ] • statement S • proof P • Theorems can be refined DT = [ DS, DP ] • Theorems can be synthesized DT·T = [ DS·S, DP·P ] • Now let’s look closer and answer a fundamental question... ASM’07

35. What is a Refinement?(or what is a feature?) Do we agree what this means? ASM’07

36. What is a Refinement? • A feature is a monotonic transformation that maps an input artifact to a modified artifact • Structure preserving: • new elements can be added • existing elements can be extended but not deleted • Refinements of ASMs take several forms [Börger 2003] • AHEAD uses three • (my term) introductions • conservative extension • parallel addition ASM’07

37. Introduction • Adding new elements is most common form of refinement • mapping null elements to non-null elements in a vector • called introduction • Adding new ASMs is very common • Adding new methods & new fields to existing classes,adding new classes common in AHEAD • Dominant action of features – virtually all introductions • Exceptional cases that existing elements are refined... ASM’07

38. Conservative Extension of ASMs • Analogous to conservative theory extensions in logic • Define condition for the ‘new case’ in which extended ASM either has no well-defined meaning or should be prevented from executing • Define new ASM with the desired additional behavior • Add the new ASM and restrict the old ASM by guarding it with the negation of the ‘new case’ condition ASM’07

39. Conservative Extension in AHEAD • Base method • Refinement adds guard, new actions ASM’07

40. Parallel Addition of ASMs • Define a new ASM to add the extra behavior • Guard the new ASM with the same conditions of the original ASM • Equivalently – extending rule (a) ASM’07

41. Parallel Addition in AHEAD • AHEAD refinement: • CLOS terminology: • after-method • before-method • around-method ASM’07

42. Frequency of Element Extensions • Conservative least frequently used • Parallel addition most common in refining existing elements • Hard to find corresponding statistics in JBook ASMs • differences in languages • before/after refinements largely hidden or encoded as state transitions than straight-line code Basically we are doing the same thing... ASM’07

43. Examples From JBook remember: refinement = adding new elements + changing existing elements ASM’07

44. Grammars & ASMs ASM’07

45. Grammars • ExpI (imperative expression) grammar is: • ExpC refines this grammar with object field & method calls • adds new productions • refines existing productions • Exactly the same technique used by AHEAD in modularizing and refining grammars ASM’07

46. Phrase Val Abruption Normal Break Continue Return Exception Reference PrimValue Null ExpI StmI StmC ExpO ExpE Inheritance Hierarchies • Phrase – result of executing an expression or statement This progressive elaboration of inheritance hierarchies is typical of FOP designs ASM’07

47. StmC defines actions taken when a method is executed StmO adds constructors, exitMethod refined to handle the actions of constructors Easy to express as an AHEAD refinement of Java methods Last Example: ASMs ASM’07

48. Theorem of Correctness the statement ASM’07

49. Statement of T isa list of invariants 14 invariants in all Don’t need to knowthe specifics of theinvariants for thispresentation Theorem T is Correctness of Compiler ASM’07

50. (stack) (exc-clinit) (begS) (new) (clinit) (stm) (fin) (exp) (abr) Java10 = StmE · ExpE · ExpO · ExpC · StmI · Statement of Correctness Theorem 14.1.1 (Correctness of the Compiler). There exists a monotonic mapping s from the run of the ASM for a Java program into the run of the ASM for the compiled JVM program such that the following invariants are satisfied: (reg) (bool1) (exc) As features are composed, the theorem statementis refined by adding new invariants but there’s more... (bool2) (begE) StmC · ExpI ASM’07