GenAWeave: A Generic Aspect Weaver Framework based on Model-Driven Program Transformation

1. GenAWeave: A Generic Aspect Weaver Framework based on Model-Driven Program Transformation Suman Roychoudhury roychous@cis.uab.edu Ph.D. Dissertation Defense http://www.cis.uab.edu/softcom/GenAWeave/ Committee Members: Dr. Jeff Gray (Advisor, Chair) Dr. Purushotham Bangalore Dr. Barrett Bryant Dr. Marjan Mernik Dr. Anthony Skjellum Dr. Randy Smith May 19, 2008

2. Outline of Presentation Lack of Modularization (Legacy software decay) Several Billion lines of Legacy code Motivation Challenges Lack of Reusability + Accidental Complexities Program Transformation ResearchGoals Approach Model-Driven Engineering Generic Aspect Weaver Framework Evaluation Aspect FORTRAN Aspect Pascal

3. Challenges with Maintaining Legacy Systems 50% of the annual $230 billion spent on all software budgets Commercial + Scientific Applications > 200 Billion Lines of Legacy code A need for constant fixes Migration is not an easy task ! Huge investment in infrastructure, effort, manpower Limited knowledge of the entire system Too much time required to make simple changes

4. Laws of Evolution – Lehman, 1997 • Systems must be continuallyadaptedelse they become progressively less satisfactory and are subjected to decay over a period of time. • As systems evolve, they become more complex  must take some action to reduce the complexity. Brian Foote and Joseph Yoder

5. Tools and Techniques to Support Evolution of Legacy Software • Aspect-Oriented Programming AOP is a programming technique that allows programmers to modularize crosscutting concerns (i.e., program features that cut across the typical divisions of modularity, such as logging). Such features often cannot be cleanly decomposed from the rest of the system in both the design and implementation, and result in either scattering or tangling of source code. • Software Refactoring “The process of changing a software system in such a way that it does not alter the external behavior of the code, yet improves its internal structure” [Fowler et al., 1999].

6. Crosscutting Concerns XML Parsing in Apache Tomcat Server Profiling in Apache Tomcat Server http://www.parc.com/research/projects/aspectj/

7. Separation of Crosscutting Concerns - AOP Module A Module B Crosscutting concerns CC1/CC2 superimposed on Module A and B CC1 CC2 Concern CC2 Concern CC1

8. Aspect Weaving Aspect Specification Aspect Program Aspect Weaver Base Program Modified Target Program • The term “Aspect Weaving” refers to combining aspects (i.e., crosscutting features) with non-crosscutting pieces of source code. • The tool that is responsible for merging the separated aspects with the base code is called an aspect weaver.

9. Aspect Weavers: Current Limitations Several tools and language extensions have been developed to enable aspect weaving However – • Specific to a few popular languages (e.g., Java) • New tools are developed from scratch without preserving knowledge of previous construction • No emphasis on reusability in a different language and platform context • Generally do not support custom aspect weaving functionality (e.g., join point for loops, with construct in Object Pascal)

10. Research Questions • Is there a technique to construct aspect weavers for legacy languages without extending or inventing a new parser (or compiler) from scratch? • Can such construction be supported in a more generic or language-independent way? • Can the knowledge of building a weaver from a previous construction be reused in a different language and platform context? • Can we add custom features to an existing weaver and the same knowledge transferred to new weavers?

11. Outline of Presentation Lack of Modularization (Legacy software decay) Several Billion lines of Legacy code Motivation Challenges Lack of Reusability + Accidental Complexities Program Transformation ResearchGoals Approach Model-Driven Engineering Generic Aspect Weaver Framework Evaluation Aspect FORTRAN Aspect Pascal

12. Challenges in Language-Independent Legacy Adaptation Challenge C1 – The Parser Construction Problem Challenge C2 – The Weaver Construction Problem Challenge C3 – Accidental Complexities of Transformation Specifications Challenge C4– Language-Independent Generalization of Transformation Objectives

13. Transformation System Higher- order Transforms Lower- order Transforms Parser Legacy System User Inputs Aspect Language Challenge C1 – The Parser Construction Problem • Scalability • Extensibility • Reusability • C++ • Object Pascal • Fortan • Cobol • Ada • Jovial • VHDL

14. Challenge C2 – The Weaver Construction Problem Transformation System • Availability of AST representation • Low-level Transformation Infrastructure pattern probe_id_pattern1() : unqualified_id = "printf". rule insert_probe1 (s: statement_seq): function_body -> function_body = " { \s } " -> "{ \probe_id_pattern1\(\) (\"Entering Method…\"); { \s } }". Higher- order Transforms Lower- order Transforms Parser Legacy System User Inputs Aspect Language

15. Transformation System Higher- order Transforms Lower- order Transforms Parser Legacy System User Inputs Aspect Language Challenge C3 – Accidental Complexities of Transformation Specifications • A high-level aspect language • layered on top of the transformation system • Translates to low-level • specifications aspectinsert_probe1 { before(): execution (function_declaration) { // insert print statement } }

16. Challenge C4 – Language-Independent Generalization of Transformation Objectives Transformation System rule BeforeAdvice2Pattern { from s : APascal!BeforeAdvice to t : RSL!Pattern ( phead <- ph, ptext <- spt, token <- 'statement_list', ), ph : RSL!PatternHead ( name <- 'before_ advice_stmt' ), ... } Higher- order Transforms Lower- order Transforms Parser User Inputs Aspect Language

17. Outline of Presentation Lack of Modularization (Legacy software decay) Several Billion lines of Legacy code Motivation Challenges Lack of Reusability + Accidental Complexities Program Transformation Research Goals Approach Model-Driven Engineering Generic Aspect Weaver Framework Evaluation Aspect FORTRAN Aspect Pascal

18. Research Goals Goal 2 : Reuse core transformations across language domains usinghigher-order transforms (C2, C4) Goal 1 : Raise the level of abstraction (aspect layering) for end-programmers (C3) Program Transformation Engine (C1,C2) Higher-Order Transforms Low-Level xForm constructs High-level language constructs Increase the level of abstraction and reuse the core transformationsrules across language domains Aspect Weavers

19. Model-Driven Program Transformation based Aspect Weaving Framework • Reuses existing parsers • Reuses several artifacts while creating weavers from one language to another The high-level aspect language is used to raise the level of abstraction and hide the accidental complexities that are associated with program transformation rules The program transformation rule generator produces program transformation rules using higher- order model transformation rules The generated program transformation rules are processed along with the source program to accomplish the desired weaving

20. Outline of Presentation Lack of Modularization (Legacy software decay) Several Billion lines of Legacy code Motivation Challenges Lack of Reusability + Accidental Complexities Program Transformation ResearchGoals Approach Model-Driven Engineering Generic Aspect Weaver Framework Evaluation Aspect FORTRAN Aspect Pascal

21. Overview of Approach Model-Driven Engineering (MDE) Technical Space (TS) Grammarware TS Grammarware TS EBNF EBNF KM3 M3 Aspect.gGrammar Aspect Metamodel RSL Metamodel RSL.g Grammar M2 Source Program Myaspect.ap AspectPascal Myaspect Model MyRsl Model MyRsl.rsl RSL program M1 Transformation System Extraction Transformed Aspect2RSL Transformation Injection Program PT Based Back-End Current State-of-the-Art Techniques Program Transformation Back-End Model-Driven Front-End Model Transformation GenAWeave in Action

22. Current State-of-the-Art: Transformation Techniques • Object-based transforms, such as a visitor object applied to an object model • Intermediate representations that permit primitive transformations to be applied to a set of languages (e.g., .Net CodeDOM) • XML-based transforms that use an XML DOM structure • Term rewriting, such as a transformation rule

23. Current State-of-the-Art:SourceWeave.Net • Uses Microsoft’s CodeDom Architecture as the underlying AST Representation • Uses XML descriptor to specify interaction between aspects and .Net components • Limited availability of CodeDOM providers • CodeDOM biased towards • C#, hence lack of expressiveness for other languages • Verbosity of XML representations a serious concern for scalability http://www.dsg.cs.tcd.ie/dynamic/?category_id=438

24. Aspect Crosscutting Statements (XML) Behaviour (.NET Assembly) Component (.NET Assembly) Weave.NET Woven Component (.NET Assembly) Current State-of-the-Art:Weave.Net • Uses existing .Net Binary Component and crosscutting specifications in an xml file as input • Advice is given separately in another .Net assembly • Limited to applications • hosted within .Net • Aspect specification is difficult to comprehend due to lack of expressiveness of XML constructs http://www.dsg.cs.tcd.ie/dynamic/?category_id=194

25. Current State-of-the-Art: Aspect Cobol • Parse trees are exported to XML • Weaving is achieved by XML DOM tree processing in Java • XML representation of source code (intermediate form) may not scale with code size • It has been reported that such internal representations are 50 times larger and much slower to transform • Limited only to COBOL • A more generalized approach is missing http://homepages.cwi.nl/~ralf/AspectCobol/

26. Current State-of-the-Art: Aspicere + GCC 4.0 • Proposes to use GCC 4.0 GENERIC trees • Weaving is done by modifying the GENERIC • AST representations • Lack of support for .Net Languages like C#, VB.Net and other languages like Object Pascal, Cobol etc, currently supports only C • Still in a proposal stage, not clear whether API support is available for modifying GENERIC trees • Very little documentation available from project site http://users.ugent.be/~badams/aspicere/

27. Overview of Approach Model-Driven Engineering (MDE) Technical Space (TS) Grammarware TS Grammarware TS EBNF EBNF KM3 M3 Aspect.gGrammar Aspect Metamodel RSL Metamodel RSL.g Grammar M2 Source Program Myaspect.ap AspectPascal Myaspect Model MyRsl Model MyRsl.rsl RSL program M1 Transformation System Extraction Transformed Aspect2RSL Transformation Injection Program PT Based Back-End Current State-of-the-Art Techniques Program Transformation Back-End Model-Driven Front-End Model Transformation GenAWeave in Action

28. Program Transformation Back-End Source Program parser definitions Parser AST Graph Symbol Table Analyzer Domain Definitions unparser definitions Program Transformation Rules Target Program Pretty Printer DMS Transformation Engine

29. Program Transformation based Aspect Weaving (Language-specificweavers) Experiment conducted on support utilities for a commercial distributed application written in Object Pascal Several aspects identified • Processing dialog meter • Updating of progress meter  Appears in 62 different places of the schema migrator • Exception handling of meter • Logging of SQL query statements The methods of the logging object are invoked in over 50 different places in the schema migrator • Synchronization in database error handler The addition of this concurrency concern resulted in a manual invasive change to over 20 classes. • Dirty Bits in language internationalization utility  This appears in 29 unique places in the language internationalization source code

30. default base domain ObjectPascal. external condition func_sig_has_click(id1:IDENTIFIER,id2:IDENTIFIER) = 'func_sig_has_click'. pattern advice(slist:statement_list) : statement_list = "if EditMadeDirtyBit then SaveDBControls; \slist". pattern isClick(id:IDENTIFIER): IDENTIFIER = id if func_sig_has_click(click(), id). pattern click (): IDENTIFIER = "Click". rule probe_dirty_bit (id1:IDENTIFIER, id2:IDENTIFIER,fps:formal_parameters, slist:statement_list): implementation_decl -> implementation_decl = "procedure \id1 . \isClick\(\id2\) \fps ; begin \slist end;" -> "procedure \id1 . \id2 \fps ; begin \advice\(\slist\) end;". if ~[modslist:statement_list .slist matches "\:statement_list \advice\(\modslist\)"]. publicruleset applyrules = { probe_dirty_bit }. Object Pascal Example (define func_sig_has_click (lambda Registry:MatchingCondition (let (;;(= [const_string (referencestring)] (AST:GetString arguments:1)) (= [search_string (reference string)] (AST:GetString arguments:2)) (= [scanner StringScan:Scan ] (StringScan:MakeScan search_string)) );; (value (while (== (StringScan:End? (. scanner)) ~f) (ifthenelse (StringScan:MatchString? (. scanner) const_string) (return ~t) (StringScan:Advance (. scanner)) )ifthenelse )while ~f )value )let )lambda )define // The user wants to perform another search procedure TLangMan.SearchAgainClick(Sender: TObject); begin // Perform an update if an edit occurred that //might change the focus of the listview if EditMadeDirtyBit then SaveDBControls; … end;

31. Justification of PT Back-End • A mature PTE (e.g., DMS) can ease the construction effort for weavers of legacy languages by offering a direct solution to Challenges C1 and C2. • A PT model offers complex Join Point Shadows (e.g., nested conditional statements) and a rich pointcut to join point binding. • PTEs offer powerful pattern matching and efficient tree traversal strategies (e.g., using visitors over ASTs) that can scale to several million lines of code. • PTEs offers internal APIs that enable transformation of ASTs in an arbitrary manner, thereby allowing more complex and flexible transforms required by legacy applications (e.g., aspects and loops). • In contrast to the verbose AST representation in XML-based approaches, DMS provides internal data structures (e.g., hypergraphs) to represent the underlying AST. This offers an improved level of optimization to support parsing and transforming large legacy applications.

32. Challenges using PT Engines • The rewrite rules used to modify base programs are difficult to compose, which makes it accessible to only language researchers (accidental complexities, Challenge C3) • The transformation rules are generally hard to comprehend by average software developers (need to know about the grammar and semantics of the underlying PTE, Challenge C3) • The transformation rules are tied to the grammar of the base language (Challenge C4) Moreover…. • The entire weaver is rendered unusable if the base transformation engine is replaced with another one (interoperability problem) • The PT engine may be proprietary, i.e., may not be available for use by all desired parties (e.g., DMS)

33. Overview of Approach Model-Driven Engineering (MDE) Technical Space (TS) Grammarware TS Grammarware TS EBNF EBNF KM3 M3 Aspect.gGrammar Aspect Metamodel RSL Metamodel RSL.g Grammar M2 Source Program Myaspect.ap AspectPascal Myaspect Model MyRsl Model MyRsl.rsl RSL program M1 Transformation System Extraction Transformed Aspect2RSL Transformation Injection Program PT Based Back-End Current State-of-the-Art Techniques Program Transformation Back-End Model-Driven Front-End Model Transformation GenAWeave in Action

34. Model-Driven Front-End - Justification • Modularize the weaver construction process by decoupling the model of the high-level aspect language from the model of the target program transformation language (RSL). • The source aspect metamodel need not be altered even if one chooses to opt for a different target PTE, only a new PTE metamodel needs to be developed. • Conversely, for every new language, one needs to add the appropriate metamodel extensions to the base aspect metamodel, but no change to the target metamodel is needed. • Both the aspect language (source) and rules language (target) can evolve independent of each other. This leads to new features being added to the weaver.

35. Metamodel for Aspect Pascal

36. KM3 and TCS Specification

37. KM3 and TCS Specification

38. Support for Generic Front-End (Aspect Pascal)

39. Support for Generic Front-End (Aspect FORTRAN)

41. Target Metamodel (RSL)

42. Overview of Approach Model-Driven Engineering (MDE) Technical Space (TS) Grammarware TS Grammarware TS EBNF EBNF KM3 M3 Aspect.gGrammar Aspect Metamodel RSL Metamodel RSL.g Grammar M2 Source Program Myaspect.ap AspectPascal Myaspect Model MyRsl Model MyRsl.rsl RSL program M1 Transformation System Extraction Transformed Aspect2RSL Transformation Injection Program PT Based Back-End Current State-of-the-Art Techniques Program Transformation Back-End Model-Driven Front-End Model Transformation GenAWeave in Action

43. Model Transformation Model-Driven Engineering (MDE) Technical Space (TS) Grammarware TS Grammarware TS M3 EBNF EBNF KM3 Aspect.gGrammar Aspect Metamodel RSL Metamodel RSL.g Grammar M2 Myaspect.ap AspectPascal Myaspect Model MyRsl Model MyRsl.rsl RSL program M1 Extraction Injection Aspect2RSL Transformation M1 = model level; M2 = metamodel level; M3 = meta-metamodel level

44. ATL Transformations rule Aspect2Rsl { from s: Aspect!Aspect to t: Rsl!Rsl ( domain <- s.domain, pattern <- s.advice, rule <- s.pointcut, ruleset <- rs ), rs: Rsl!RuleSet ( rsname <- s.aname, rname <- s.pointcut-> collect(e|e).pctname), … } rule AfterAdvice2Pattern { from s : Aspect!AfterAdvice to t : Rsl!Pattern ( ptoken <-'statement_list', ptext <- s.advStmt.stmt ... ), ... } Core transformation library : call, exec, loop, with, withincode (i.e., one transformation for each type of pointcut) • The transformations generate the corresponding RSL rule • for the given aspect

45. Generalizing the Rule Generator Design generic_advice_call ( { program_root_ Fortran90_program}, { method_id_ Name}, { proceed_ Name}, { before_advice_ execution_part_construct_list}, { after_advice_ execution_part_construct_list}, ) Fortran90_program generic_advice_call ( { program_root_ ObjectPascal}, { method_id_ IDENTIFIER}, { proceed_ IDENTIFIER}, { before_advice_ statement_list}, { after_advice_ statement_list}, ) ObjectPascal

46. Support for Reusable Back-End Functions • To support back-end construction, GenAWeave provides a reusable library of external functions that can be used to construct part of the low-level weaving infrastructure for a given weaver • name_ends_with- useful for matching identifiers (e.g., function name) whose name ends with a given input. This is equivalent to a wildcard search *name in an aspect program • name_begins_with - useful for matching names (e.g., function name, identifiers), which begin with the given input. This is equivalent to a wildcard search name* in an aspect program • GetChildFromParent - helper routine useful for finding a child node with a given property from the parent node • GetParentFromChild - helper routine useful for finding a parent node with a given property from the child node

48. Overview of Approach Model-Driven Engineering (MDE) Technical Space (TS) Grammarware TS Grammarware TS EBNF EBNF KM3 M3 Aspect.gGrammar Aspect Metamodel RSL Metamodel RSL.g Grammar M2 Source Program Myaspect.ap AspectPascal Myaspect Model MyRsl Model MyRsl.rsl RSL program M1 Transformation System Extraction Transformed Aspect2RSL Transformation Injection Program PT Based Back-End Current State-of-the-Art Techniques Program Transformation Back-End Model-Driven Front-End Model Transformation GenAWeave in Action

49. GenAWeave in Action Input Source Program Generic Aspect Metamodel Parser definitions Lexer/ Parser Symbol Table AST Graph Generic Model Transform Library Aspect Source Metamodel RSL Metamodel Domain Reader Analyzer Unparser definitions Transformed Target Program Pretty Printer Aspect Source Model RSL Model Aspect Source Program Concrete RSL Transform DMS Program Transformation Engine ATL Model Transformation Engine Program Transformation Back-End Model-Driven Front-End

50. GenAWeave in Action Input Source Program Generic Aspect Metamodel Parser definitions Lexer/ Parser Symbol Table AST Graph Generic Model Transform Library Aspect Source Metamodel RSL Metamodel Domain Reader Analyzer Unparser definitions Transformed Target Program Pretty Printer Aspect Source Model RSL Model Aspect Source Program Concrete RSL Transform DMS Program Transformation Engine ATL Model Transformation Engine Program Transformation Back-End Model-Driven Front-End