The Ladder To Abstraction

1. The Ladder To Abstraction Matthew Fowler matthew.fowler@nte.co.uk

2. Agenda • Introduction, Background • Techniques • Pluggable Meta-models • Logical Modelling • Externalisation of Control • Model-to-model Transforms (Patterns) • Two-dimensional inheritance • Framework approach • Conclusion

3. The Ladder Of Abstraction • Chunking up • Higher levels of abstraction • More power of thought and action • Modelling - the more abstract the better/faster • Removal of detail • Relativity: logical and physical • Chunking down: recreating the "lost" information • extract from super-classes • multiple outputs per concept

4. JeeWiz Introduction • 2001: BEA: • "Stop customers messing up Architecture" • Pre "MDA"/"MDD" • Goal: Express & automate • the integration programmers do • best practice use of frameworks • Large-scale end-to-end system generation • UI generation a difficult challenge • Challenges => new concepts and techniques

5. Background 1. Pluggable Meta-models • Velocity as templating engine • Required Java model objects as "the context" • Classes for Java Objects = meta-model • Problem: construction and maintenance Templates Velocity Model Java model objects Text class Entity field name="Customer" <entity name="Customer">

6. Challenges Pluggable Meta-models • 1. Maintenance Size of meta-model (Java) code Difficult to add new features • 2. How to do .NET, new frameworks • But re-use surrounding pieces

7. Pluggable Meta-models Solution 1. Meta-models • Define in "Simple XML" • Generate Java classes for model obects • 75% of meta-class code generated • Very simple constructs: (cf EMF) • <meta-model>, <meta-class> • <property>, <list>, <validator> • c-data: Java impl, expressions, descriptions • 320 meta-classes in all JeeWiz meta-models • (Sometimes we don't bother with MMs)

8. Example Pluggable Meta-models <meta-class name="entity" extends="business-object" description="..." > <property name="table-name" /> <property name="generate-table-schema" type="boolean" default="true"/> <list name="query" description="Select and finder methods." > <validator message="An entity must have at least one attribute" expression="getAttributeList().size() > 0" /> // getKeys gets either the autoKey or the primary key attributes list public List getKeys() { initLists(); return ( autoKeyAttributeList.size() > 0 ) ? autoKeyAttributeList : keyAttributeList; } </meta-class>

9. Pluggable Meta-models Solution 2. Pluggability Company Company • Multiple meta-models per build • Current record -13 meta-models in a stack • Late binding: configure stack at generate-time • Modular • Abstract/Physical separation WebSphere JSF J2EE Spring Screen Screen Bizobject Bizobject Java Java

10. Cross-MM Inheritance Pluggable Meta-models Class Reference Type Java Meta-model Business Object Meta-model Impl Class Business Object Entity J2EE Meta-model EJB Entity Entity JDO Entity

11. Benefits Pluggable Meta-models • Concept inheritance • building up the ladder of abstraction • as far as you need to go • go beyond classes (e.g. page components) • inheritance through multiple meta-classes/models • Modularity and reusability • Pattern for addressing new technologies/processes

12. Challenges 2. Logical Modelling • Modellers & UML Tools • One set of concepts (stereotypes) • Need to remove concepts (e.g. 'physical' detail) • e.g. public/protected/private in Java • fixed for business objects and business methods • <business-method> => public access • Fixups for modelling tool deficiencies • e.g. constraints in RSA/RSM • Customer modelling requirements

13. Background Logical Modelling • Additional models • Typically used as control information • Additional models normally "Simple XML" Templates Model Java model object tree Artifacts AdditionalModel(s) Added as second-level object in tree

14. Solution Logical Modelling • Modular profile construction • No limit on # meta-models/meta-classes • Adaptation to UML tools and customer reqts. Screen Business Object Java CustomisedMeta-model, ready for - modelling tool - customer envt. UML Tool Specific Profile Screen Business Object Java Filter Spec Aggregate Filter Convert

15. Benefits Logical Modelling • Separation of concerns • Aggregation • Filtering (& compensation for UML tool deficiencies) • Filter customisation by end user • Easy to produce a "logical profile" • Any level of abstraction • Reduction in number of concepts and detail • General - any combinations of meta-models • separate adaptation to modelling environment

16. Interlude • That was the easy bit - building the abstractions • Now for the interesting bit: • Bridging the semantic gap to operational systems • Expansion: adding in the 'physical' • Derivation: PIM1 → PIM2 concepts • Variation: adapting to customer requirements • User Interface generation • Incorporating "Best Practice" • Smart use of the technical platform

17. Challenges 3. Externalisation of Control • Component assembly from model objects • Build automation • Move the workflow into the engine • "Externalisation of control" • Number of configuration properties • (Let's face it) We don't like build/configuration in Ant

18. Solution Externalisation of Control • A feature-rich framework for MDD • External workflow • initialisation (overall and per model object) • model-to-model transforms and validation • " model-to-artifact" (i.e. create various artifacts using Ant, deploy etc.) • Pull in features by well-known names, e.g. • "component.properties" sets model object props • "build.xml" builds a model object using Ant • "uptodate.xml" and avoiding rewriting

19. Benefits Externalisation of Control • Model-to-model transforms • Created model objects produce output • No additional config / selection required • Automatic expansion of model object tree (x 30) • Revisit later • Complete generation systems as components • use automatically in a larger build • e.g. Java Jar build within overall application build • e.g. model-driven deployment solution

20. Challenges 4. Model-to-model Transforms • Building DotNet and J2EE • Too many similar M2Text transforms • One for .NET, another for J2EE • Only M2Text available, mixed together • intention (give me some data behind a page) • implementation (here's a class generation) • Aspects difficult to do • e.g. change class tracing at one point • Jan 03

21. Operation Model-to-model Transforms • Transforms in templates with well-known names • Produce XML text as output • Embellishes, or creates new, model objects • "Pattern" - practical application of design pattern Velocity Scripts Model toModel Model to Text Model Modelobjects Artifacts + <xml ...>

22. Sample Pattern Model-to-model Transforms • Entity: fix up for missing keys • Various types of patterns: • "Fixup" - fix myself or my peers, before validation • Create other objects #if( $keys.size() == 0 ) <this> <attribute name="oid" autokey="true" type="Integer" /> </this> #end

23. Details Model-to-model Transforms • Separation of concerns • intention = original model objects • elaboration = generated model objects • implementation - downstream artifacts produced • "<page>" different on Struts/JSP/Spring MVC • Do it in place - enhance the input model • Completely automatic; regen every run • No update by developers • This is where architecture is generated • Tinkering by developers is a bad idea!

24. Cross-Tier/MM Patterns M2M Transforms Entity DAO Attribute DAO attribute Relation DAO reln Pages Search/Select/Edit MaintenanceSession DAO ButtonsCRUD

25. Benefits Model-to-model Transforms • Implement higher concepts via lower ones? • Expansion • half of the "fill in the physical part" solution • Key to "brains in a box" • executable design patterns • create architecture, e.g. derived tiers, logically • Understandable by developers • no new languages to learn; easy to change • Intermixing of logic and output architecture

26. Background 2D Inheritance • Many existing features for variation • properties files for system init • properties files per model object • links to Java • But for composability... we need more

27. The Challenge 5. 2D Inheritance • Reuse of "Renderings" • Higher-level concepts build on lower-level impl • Customer does too - naming, logging, security, L&F • Variant Injection - easy insertion into lower layers • Java Inheritance is not enough • Variant generation at similar logic levels • C#, Java • .NET, J2EE (some similarities) • Struts/JSF (different)

28. 2D Inheritance Solution 1 • Generalise meta-model stack • "Domain-specific generation plug-in" per-level config, namingconventions Company per-level macros WebSphere J2EE Screen Object Type Bizobject Ant build.xml config, namingconventions transforms Java methods

29. Details 2D Inheritance • Operation • Search top-down • Merge - add new names (inheritance) • Override - existing names • Externalisation of control  • create well-known name-files at any level • only linkage is configuration of stack at build time

30. Per-object inheritance 2D Inheritance • Second level of inheritance... groundwork: Class Reference Type Java Meta-model Business Object Meta-model Bean Impl Class Bus. Object Entity New EJB Entity Entity J2EE Meta-model JDO Entity

31. Bean model-to-model 2D Inheritance <this access="private"> #foreach( $field in $fieldList ) #if( $field.readable ) <method name="get${field.nameCapitalised}" return-type="${field.type}" access="public" > return ${field.name}; </method> #end #if( $field.writeable ) <method name="set${field.nameCapitalised}" return-type="void" access="public" > <parameter name="${field.name}" type="${field.type}" /> this.${field.name} = ${field.name}; </method> #end #end </this>

32. Solution 2 2D Inheritance • We need two dimensions of inheritance • Model object type (bean → class) • Generation plug-in (BOM → Java MM) • Details • Separate from meta-class (Java) inheritance • Under control of meta-programmer • 'BOM:bean' says: "now try 'class'" object type • Redirections search the whole stack of plug-ins • Override at multiple layers/concepts

33. Benefits 2D Inheritance • Incremental creation of generation plug-ins • Use of "inherited" plug-ins • Above M2M example works for Java or C# • Plug-in replacements - Struts/JSF/SpringMVC • Variant injection • Change small details in a lower-level plug-in • e.g. "is()" for beans • Flexible approach to climbing down "the ladder"

34. Challenge 6. Framework Approach • Encapsulating is hard • Architecture, design patterns, best practice • Sheer volume • 1000's of meta-programs

35. Solution Framework Approach • Many other features/refinements • Non-model "controls" for generating variant text • In Java, Velocity or both • OO in Velocity - methods, inheritance, overriding • 180 methods • Framework (externalisation of control)

36. Benefits Framework Approach • Less volume in the meta-programs • Easier to • Express best practice and architecture • See the wood for the trees • Change

37. Conclusion • Feature Checklist • Pluggable Meta-models, Logical Modelling • Externalisation of Control • Model-to-model Transforms (Patterns) • Two-dimensional inheritance • Framework approach • Relevance • More powerful frameworks • less need for simple codegen • more need for • integration / best practice / architectural generation