Agile Development with Domain-Specific Languages

1. Agile Development withDomain-Specific Languages 19th June 2006 Aali Alikoski, Microsoft Steven Kelly, MetaCase

2. DSM with full code generation: the ultimate refactoring tool? 19th June 2006 Aali Alikoski, Microsoft Steven Kelly, MetaCase

3. Schedule Part I: What’s it all about? • 14:00-14:15 Welcome & Introductions • 14:15-14:45 Agile & DSM introduction & definitions • 14:45-15:05 Software Factories & DSL Tools demo • 15:05-15:15 MetaEdit+ demo Part II: OK, I can do that! • 15:15-15:30 Interactive TV Applications Domain • 15:30-16:00 Coffee • 16:00-17:00 You build your first DSM Language! • 17:00-17:20 Final topics, Conclusions & Wrap-up

4. Contents • Introduction • What is domain-specific modeling (DSM) • DSM examples & case studies • Benefits of DSM • How to implement domain-specific methods • Modeling language • Generators

5. How has productivity improved? • "The entire history of software engineering is that of the rise in levels of abstraction" Grady Booch • New programming languages have not increased productivity • UML and visualization of code have not increased productivity • Abstraction of development can be raised above current level... • ... and still generate full production code (and ignore it!) *Software Productivity Research & Capers Jones, 2002

6. Map to code, implement Assembler Map to code, implement Code Generate,Add bodies Map to UML UML Model No need to map! Generate callsto components Modelin DSMlanguage Components Modeling domain vs. modeling code DomainIdea FinishedProduct Solve problem in domain terms

7. What is domain-specific modeling • Captures domain knowledge (as opposed to code) • Raise abstraction from implementation world • Uses domain abstractions • Applies domain concepts and rules as modeling constructs • Narrow down the design space • Focus on single range of products • Lets developers design products using domain terms • Apply familiar terminology • Solve the RIGHT problems • Solve problems only ONCE! • directly in models, not again by writing code, round-trip etc.

8. Map to code, implement Assembler Map to code, implement Code Generate,Add bodies Map to UML UML Model No need to map! Generate callsto components Model in DSM language Components Example: Digital wristwatch DomainIdea FinishedProduct • Product family • Models: His, Hers, Sport, Kid, Traveler, Diver… • Reusable component applications • Time, Alarm, Timer, WorldTime, StopWatch… • Hide complexity from modeler • Model-View-Controller separation • Separate thread for real-time display • Implementation in Java Solve problem in domain terms

9. Case1: Insurance products & eCommerce • Developing portal for insurances and financial products • Need to specify several hundred financial products • Insurance experts specify visually insurance products and generate code to the portal • Comparison to writing directly Java after first 30 products = DSM at least 3 times faster

12. Case2: Configuration of services • Telecom services and their configuration • Users visually specify new configuration models • Generate various configurations from single design • One model • Multiple outputs • Reusable component library • Code generators refers to external files

15. Case3: Web application • Web application for e-commerce; product catalogs, events, press releases, and discussion forums • Core components and basic functionality available for reuse and customization needs • Each customer can specify own data content, behavioral logic and user interface • Code generators produce running Java applets, stylesheets and xml files • Generation of documents for both internal and external use

18. Voice VoiceMenu for micro controller based home automation system Remote control for lights, heating, alarms, etc. VoiceMenus are programmed straight to the device with assembler-like language (8bit) Modeling language to define overall menu structure and individual voice prompts Code generator produces 100% of menu implementation Development time for a feature from a week to a day! Case4: VoiceMenu for micro controller

21. Symbian/Series 60 for enterprise application development Platform provides basic services Modeling language to define application logic using basic widgets and services Code generator produces 100% of implementation Complete chain from model to running app Libraries possible to integrate Case5: Enterprise apps in Smartphone

24. Case6: Call Processing Language • Specify services than can run safely on Internet telephony servers • Designs can be considered valid and well-formed already at the design stage • Language use concepts familiar to the service developer • Switches, Locations and Signaling actions etc. • Generate full service from the model • There are also cases where the language has been extended to cover also domain extensions and new requirements e.g. for Java and VoiceXML.

27. Let’s look at industry experience:Some reported cases • Nokia; Mobile Phone product line • Bell Labs / AT&T / Lucent; 5ESS telecommunications switch, • Honeywell; embedded software architectures • ORGA; SIM toolkit & JavaCard • Pecunet; B2B E-Business: insurance • LexiFi; mlFi, financial contracts • DuPont; Activity Modeling • NASA; Architecture Definition Language • NASA ASE group; Amphion • NASA JPL; embedded measurement systems • USAF; Message Transformation and Validation • … • See more at www.DSMForum.org

28. DSM Case Study: Nokia • DSM and related code generators for mobile phone* • Order of magnitude productivity gains (10x) • "A module that was expected to take 2 weeks... took 1 day from the start of the design to the finished product" • Focus on designs rather than code • Domain-oriented method allows developers to concentrate on the required functionality • Training time was reduced significantly • “Earlier it took 6 months for a new worker to become productive. Now it takes 2 weeks” * MetaCase, Nokia case study, 1999

29. DSM Case Study: Nokia Tetra • DSM for Tetra terminals* • Earlier development was based on SDL and manual coding • Faster development • ”MetaEdit+ has clearly boosted development speed due to its capabilities for supporting extensive reuse and managing variation” • Quality of product code improved • ”DSM rules out errors, eliminating them already in the design stage” • Hiding complexity made development easier • ”Developers with less C code experience can effectively develop features using DSM” * MetaCase, Nokia case study, 2003

30. DSM Case Study: Lucent • 5ESS Phone Switch and several DSMs * • Reported productivity improvements of about 3-10 times • From several cases • From several DSMs • Shorter intervals between product releases • Improved consistency across product variants • “DSM should always be used if more than 3 variants” * D. Weiss et al, Software Product-Line Engineering, Addison-Wesley

31. DSM Case Study: USAF Development of message translation and validation system (MTV)* • Declarative domain-specific language • + code generators and customization of components Compared DSM against component-based development: • DSM is 3 times faster than code components • DSM leads to fewer errors: about 50% less • DSM gives “superior flexibility in handling a greater range of specifications” than components * Kieburtz et al., A Software Engineering Experiment in Software Component Generation, ICSE’96

32. Why Domain-Specific Modeling? • Development becomes faster • less to ”write” as abstraction is raised • no need to model and then code too • Development becomes easier • lower level details hidden (cf. Assembler to C) • models easier to read, understand and remember • Expertise can be leveraged • domain to code mappings made only once – by the expert developer • Routine tasks minimized • Developers can focus on the problem, not its implementation details

33. Why it is possible (now)? • Need to fit only one company’s requirements! • Modeling is Domain-Specific • Works for one application domain, framework, product family etc. • Language has concepts people already are familiar with • Models used to solve the problem, not to visualize code • Generator is Domain-Specific • Generate just the code needed from models • Efficient full code • No manual coding afterwards • No reason for round-tripping • Generator links to existing primitives/components/platform services etc. • Can produce Assembler, 3GL, object-oriented, XML, etc.

34. Where to apply? • Repetitive development tasks • Large portion of the work similar to earlier products (or several products made in parallel) • Domain expertise needed • Non-programmers can participate • These normally include: • Product Family • Platform-based development • Configuration • Business rule definitions • Embedded devices

35. Map to code, implement Assembler Map to code, implement Code Generate,Add bodies Map to UML UML Model No needto map! Generate callsto components Modelin DSM language Components Modeling domain vs. modeling code DomainIdea FinishedProduct Solve problem in domain terms

36. Done a few times before! Expert (few) DSM language Code generator Domain framework How to implement DSM DomainIdea FinishedProduct Easy! Generate callsto components Model in DSM language Normal (many) Components

37. Tool support is essential • Building DSM must be fast, cheap and easy • A variety of tools available • Lex & Yacc for textual DSLs • Customizable IDE (e.g. Eclipse, Visual Studio) • MetaCASE tools (e.g. MetaEdit+) • 5 ways to get the tools • Write own tool from scratch • Write own tool based on frameworks • Metamodel, generate tool skeleton, add code • Metamodel, generate full tool • Integrated modeling and metamodeling environment

38. Schedule Part I: What’s it all about? • 14:00-14:15 Welcome & Introductions • 14:15-14:45 Agile & DSM introduction & definitions • 14:45-15:15 Software Factories & DSL Tools demo • 15:05-15:15 MetaEdit+ demo Part II: OK, I can do that! • 15:15-15:30 Interactive TV Applications Domain • 15:30-16:00 Coffee • 16:00-17:00 You build your first DSM Language! • 17:00-17:20 Final topics, Conclusions & Wrap-up

39. Difference from MDA? • Same idea on using models and transformations, but... • DSM is always full code direct from models • Not OMG MDA (elaborationist) • Simpler in terms of versioning and management • DSM = domain-specific language and generators • MDA is UML-based* • No reverse- or round-trip engineering in DSM • We want a real lift in the level of abstraction • How often do you reverse engineer assembler to code? • Separation of concerns • You are the experts in your domain and code (not the vendor) • DSM is agile: as much or as little as you want * official definition, www.omg.org

40. How to implement DSM • Expert developer defines the DSM, others apply it • expert always defines the domain better than less-experienced developers • Always better to define the concepts and mappings once, rather than let everyone do it all the time • Identify your current concepts and rules • Formalize them • No need to learn new semantics and notations • Models easier to read, remember and check • Map to implementation framework • Generators: • Call component interfaces • Fill template code • Map to different target platforms

41. Implementation covers • Initial: • Metamodel: concepts and rules of the language • Notation: symbols and their behavior • Tool behavior: dialogs, icons, browsers etc. • Generators: for code, checking, inspection, docs etc. • Language help • Connectivity with other tools • Continuously: • DSM language or tool sharing • DSM updates (of metamodel, rules, notation) • Generator updates • Model updates based on changed language • …often in multi developer settings

42. Steps for implementing DSM Rules Generators 3 4 1 2 Symbols Concepts

43. 1. Design domain concepts • Map modeling concepts accurately to domain concepts • What information is stored with each concept • Concentrate on semantics! • Add extensions for software production later • Design to make reuse precise and easy

44. Metamodeling with GOPRR • Concepts • Graph • Object • Property • Role • Relationship • Rules, e.g. • Max 1 • Min 1 • Unique, etc • Ports for connections • Links between models • Reuse rules

45. 2. Define domain rules • Define semantics and rules as they exist in the domain • Examples of rule types: • Links between concepts • Layering abstractions • Reuse rules • etc.

46. 3. Draw symbols (notation) • Define symbols illustrating as well as possible the corresponding domain concept’s natural ”visualization” • e.g. end-user’s notation, customer’s notation

47. 4. Implement generators • Reporting language fast and easy to use: no programming • Cost of developing generators defrayed over a few users • Write report definitions based on domain concepts: • Component use – Configuration data • Code generation – Testing and analysis • Automated build – Reports that make reports • Documentation – Review

48. Apply in software production • Develop applications using the DSM infrastructure • 5-10x faster! Cf. NASA, Nokia, Lucent, USAF, Pecunet • Continuously evolve your DSM • Domain & platforms evolve • Models update automatically

49. Code generator principle • Generator translates the models into a required output • Crawl through the models  navigation according to metamodel (modeling concepts and rules available) • Extract required information  access data in models • Translate it as the code  translation semantics and rules • Using some output format  possibility to define output format

50. Symbian/Series 60 application development Platform provides basic services Modeling language to define application logic using basic widgets and services Code generator produces 100% of implementation Complete chain from model to running app Libraries possible to integrate Python for smartphones