Code generation tools

1. Code generation tools Galit Keret Moti Zviling

2. Contents • Why is code generation technology required? • What is it and what kind of a solution it offers? • CGN – Code generation network • Features of a good code generation tool • Related terminology • Active and passive code generation • RAD and CASE • Code generation products • How to find the right tool for your needs? • Code generation models • Code samples for each model • Partial Products list for each model • So why is it not vastly used? • Demo code generation tools

3. Sometimes it’s a bummer to be a software engineer… Ever felt like this? Writing thousands and thousands of tedious code, your deadline is impossible to reach (as always), and in the end you found yourself facing dozens of bugs and worse than that - you discover your application’s performance is poor… code generation tools

4. Some scary facts about software world (from OOD course) • 84% of software projects are not on time • 31% of software projects never complete • ~60% of completed code is never used • ~200 Billion $ a year lost to software bugs Let’s say this in words: • Most software is buggy, unstable and insecure • A lot of software is totally unusable Introducing a new concept: Code generation tools. code generation tools

5. What is “Code generation”? • Strictly speaking: Code generation is the technique of building code not manually but using other programs. • You simply need to define a specification of abstract requirements to the tool. Using templates, it will then build one or more output files based on your requirements . • Saves you the need to write the tedious parts or parts that are not in the areas of your expertise. code generation tools

6. CGN – Code generation network Covers all aspects of code generation: Includes interviews with engineers using and building code generators as well as articles, tutorials and example code. Holds an alphabetically database of generators and a list of books about code generation. code generation tools

7. Features of a good code generation tool • Quality - Generated code always increases in quality over time because as bugs or shortcomings are found they can be fixed in the code templates and applied across the code base. • Consistency - Generated APIs are consistent in class structure, variable naming and signatures. • Productivity – • Generators build code in a fraction of the time that it takes an engineer to produce the equivalent amount of code. • Generators allow the engineer to offload grunt-work so that they can concentrate on tasks that require more creative solutions. • Save testing time – don’t need to test the generated code. • Abstraction - Generators provide a layer of abstraction between the design and the code base. The definition files used by these generators hold the schema and business rules for your application in a declarative form. code generation tools

8. Related terminology:Active and passivecode generators • There are two types of code generators: • Active generators - intended to be run multiple times on the same output code as the design of the input or generator changes. • Passive generators - create the initial code, and then it is the responsibility of the developer to maintain it (example: a wizard). • Although both types will provide an initial boost to productivity, active generation is superior , because in this type of code generation the code is maintained and bugs in the templates can be rolled out across the whole code base. code generation tools

9. Active and passive code generation workflow • The standard workflow in software engineering: edit-compile-test. • The workflow cycle of a passive generator: run the generator, then follow the edit-compile-test cycle repeatedly on the code created by the generator. • The workflow cycle of an active generator: run the generator, then compile, edit and test the output. Found a problem in the generated code? alter the templates or input of the generator and re-generate. Found a problem in the hand-written code portion? follow the standard edit-compile-test workflow. code generation tools

10. Related terminology:RAD and CASE • CASE - computer assisted software engineering • Similar to code generation tools. • The analyst could define the business logic using visual tools and a generator would build all of the application code. • RAD – Rapid application development • An approach to building computer systems which combines CG/CASE tools and techniques, user-driven prototyping, and stringent project delivery time limits. • The typical RAD project delivers a fully functional computer system in just a few days! code generation tools

11. CASE v. Code generation tools • In both methods a generator build an application from an abstract description. • The difference is in the approach: CASE tools attempt to replace trained engineers with tools that do the same work. CG tools are built by engineers for themselves – to make their life easier. • Both tools can end up in the same place! code generation tools

12. Some facts about code generation products • There are almost 200 code generation tools in the code generation network, and this list is far from being complete. Yet only few engineers know about code generators and fewer still have used them. CGT claims that the reason for this is lack of advertisement and clear information about it. • The most common uses of code generators are database access and UI generators. • There are more generators written for Java users than all of the other technologies combined. code generation tools

13. Finding the right code generatorfor your needs • The code generation decision tree will help you look for the right generation tool for your project. • If none of the generators listed address your needs you can use one of the ‘custom code generators’ which supports scripting in a variety of languages. • Failing that, you can always write your own generator…(just be sure to first read the DO IT YOURSELF section at the Code generation network) code generation tools

14. Code generation models • Code munger • Inline code expander • Mixed code generator • Partial class generator • Tier generator • Domain language code generation tools

15. Code Munger Model • The most common form of code generators • The code munger reads the source file, analyzes it, and creates one or more output files. • Processing Flow: code generation tools

16. A code munger model example –a translator • A translator takes a code written in some programming language and translates it to a code written in some other language. • The input and output of a ‘C to Java’ translator will look like this: output class Wrapper { char c; int x; int foo(int y, char d) { int i; for (i = 0; i < 3; i = i + 1) { x = 3; y = 5; } } } Input char c; int x; int foo(int y, char d) { int i; for (i=0; i<3; i=i+1) { x=3; y=5; } } code generation tools

17. Some Code Munger Model products • JavaDoc: reads and parses the comments within a Java source code file, then creates HTML documentation. very easy to learn and use. • ANTLR: helps to build programming language translator. Generates recognizers in Java, C++, C#, and soon Python. • C2J: a C/C++ to Java translator. The generated code is not understandable! • Jazillian: a C to Java translator. The generated code is natural (as if it was written by hand), but there are some minor translation bugs. (presented in demo) code generation tools

18. Inline Code Expander Model - overview • It starts with designing a new language. Usually this new language is an existing language with some syntax extensions. The inline code expander is then used to turn this language into production code in a high level language. • Processing Flow: • The input is a source file for a new language. • The output is a source file in a high level language suitable for compilation. code generation tools

19. An Inline Code Expander example: C with embedded SQL • Embedded SQL is a method of combining the computing power of a high-level language like Java or C/C++ and the database manipulation capabilities of SQL. It allows you to execute any SQL statement from an application program. • In the example code, SQL is embedded into C source code files using special markup. These hybrid C files are given their own extension, such as '.sqlc‘: // main.sqlc int main( int argc, char *argv[] ) { < SQL select * from users > return 0; } code generation tools

20. An Inline Code Expander example - continued After running the generator the corresponding created C file will look like this: #include "db.h“ int main( int argc, char *argv[] ) { DBHandle *db_handle = db_connect(); DBQueryDara *db_data = db_db_query( db_handle, "select * from users" ); for( int record = 0; record < db_data->length; record++ ) { // fetch and process data } return 0; } code generation tools

21. Some Inline Code Expander products • Pro*C (Oracle) - Allows embedding of SQL into C source code. It takes some time to learn the Pro*C syntax, which is not intuitive, and it’s difficult to use in complex operations and also the generated code is not object oriented. Only 2 links in Google refer to this technology which means it’s not a success story. • SQLJ (Oracle) - Allows embedding of SQL into Java source code. a more intuitive syntax and some what more object-oriented approach to the Pro*C way of embedding SQL statements in programs, but still the recommendation is to use it only when the data being mapped is quite simple (don’t use it in case of multiple mappings of Java objects to tables and vice versa) code generation tools

22. Mixed Code GeneratorModel • The same processing flow as the Inline Code Expander, except that the input file is a real source file that can be compiled and run. • This technique can be used for a variety of purposes: • Building the user interface glue code to attach variables to user interface controls. • Building test case code from test case data stored in comments. • Creating database access code from access specification stored in comments. code generation tools

23. Mixed Code Generator : example an example input file for the SQL handling mixed code generator: #include "db.h"int main( int argc, char *argv[] ) { // SQL: select * from users // SQL end return 0; } Mixed Code Generator After running the generator it will fill the space between the comments with an implementation of the requested SQL statement (it will look like the output showed in the previous example). The comments are preserved so that if the generator is run again, the same replacement can be done. code generation tools

24. Mixed Code Generators v. Inline Code Expanders • a major disadvantage of an Inline Code Expanderis that the debugging is done against the output file. This means that you need to integrate any changes you make to the output code back into the input file after debugging is finished. • In mixed code generation model the source code can be debugged directly (the input and output files are the same). They can also integrate nicely into IDEs as external tools. code generation tools

25. Mixed code generators v. Other generators • Most generators build or manage entire files or even generate a whole application tier. • This creates a barrier to entry of using code generators on existing products which don't take well the fact of having large sections of code replaced completely. • Mixed code generation tools provide genuine aid in developing both new and existing applications, by enabling code generation for multiple small code fragments within a single file or distributed throughout multiple files. code generation tools

26. Some Mixed Code Generator products • Codify -A code generator that integrates into the Visual Studio .NET framework which can generate multiple code fragments within a single file. For example: can generate the code for get/set methods of class properties. code generation tools

27. Partial Class Generator Model • A partial class generator builds a set of base classes that are used as a platform to build the production classes. • The base classes are designed to do the majority of the low level work of the completed class, leaving the derived class free to override specific behaviors on a case-by-case basis. code generation tools

28. Partial Class Generator – processing flow The generator reads a definition file, then using a set of templates, builds one or more base classes. These base classes are then augmented with derived classes All of the classes are then compiled together to build the final product. code generation tools

29. Partial class generator – database access classes • An typical use of a partial class generator is to build database access classes. • The generator builds the basic persistence code for each class and its fields. The derived classes are responsible for any custom behavior. code generation tools

30. Some Partial Class Generator products • Expert Coder– a toolkit that supports the creation of code generators • Codegen - an open source (LGPL), UML Class Diagram to C#, Java and VB.NET code generation framework. code generation tools

31. Tier Generator Model • A tier generator builds and maintains an entire tier within an application. • Processing flow: The tier generator takes an abstract definition file and using a set of templates builds one or more output files that constitute all of the functionality for an entire tier of the application. code generation tools

32. Example of a tier generation tool • a good database/web application is built as a n-tiers application (e.g., database access, business logic and user interface). • Most of the tier generators builds the database access tier or the user interface tier of such applications. • There are so many CG tools that generate the database access tier for database applications that they even got a special name: O/R mapping tools (“Object to Relational mapping”) code generation tools

33. Tier generation example (continued) • O/R mapping tool - a code generation tool that connects to your database and reads its schema, then lets you map objects to database tables and views, specify insert, update, load, and delete operations, queries and stored procedure calls, as methods to these objects. • It also lets you define the relationships between objects (one-to-one, one-to-many, etc.) based on relationships between tables in the database. • It then generates fully working persistence objects code for you. code generation tools

34. Tier generation example (continued) An O/R mapping tool processing flow: code generation tools

35. Tier generation example (continued) Below is an example of code a user will write, using an O/R mapping tool: • try { Employees objInfo = new Employees(); • EmployeesFactory objFactory = new EmployeesFactory(); • objInfo.EmployeeID = EmployeeID;objFactory.Load(objInfo); // code here to use the “objInfo” object} catch(Exception ex) { // code here to handle the exceptionreturn;} code generation tools

36. Some Tier generation products • Tier developper – an O/R mapping tool that runs on .NET platform. Very easy to learn and use. Excellent demos are available on site. • XLInesoft ASPRunner Professional – Creates a set of ASP pages to access and modify MS Access, MS SQL, Oracle and other databases. Also very easy to learn and use but has less configuration options and generates less languages than ‘Tier developper’. (presented in demo) • BrightSword™ Designer – build database-driven web applications in ASP, ASP.NET, JSP and PHP. Not user friendly as ASPRunner and the trial version crashes. code generation tools

37. Tier generators Vs. Partial class generators • Tier generators are harder to write: • Tier generator maintains all of the tier’s code, including all edge cases. • Partial class generator follows the 80/20 rule: it handles 80% of the standard cases, and the rest 20% is handled by customized derived classes. • But tier generators are better: • Better business rules abstraction • Easy to port the application to other platforms • The result: generators often start as partial class generators. As the code solidifies and the problem domain becomes better understood the 20% become smaller and smaller until the whole tier is auto-generated. code generation tools

38. A domain specific language model • The tool provides the user with a new language that has types, syntax and operations that map directly to the concepts of the user’s domain. • Examples: Mathematica and Matlab These languages make it easy for the scientist to represent constructs that are difficult to code in general purpose languages, such as multi-dimensional matrix math. • Another example, although not as clean as the pervious example because it obligates the user to describe her domain in terms of classes and relationships between classes is MDA. code generation tools

39. MDA - Model-Driven Architecture • MDA is a set of related standards specified by the Object Management Group (OMG). • These standards: UML, XMI, MOF, OCL, CWM are used to turn a model written in Unified Modeling Language [UML] to engineering artifacts in the form of source code, or documentation. • MDA is targeted mainly at the professional production languages: Java, C#, and C++. code generation tools

40. MDA : the 3 tiers model • The MDA 3-tiers model for code generation: • How is it done? Using XMI • XMI is an XML based export standard for UML models. • The platform-independent UML model is maintained in the Meta Object Facility (MOF) repository and exported via XMI to the generator. • The generator then reads the XMI and applies transformations to it to create the platform-specific model. This model is then used as input to a set of templates which build the output code. Platform independent model (PIM) Platform specific model (PSM) Templates Code code generation tools

41. MDA processing flow code generation tools

42. MDA processing flow and the 3 tiers model code generation tools

43. Some MDA products • IBM Rational Rose - a popular UML to C/C++ generator. Undo/redo operations on the UML diagrams are not supported. • BridgePoint – a UML to C/C++ generator. A good product but no free trial version. • UModel by Altova – a UML to Java and vice versa generator. Very easy to learn and use, supports undo/redo operations (presented in demo) BridgePoint code generation tools

44. If it’s so great than why aren’t we using it? 8 traditional arguments against using code generation: • Why not just use cut and paste? • Code generation doesn’t work on existing code bases. • The quality of the output code is bad. • Generators are too complex. • Generators are great in the beginning, but they are never maintained (Engineers will ignore the 'do not edit' comments). • My application isn't complex enough for generation. • My application is too complex for generation. • Code generation will eat my job . code generation tools

45. Summary of tested products Legend: Recommended Worth trying don’t try code generation tools

46. Products summary (continued) code generation tools

47. Products summary (continued) code generation tools

48. Demo Part code generation tools

49. Types of code generation • A reminder: Code generators are programs that automatically generate high level code (e.g. C, C++, C#, Java, Perl, Python, Ruby, etc.). • List of code generation applications: http://www.codegeneration.net/generators.php code generation tools

50. Code generation – decision tree code generation tools