TransWarp and the Future of ZPatterns

1. TransWarp and the Future of ZPatterns IPC 9 Zope Track

2. Ambiguous Praise For ZPatterns • "... if I could understand what it does I'd probably use it." - Brad Clements • "... everytime I've come across a technology I had trouble understanding, it turned out to be a worthless technology or a good one where the authors had trouble explaining their stuff. For the time being, I've placed ZPatterns in the latter category" - Cees de Groot

3. Praise For ZPatterns at IPC 9: • “It’s certainly made my life more… interesting.” • Steve Spicklemire • “We had been taught the value of commenting code in our computer classes, but this experience [with ZPatterns] really made us see why it is important to do so.” • Lex Berezhny & Jason Straw

4. Issues with ZPatterns • Difficult to teach/grasp • Tedious development: same names retyped • As ZClass properties • As SQL fields in queries and DDL • As form text and field names • In SkinScript • etc. • Inherently a hack, promoting further hacks ad infinitum

5. Our Quest for Automation • Write once, use many • Metadata was the key • UML is standard for metadata, XMI for interchange of UML • But no UML or XMI tools for Python • Code generation from XSLT was ridiculous • Had to create our own system

6. Introduction Components: The Future (And Present)Of Programming

7. Where did ZPatterns come from? • Ty and PJE needed a component model for Zopelications • Zope component capabilities were poor or non-existent for: • Distributing components without upgrade breakage • Configuring components to work with components outside their own “product” • Separating data storage from logic and UI code

8. Concepts/Assumptions • Objects are domain objects, representing real-world things • Real-world things have data stored about them in many databases • For a given piece of data, there may be many “conceptual” objects • Thus, a useful framework must allow many-to-many mappings between physically stored data and conceptual objects

9. Therefore… • An object’s attributes should be able to be stored in any database, without forced one-to-one mappings • There should be a way to create or access instances of objects, knowing only their interface, not their implementation class or storage mechanisms

10. The RIPP Model • Not tool-specific or implementation specific, but a generally reusable architecture pattern • A way of structuring Zope or ZPublisher applications as: • Reusable, configurable components, with • Separation of application logic from storage and user interface code, and • Separation of implementation class choices (variable) from application roles (fixed)

11. RIPP Overview

12. Component Architectures • Design Patterns/Idioms are key to an effective component architecture: • What components should exist? • How is the work divided among them? • How are they put together to form larger structures? • The RIPP patterns are a first step in this direction - TransWarp is the next

13. TransWarp: It’s Not Just Components Any More (It’s component architectures, which means patterns, which means repetition, which means automation!)

14. The “Why” of TransWarp • “Easy as 1-2-3” • Draw applications with UML • Add application-domain logic • Mix with implementation frameworks + =

15. The “What” of TransWarp

16. What is AOP? • Separately Specified Areas of Concern • e.g. Persistence, User interface, domain logic • Ability to “weave” these areas into a single program • Modular replacement of implementation for each area in a given application

17. Generative Programming (GP) The Automation Assumption:“If you can compose components manually, you can also automate this process.” • Czarnecki and Eisenecker, “Components and Generative Programming”

18. Plan + Parts = Products

19. “Structural Aspect” • The aspect of a component that describes its class hierarchy and “structural features” • Structural feature = Attribute, Association • Can be derived directly from UML model • Reusable with different implementations of domain logic, storage, user interface

20. Structural Models • Structural model provides implementation for a structural aspect • Defines classes to be used for “attribute”, “associationEnd”, etc. • Standardized storage interface for model updates and queries, including predicates • Based on the “Services, Elements, and Features” pattern

21. Services, Elements, And Features

22. Cool Tools CASE, UML, XMI, SQL…It’s 100% Buzzword Compliant!

23. TransWarp CASE • UML Model Library • Built using a StructuralAspect representing the UML MetaModel • Full StructuralModel features available: • use predicate queries • create new interfaces for UML model manipulation • your choice of storage mechanisms • XMI Library

24. Objects + SQL = Warp CORE • CORE = Customizable Object-Relational Environment • Pattern language for storing “business objects” in a relational database • Conceptually well-suited to RIPP model • Handles aggregation, composition, temporal relationships, inheritance, polymorphism • Highly efficient design using few tables and well-chosen indexes

25. CORE = A Design Pattern • Pattern language for database design, not persistence mechanism for arbitrary objects • (although you could implement one using CORE as a base) • Not suitable for retrofitting existing databases - it is for designing new ones • Does not require any special tools - we’ve designed and built databases by hand with it

26. The CORE Tables

27. Using CORE with TransWarp • Generate databases from UML • Stereotypes and/or tagged values used to identify event classes, other pragmas • Weave CORE storage aspect with structural model to create an app - no SQL or coding required for standard retrieves and updates • Put domain-specific queries (e.g. “getTasksForParty”) together in one aspect for easy reference/maintenance

28. So What’s Going ToHappen To ZPatterns? (I’m so glad you asked me that… That’s an excellent question…)

29. Zope Directions • Component-driven architecture • Interface-oriented • More Python-centric • Things are more “explicit” • Content management framework to be seperated from “pure” application server • Less need for weird hacks

30. DataSkins: Stupid Zope Tricks • Implementing storage separation in a through-the-web configurable way turned out to be very hard in Zope • Zope 2 makes lots of internal assumptions: • ZODB-based persistence • Object-based (not action-based) transactions • Local roles stored as dictionary of lists • Standardized web-based management GUI

31. Whither goest ZPatterns? • Now on maintenance-only track • Many former ZPatterns functions will probably “disappear” into Zope core: • PlugIns may be replaced by a more interface-savvy management UI • Factory mechanisms may allow straightforward creation of objects without reference to a Specialist, allowing fewer Specialists to be used in many applications

32. Even if there is a “Z3Patterns”… • Will only implement RIPP model • SkinScript, Providers may go away • Primary development focus will be on TransWarp, which will be much more useful and powerful for future Zopes

33. So When Can WeUse TransWarp? (I had a feeling that you were going to ask me that…)

34. TransWarp Project Status • AOP tools available today • GP, UML, and XMI tools being ported from pre-AOP library • WarpCORE patterns have been used for production systems, but automation not developed. • Solid automation requires the GP, UML, and XMI tools to exist first

35. Licensing & Contributions • License: BSD-style, minus any advertising or credit requirements. No warranty. • Contributions Wanted: “Horizontal” aspect libraries (UI, DB/persistence, logging, testing, documentation generation, code generation, etc.)

36. TransWarp Links • TransWarp Documentation Wiki http://www.zope.org/Members/pje/Wikis/TransWarp • TransWarp Software Releases http://www.zope.org/Members/pje/TransWarp/