Understanding Message Brokers in Advanced Web Applications

1. Message Brokers and WorkflowsAdvanced Web Applications DevelopmentTechnion CS 236606 Spring 2003, Class 10 Netta Aizenbud-Reshef

2. Outline • Message Brokers • MQ Integrator • Workflow • MQ Workflow • The Future?

3. Message Brokers

4. Message-Oriented Middleware (MOM) - Reminder • Asynchronous communication:The sender and receiver do not have to be available at the same time in order to communicate. • Message: “a package of business data that is sent from one application to another over the network. Self-describing – contains all the necessary context to allow the recipients to carry out their work independently” • Fault tolerance • Load balancing • Scalability • Transactional support

5. Publish/Subscribe Messaging - Reminder • Topic – “destination” (virtual data channel) • Producer is a “publisher”Consumer is a “subscriber” • Publishers and subscribers are generally anonymous

6. Point-to-Point Messaging - Reminder • Queue: virtual data channel (“destination”) • Producer is a “sender”Consumer is a “receiver”

7. Enterprise Application Integration - Reminder Legacy System E-Mail gateway JMS Client JMS Client JMS Client JMS Server Vendor A JMS Server Vendor B JMS Client JMS Client JMS Client JMS Client Internet (http) Gateway JMS Client

8. Message Broker • A broker among one or more target entities (such as a network, middleware, applications, and systems) • Integrates multiple business activities (applications) whether they are new, old, legacy, centralized, or distributed • Ties together many different platforms and application development solutions • Facilitates the integration of business processes • Main goal - provide a simple, central point of integration

9. Message Broker - Motivation • Traditional applications no longer have value by themselves • Organizations want to hook new applications/systems (ERP, CRM) to their traditional applications • No common technological frameworks – very costly integration

10. Message Broker – Advantages • Minimal changes to applications (systems) • Faster implementation and more flexibility • A middleware that adapts to the applications, not vice versa

11. Message Broker – Structure • Common components of message brokers: • Message Dictionary • Data Translation Layer (DTL) • Deconstructs messages • Reconstructs data into new message • Rules engine • Content based routing decisions • Boolean logic • Adaptors to external systems • Repository • Stores copies of all exchanged data • Message source, message format, target system, target system format

12. Message Broker vs. MOM • MOM • Provides basic communication • Message persistence and guaranteed delivery • Solves only part of the problem • Message brokers • Build on top of existing middleware technology • Provides enterprise application integration

13. Application B Application C Application A Message Broker DTL Messaging API Messaging API Messaging API Rules Engine Messaging Clients Messaging Clients Messaging Clients Repository MOM

14. Message Broker Products • WebSphere MQ Integrator, IBM • BusinessWare, Vitria • Financial Fusion (Sybase) Message Broker • Helio Message Broker, HelioGraph

15. WMQI – WebSphere MQ Integrator • Supports multiple environments • Transforms, augments and applies rules to message-based data • Routes and distributes data between systems • Supports topic-based publish/subscribe functions • Visualizes the application flow through a GUI • Allows message formats to be defined • An open framework which allows the use of built-in components together with third-party offerings

16. WMQIArchitecture • Built on top of MQSeries • Base messaging and queuing capabilities • Supports 35 platforms

17. Queue Manager Client Applications WMQI Architecture Configuration Manager Broker MQ Integrator Layer Control Center Message Flows Message repository Configuration repository MQSeries Layer Queues

18. WMQI – Message Dictionary • Supports messages from several sources • Predefined • Messages that their content and structure is defined in the MRM (Message Repository Manager) • Supported wire formats • CWF (Custom Wire Format) • XML • PDF • Self defining • Structure, content and format is within the message • JMS, XML • Undefined • Content and structure is undefined • BLOB

19. WMQI – Message Definition • Interface to the MRM • Hierarchical Structure

20. WMQI – Rules Engine • A set of actions, rules • An action is performed by a Message Processing Node • Message Flow • A sequence of message processing nodes • Can be built hierarchically • Provides a particular service • Primitive processing nodes • Receiving and routing messages • Transforming a message to an alternative representation • Selecting a message for further processing based upon the message’s content • Interacting with an external repository to augment a message or store the whole or part of a message • Responding to events and errors • User defined processing nodes • Implement a supplied Interface

21. WMQI - Message Flow Example • Stock Market • NYSEtrade message contains date, company symbol, company name, quote price and total value traded • FTSEtrade message contains the same information as NYSEtrade in the identical format • AMEXtrade message contains date, company name, quote price and amount traded • The information that applications want to acquire • Rationalized trades - all trades in a consistent data format. These are needed for known trading analysis application A • Major trades - rationalized trades with ‘netvalue’ greater than $100,000. These should be logged in the message warehouse and delivered to application C, which connects intermittently to perform historical analysis • PE Update - triggered by earnings reports, computed and added to earning report data and delivered to application B which analyses PE changes • The stream of major trades should be published

22. WMQI - Message Flow Example

23. WMQI – Message Flow Development

24. WMQI – Message Flow Deployment

25. WMQI - Topics and Publish/Subscribe configuration

26. Workflow

27. Workflow Definition • Workflow is a term used to describe the tasks, procedural steps, organizations or people involved, required input and output information, and tools needed for each step in a business process • A workflow engine is the component in a workflow automation program that knows all the procedures, steps in a procedure, and rules for each step. The workflow engine determines whether the process is ready to move to the next step

28. Workflow Example – Purchase Order

29. The Key Benefits of Workflow • Improved efficiency - automation of many business processes results in the elimination of many unnecessary steps • Better process control - improved management of business processes achieved through standardizing working methods and the availability of audit trails • Improved customer service – consistency in the processes leads to greater predictability in levels of response to customers • Flexibility – software control over processes enables their re-design in line with changing business needs • Business process improvement - focus on business processes leads to their streamlining and simplification

30. Workflow Management Systems • Lotus Workflow, IBM Lotus • WebSphere MQ Workflow, IBM • FLOWer, Pallas Athena • Staffware Workflow, Staffware • Verve Workflow, Verve inc. • i-Flow, Fujitsu • InConcert, TIBCO

31. Workflow Definition Languages • BPML – Business Process Modeling Language, BPMI.org • WSFL – Web Services Flow Language, IBM • XLANG – extension of WSDL, Microsoft • UML – Unified Modeling Language, Rational • BPEL – Business Process Execution Language, Based on WSFL and XLANG • XPDL – XML Process Definition Language, WfMC • WSCI – Web Service Choreography Interface, BEA, SAP, Sun

32. The Workflow Management Coalition (WfMC) • A non-profit, international organization of workflow vendors, users, analysts and university/research groups.  • Mission - to promote and develop the use of workflow through the establishment of standards for software terminology, interoperability and connectivity between workflow products. • Contribution - Workflow reference model

33. Workflow – Basic Architecture • Workflow systems provide support in the following function areas • Build time functions – defining and modelling the workflow process and its constituents activities • Runtime control functions – managing the workflow processes in an operational environment and sequencing the various activities to be handled as part of the process • Runtime interactions – with human users and IT application tools for processing the various activity steps

34. Workflow – Basic Architecture

35. Workflow – Build Time Functions • Business Process Definition • Translation from the real world into a formal definition • Use of analysis, modelling and definition techniques • Result • Process Model • Process Template • Process metadata • Process Definition • Comprised of • Discrete activity steps • Associated computer and/or human operations • Rules for the progression of the process through the various activity steps • Can be expressed in text, graphical form or formal language notation

36. Workflow – Runtime Control Functions • Interpretation of the Process Definition • Creation and Control of operational instances of the process • Scheduling various activity steps within the process • Invoking human and IT application resources • Act as the linkage between the process as modelled by the Process Definition and the process as it is seen in the real world • The core component is the workflow management control software • Can be distributed across a number of computer platforms

37. Workflow – Runtime Activity Interactions • Human operations • Invoke application tools • Information processing operations • Pass appropriate data • Transfer control between activities • Ascertain the operational status of a process

38. Workflow Reference Model • Process Definition – specifications for process definition data and its interchange with the workflow execution environment • Workflow Interoperability – interfaces to support interoperability between different workflow systems • Invoked Applications – interfaces to support interaction with a variety of IT application types • Workflow Client applications – interfaces to support interaction with user interface desktop functions • Administration and Monitoring – interfaces to provide system monitoring and metric functions to facilitate the management of composite workflow application environments

39. Workflow Reference Model

40. WebSphere MQ Workflow • Supports multiple environments • Supports long-running business process workflows as they interact with systems and people • Provides integration processes with rich support for human interactions • Allows to bring systems and people into a managed process integration environment for EAI, B2Bi, and BPM solutions • Built to a service-oriented architecture based on standards • Offers deep application connectivity to leverage WebSphere MQ, XML, Java™ 2 Enterprise Edition (J2EE) and Web services

41. Build Time Runtime WebSphere MQ Workflow Architecture Model the Business Process Manage Workflow Execution Perform Operations Worklist

42. WebSphere MQ Workflow Build Time • Graphical editor for process definition • Define staff • Register programs • Define data structures • Define process flow • Activities • Logic connectors • Transition condition • Data connectors – data to be ready for activities

43. WebSphere MQ Workflow Build Time Model

44. WebSphere MQ Workflow Build Time– Define Staff

45. WebSphere MQ Workflow Build Time– Register Programs

46. WebSphere MQ Workflow Build Time– Define Activities

47. WebSphere MQ Workflow Build Time– Define Activities • Define the program that needs to be executed • Define the responsible person/role • Define the duration of the activity • A person to notify on delay • Define the data structures • Input data structure • Output data structure

48. WebSphere MQ Workflow Runtime • Manage Workflow execution • Navigate through the entire process • Assign individual tasks to the right persons • Start linked applications automatically • Notify administrators of delay • Write every action to an audit trail for evaluation purposes • Performed in the background

49. WebSphere MQ Workflow Worklist • A client to perform operations • View the list of activities • Find • Filter • Sort • Start, Interrupt and Resume activities • Transfer an activity from one person’s worklist to another’s

50. WebSphere MQ Workflow Worklist