Transactional Middleware

1. Transactional middleware, which includes TP monitors and application servers, provide benefits to distributed computing and EAI that cannot be found in traditional development tools. Scalability, fault tolerance, and an architecture that centralizes application processing are hallmarks of transactional middleware. In addition, transactional middleware has the ability to provide virtual systems and single logon capabilities and in many cases, has the ability to reduce the overall system cost. Also it ensures delivery of information from one application to the next and supports a distributed architecture (see Fig. 8.1) However, the cost of implementing transactional middleware may preclude its use with EAI. Transactional Middleware

2. Fig. 8.1 Transactional Middleware

3. There are some limitations to what transactional middleware can accomplish within most of the EAI problem domains. None of the TP monitors or application servers that exist today support either “out-of-the-box” content transformation or message transformation services, at least not without programming. What transactional middleware excels at is method-level EAI or, at least the sharing of common business logic to promote EAI. But Message Brokers or traditional, message-oriented middleware are much better tools for the simple sharing of information at the data level or the application interface level. Transactional middleware requires that complex applications be divided into small size units called transactions. Transactional middleware controls transactions from their beginning to their end, from the client to the resource server and then back again.

4. Fig. 8.2 Using Transactions to tie together back-end users

5. The ACID Test A transaction has ACID properties: Atomic refers to “all or nothing” quality of transactions. In other words the transaction either completes or aborts and there is no middle ground. Consistent refers to the fact that the system is always in a consistent state, regardless of whether or not it completes the transaction. Isolated refers to the transaction’s ability to work independently of other transactions that may be running in the same TP monitor environment. Durable means that the transaction, once committed and complete can survive system failures.

6. Scalable Development Transactional middleware processes transactions on behalf of the client or node and can route transactions through many diversified systems, depending on the requirements of the EAI problem domain. It also provides load balancing, thread pooling, object recycling and the ability to automatically recover from typical system problems. Database Multiplexingis one of the main benefits of transactional middleware. It has the ability to multiplex and manage transactions that result in the reduction of the number of connections and processing loads that larger systems place on a database. With transactional middleware in the architecture, it is possible to increase the number of clients without increasing the size of the database server. By funneling the clients’ requests, transactional middleware removes the “process per client” requirements (see Fig. 8.4)

7. Fig. 8.4 Database Multiplexing In this scenario, a client simply invokes the transaction services that reside on the TP monitor, and those services can share the same database server connections (threads and processes)

8. Load Balancing When the number of incoming client requests surpasses the number of shared processes that the system is able to handle, other processes start automatically. Some transactional middleware can distribute the process load over several servers at the same time dynamically or distribute the processing over several processors in multiprocessing environments. This load balancing feature of transactional middleware also enables transactional middleware to prioritize. Fault-Tolerant  Transactional middleware was built from the ground up to provide a robust application deployment environment with the ability to recover from any number of system related problems. Transactional middleware provides high availability by employing redundant systems.

9. The transactions work through a two-phase-commit protocol to ensure that the transactions complete and to guard against transactions becoming lost electrons when hardware, operating systems, or network fail. Communications Transactional middleware provides a good example of middleware that uses middleware, which includes message brokers. It communicates in a variety of ways, including RPCs, distributed dynamic program links (DPLs), inter-process communications, and MOM. Because transactional middleware is simply an API within an application, developers have the flexibility to mix and match all sorts of middleware layers and resource servers to meet the requirements of an application or EAI problem domain.

10. XA and X/Open The standards that define how transactional middleware functions are the International Standards Organizations (ISO) and X/Open’s Distributed Transaction Process (DTP) specifications. The outcome was XA Interface which defines how a transaction manager and a resource manager (such as a database) can communicate. The XA Interface is a set of function calls, split between the transaction manager and the resource manager. Among the features XA provides are those functions provided by the transaction manager, which allow the resource manager to tell the transaction manager whether it is ready to work with a transaction or whether it is in a “resting state” and unable to respond.

11. Building Transactions Building transaction services simply requires that you programmatically define the functions that the service will perform when accessed by the required TP monitor APIs. Application Servers They not only provide a location for application logic, they also coordinate many resource connections. Application servers were created for web based transactions and application development, but their usefulness to EAI is obvious, given the back end integration capabilities (their ability to bind together several source and target applications through a series of connectors provided by the applications server vendors-for example SAP, PeopleSoft, relational databases and middleware)

12. Fig. 8.5 Application Servers

13. Evolving Transactions The benefit of applications servers is clear. By placing some, or most, of the application logic on a middle tier, the developer can insert increased control over the application logic through centralization. Such a placement increases the ability to scale the application through a set of tricks, such as database multiplexing and load balancing. The end result is a traditional 3-tier client/server computing model. Fig. 8.6 Application Architecture, using an application server

14. Enterprise Java Beans Application servers are different from traditional transactional middleware (such as TP monitors) in that they are able to function around the notion of transactional components. Almost all applications servers using this transactional component type architecture are looking to employ EJB as the enabling standard. The EJB specifications define a server side component model for JavaBeans. EJB represent specialized JavaBeans that run on a remote server EJB look very similar to distributed objects such as COM and CORBA at least from the point of view of architecture. Using the same architecture as traditional JavaBeans, EJB can be clustered together to create a distributed application with a mechanism to coordinate the processing that occurs within JavaBeans.

15. Fig. 8.8 EJB define server-side component model The EJB model supports the notion of implicit transactions. The EJB model defines the relationship between an EJB component and EJB container system. The EJB execution system is called “EJB Server” and it provides a standard set of services to support EJB components.

16. Future of Transactional Middleware TP monitors will remain the dominant transactional middleware because they provide the best mix of standards, scaling features and reputations Traditional TP monitor environments, as well as the new line of Java-enabled application servers, will provide to be the best place for EJB. Integration problem domains that need to support transactions between applications, share logic as well as data, and are willing to incur cost of changing all connected applications, are perfect for transactional middleware-type solutions.

17. Message-Oriented Middleware (MOM) and RPCs based middleware tend to be traditional, point-to-point middleware. Because of that, they are not effective solutions for most integration projects. Middleware itself is a common point of integration for source and target systems. Despite the limitations of this technology for EAI solutions, the appropriate tradeoffs in both technology and product decisions may make the inclusion of this middleware important to a particular integrationproblem domain. RPCs RPCs are a method of communicating with a remote computer, in which the developer invokes a procedure on the server by making a simple function call on the client RPC & Messaging

18. Fig. 9.1 Using RPCs

19. The client process that calls the function must suspend itself until the procedure is completed. This blocking of an application might become a problem for the performance of that application. Also there is a high level of network communication between the client and the server when using RPCs, thus the RPC architecture requires a high-speed network. RPCs were the base middleware layers of the early client/server systems. In addition to many database-oriented middleware layers, they are capable of running network operating systems, such as Sun’s NFS (Network File System) and DCE (from Open Software Foundation) Distributed object technology, such as COM and CORBA, leverage RPCs effectively in order to provide object-to-object communications.

20. DCE DCE is complex (but effective) middleware solution. It was developed by OSF and it makes many diverse computers function as a single virtual system – excellent for distributed computing. Fig. 9.2 The DCE Architecture

21. Because the heart of DCE is a classic RPC mechanism, every limitation inherent in RPCs - including blocking and the need for a high speed network – also exist in DCE. But, DCE allows developers to reach across platforms to access many types of application services, including database, distributed objects, and TP monitors. DCE also provides a sophisticated naming service, a synchronization service, a distributed file system, and built-in network security. Another plus is that DCE is available on almost any platform. MOMs If the required bandwidth is not available for use with an RPC middleware, or in the situation where a server cannot be depended upon to always be up and running, message-oriented middleware (MOM) is more advisable than any other type of middleware.

22. Fig. 9.3 Using MOMs

23. Like RPCs, MOM provides a standard API across hardware, operating system platforms and networks. It is also able to guarantee that messages will reach their destination, even when the destination is not available (meaning the server is not on line, the application is not started, application crushed, etc) MOM utilizes one of the two macro-messaging models Process-to-Process both the sending and receiving processes must be active for messages to be exchanged Message Queuing allows messages to be stored in a queue, so only one process needs to be active (the sender process). This method is the most beneficial when communication is taking place between computers that are not always up and running, over networks that are not always dependable, or when there is a limitation on bandwidth.

24. Unlike RPCs, MOM is asynchronous, thus it doesn’t block the application from processing when the middleware API is invoked. MOM message functions can return immediately even though the request has not actually been completed. This allows the application to continue processing, assured that it will know when the request is completed. The queuing model is the most useful from transactions-oriented EAI applications that must transverse multiple platforms. Unlike DCE, the platform does not have to be up and running for an application to request services. If the server is down the request remains in queue. When the server comes back online, the request will be processed. In addition to these features, MOM provides concurrent execution features, allowing the processing of more than one request at the same time.

25. In conclusion, MOM is a good fit for store-and-forward communications or when dealing with applications that are not expected to be reachable at the same time. MOM is also good for “defensive” communication, or when the network between applications fails a lot. Also it’s a good option for when communications between processes need to be logged. MOM and messaging tend to be better EAI fits than RPC based middleware, but MOM alone does not provide all the infrastructure necessary for EAI. Message Brokers add value to traditional MOM products by providing data and schema transformation function, as well as intelligent routing and event-driven processing to move information on an enterprise network. (see Fig. 9.4)

26. Fig. 9.4 Message Brokers using MOM

27. Microsoft Message Queue (MSMQ) It’s a Windows NT/2000 based and COM enabled MOM. It is also a key product for Microsoft as it joints the Microsoft Transaction Server (MTS), Internet Information Server (IIS), SQL Server, and Microsoft’s Active Platform. MSMQ provides a set of Active X components that implements an API to all MSMQ features. Active X allows these other Microsoft products to access MSMQ. MSMQ can guarantee the delivery of messages by utilizing disk based intermediary storage and log-based recovery techniques. Some of the protocols MSMQ works well with are IPX/SPX and TCP/IP. Using MSMQ as a common translation mechanism these protocols can even be mixed and matched. MSMQ is tightly integrated with MTS and MTS services participate in the MSMQ transaction.

28. Some of the MSMQ features are: On time, in order delivery of messages from source to destination Message-routing services, which give the EAI applications the ability to send messages to their destination using least-cost routing Notification services, which informs the sender that the message was received and processed. Message priorities, which allow developers to assign priorities to messages. Journaling, which maintains copies of the messages moving in the system.

29. Using MSMQ When an application needs to place a message in the queue, it uses the MSMQ Open API, which has an Active X control wrapped around it. By passing in, the name and destination queue, a queue handle is created, allowing the MSMQ to identify the destination queues to the MSMQ server sending the message. Once a queue handle has been created, the next step is to create a message by allocating local memory and adding information to the message. At this step, parameters (time-out values, names of response queues, priorities, etc) can be added. The message is then sent through the MSMQ “Send API”. Finally the application closes the queue handle using the MSMQ “Close API” function (MQCloseQueue). The receiver application requires the Open API, along with queue identification information, to create the queue handle. When calling the MSMQ Receiver API, MSMQ will pass back a pointer with control information to the message.

30. After receiving the information, the application will close the connection to the queue. Fig. 9.5 The Architecture of MSMQ

31. IBM MQSeries IBM has over 65% of the middleware market and although is not the best of the message middleware software, the IBMs MQSeries is at the top of the EAI world as the preferred messaging layer for moving information through an enterprise network. MQSeries support a variety of platforms working with IBM OS/390, Pyramid, Open VMS, IBM AIX, NCR UNIX, Solaris, Windows NT, MacOS, etc. MQseries supports all the features of MSMQ, including support for transactions, journaling, message routing and priorities. It also provides a common API for use across a variety of development environments and on a variety of platforms. The interesting thing is that MQSeries provides a single multiplatform API, so messages can be sent from a Windows 2000 computer and routed via UNIX, VMS, etc, before reaching their final destination.

32. MQSeries can also be used as a transactional recovery method and a mail transport mechanism, assuring the delivery of the message. Fig. 9.6 MQSeries MOM

33. MQSeries supports what IBM calls “advanced message framework”, a framework that consists of 3 layers: Customer Application Options Trusted Messaging Backbone Comprehensive Communications Choices Fig. 9.7 The 3 Layers of MQSeries

34. Customer Application Options provide services such as basic messaging, transactional messaging, workflow computing, mail messaging, option messaging, cooperative processing, data replication and mobile messaging. Trusted Messaging Backbone guarantees that messages are delivered to their destinations and that the information encapsulated in those messages is secure. Comprehensive Communications Choices allow MQSeries to leverage any number of protocols and networks for message traffic. MQSeries Features Database-message resource coordination (DMRC) Smart message distribution Supports NT/2000 & OS/2 SQL support Ability to support large message