Integrating IBM WebSphere Application Server and IBM WebSphere MQ

1. M49: Using IBM WebSphere Application Server and IBM WebSphere MQ together Matt Leming lemingma@uk.ibm.com

2. Agenda • Why are we here? • What options are available? • Choosing between them • How to do it

3. Why are we here? • Reasons for being here “We currently use WebSphere MQ as our messaging backbone, we need to access that backbone from our WebSphere Application Server system” “We’re going to be using messaging in WebSphere Application Server, we’re trying to choose a messaging provider and are thinking that WebSphere MQ might fit the bill” …

4. The basics

5. WAS’ messaging in a slide • WebSphere Application Server (WAS) is a JEE application server, JEE includes support for asynchronous messaging using JMS (Java Message Service) • Allows Java application interactions to be loosely coupled • Temporal and transactional decoupling of applications • A standardised programming interface • Implemented by a JMS provider • JMS applications can be coded to be ignorant of the underlying JMS provider • Provider specific configuration, such as how to connect to a messaging provider, is contained within JMS resources held in JNDI, defined at application deployment time • JMS is not a transport protocol • No standardised ability to couple different JMS providers • JMS Supports two messaging models • Point-to-point – using queues • Publish/subscribe – using topics • WebSphere Application Server provides first class support for two JMS messaging providers • An embedded messaging provider, the Default Messaging provider (or Service Integration Bus) • WebSphere MQ as an external messaging provider

6. WebSphere MQ in a slide • WebSphere MQ (WMQ) • WMQ is IBM’s flagship asynchronous messaging product • Queues are managed by queue managers • Queue Managers can be connected together to form a network, or cluster • Messaging applications connect to queue managers to access queues • Applications can send messages to any queue in the network from their queue manager • Applications can receive messages from queues on their queue manager • WMQ supports the point-to-point and publish/subscribe messaging models • WMQ as the JMS provider in WAS • WMQ supports the JMS programming interface for Java applications. • Applications in WAS can use JMS to interact with WMQ queues (and topics) • WAS V6 and V6.1 contains a WMQ JMS client to enable this • WAS V7 contains the WMQ V7 JCA resource adaptor • WAS applications connect to a queue manager using: • Bindings – when application and queue manager are on the same server • Client – when application server and queue manager are on separate, networked, servers - using a TCP/IP socket

7. Service integration bus in a slide • Service Integration Bus (SIBus) • SIBus is a native component of WAS, forming the asynchronous messaging platform for application server based products • Logically: • A bus is a logical entity that contains destinations (e.g. queues/topic spaces) • Messaging applications connect to the bus to access the destinations • A bus is location transparent, all destinations and their messages are available from anywhere in the bus • Physically: • A WAS cell contains application servers, or clusters of application servers • These can optionally be made a member of a bus • Destinations are assigned to bus members • Bus members run messaging engines within the application server JVMs • Messaging engines manage the runtime and persistent state of messages in the bus • Messaging applications form a connection to a messaging engine • Messages are routed from any messaging engine in the bus to the application • SIBus as the JMS provider in WAS • SIBus supports the JMS programming interface for Java applications. • Natively supporting both point-to-point and publish/subscribe messaging models • SIBus is the default JMS provider in WAS – hence the“Default messaging provider”

8. Options

9. Options for accessing your WMQ backbone • Three key options • Use WMQ as the JMS provider in WAS • Use the service integration bus in WAS and connect that messaging system with your WMQ backbone • Use SIBus for intra-WAS messaging, use WMQ for inter-system messaging

10. Existing WMQapplication CICSapplication Option 1 WebSphere MQ as the JMS provider in WAS WAS Application Server Application Server Application Application WMQ QMgr WMQ QMgr WMQ QMgr WMQ QMgr Applications connect directly into the WMQ backbone WMQ backbone

11. Existing WMQapplication CICSapplication Option 2 SIBus for WAS application messaging,An SIBus interacts with WMQ to access the messaging backbone WAS Application Server Application Server Application Server Application Application Application Messaging Engine Messaging Engine SIBus Messages flow between SIBus and WMQ where interactions with the WMQ backbone are necessary WMQ backbone WMQ QMgr WMQ QMgr WMQ QMgr

12. Existing WMQapplication CICSapplication Option 3 SIBus for WAS-WAS application messaging,WMQ for WAS-‘other’ messaging WAS Application Server Application Server Application Server Application Application Application Messaging Engine Messaging Engine Application SIBus WMQ backbone WMQ QMgr WMQ QMgr WMQ QMgr

13. Just a simple matter of choosing…

14. ‘Side by side’

15. Just a simple matter of choosing… When accessing messages on WMQ queues in aWMQ backbone

16. Option 1: WMQ as the JMS provider in WAS When accessing messages on WMQ queues in a WMQ backbone: • Pros • The most direct route to WMQ queues • Best performing way to get messages on and off a WMQ queue • Least moving parts • Least things to configure • Least places where things can go wrong • Simplest problem determination • A single messaging system to maintain rather than two • Cons • WAS application deployment depends on knowledge of WMQ topology • E.g. the WMQ queue location is required when connecting applications • WAS applications dependent on two systems being simultaneously available (application servers and queue managers) • Can be complex to exploit WAS and WMQhigh availability and scalability features

17. Option 2: SIBus messaging system When sending or receiving messages to or from a WMQ backbone: • Pros • Messaging runtime is embedded into the same runtime as the applications • Probably better performing way to get messages produced by WAS applications into WAS applications • Ability to asynchronously queue messages in the WAS system while connectivity with the WMQ backbone is unavailable • JMS resource configuration relates to WAS-defined SIBus configuration objects • WMQ-aware configuration in WAS is minimised • Applications can be unaware of actual queue locations • Cons • Two messaging systems to configure and maintain • Additional complexity in configuring interactionsbetween SIBus and WMQ • Additional problem determination may be required to resolve issues • Performance overhead when passing from one messaging system to another

18. Just a simple matter of choosing… When starting with a JMS messaging provider

19. Option 1: WMQ as the JMS provider in WAS When used as the messaging system for WAS alone: • Pros • ‘Industrystandard’ • Proven to be robust and scalable in very large messaging deployments • Extensive knowledge base • Cons • Added complexity of two different configuration and monitoring systems

20. Option 2: SIBus messaging system When used as the messaging system for WAS alone: • Pros • Messaging configuration is completely integrated into the WAS configuration model • SIBus configuration and runtime concepts are closer aligned with the WAS configuration and runtime concepts • SIBus exploits many WAS Network Deployment features • A centrally managed, federated, configuration model • Embedded high availability of the messaging resources using application server clusters • Cons • Smaller customer base • More limited support through external tooling

21. Option 2…. options! When accessing messages on WMQ queues in a WMQ backbone

22. How to achieve Option 2 • Two ways to connect to WMQ through SIBus: • Asynchronous connectivity to WMQ • Synchronous connectivity to WMQ

23. Asynchronous connectivity to WMQthrough the SIBus

24. QueueManager QueueManager Asynchronous connectivity to WMQWMQ-to-WMQ • WMQ queue managers can be networked together, allowing messages to be asynchronously sent from applications connected to one queue manager to queues located on other queue managers • Store and forward of routed messages • Network connections using channels • Message routing using remote queue managersand remote queues WMQ network app app

25. SIBus network Bus AppServer AppServer AppServer ME ME app Bus AppServer app ME Asynchronous connectivity to WMQSIBus-to-SIBus • SIBuses, which may be a network of bus members, can be networked together, allowing messages to be asynchronously sent from one SIBus to queues located on other SIBuses • Allows SIBus messaging to span multiple WAS ND cells • Store and forward of routed messages • Network connections using links • Message routing using foreign buses and alias destinations

26. Bus AppServer AppServer WMQ network QueueManager QueueManager ME app SIBus network app Asynchronous connectivity to WMQSIBus-to-WMQ • So, it’s just a matter of joining the two together… • The WMQ queue manager sees the connected SIBus as another remote queue manager, connected via a sender and receiver channel, allowing messages to be routed through to the SIBus as for any other connected queue manager. • The SIBus sees the connected queue manager as another bus, connected via a foreign bus connection, allowing messages to be routed through to the queue manager as for any other connected bus. • Messaging applications on either side are not aware of the transition.

27. WMQ network SIBus network Asynchronous connectivity with WMQ • What does this add? • Ability to asynchronously queue messages in the WAS system while connectivity with the WMQ backbone is unavailable • All JMS resources defined are SIBus resources, referring to SIBus objects. • Fine grain authorisation access to those resources can be managed using SIBus authorisation, within the WAS configuration domain. • WMQ configuration is restricted to the SIBus configuration domain. • WMQ topology is hidden from JMS resource configuration • What does it cost? • Extra latency • Only able to send messages to WMQ queues,unable to receive from WMQ queues • WMQ applications must send to queues locatedin the SIBus (using remote WMQ queues) • Additional configuration • Additional runtime management

28. Synchronous connectivity to WMQthrough the SIBus

29. WAS config JNDI QueueManager WMQ network WMQ JMS client ConnectionFactory<QMgr location> AppServer app WMQ JMS JMS Queue<WMQ queue name> Synchronous connectivity with WMQ • When WMQ is the JMS provider in WAS, a JMS application will connect directly to a queue manager • This requires a WMQ JMS ConnectionFactory, defining where the queue manager is and how to connect to it. • The embedded WMQ JMS client in WAS is then used to synchronously connect to the queue manager. • The application will also require a WMQ JMS Queue which defines the actual queue located on that queue manager. • This allows synchronous style messaging with WMQ

30. WAS config JNDI ConnectionFactory<SIBus name> QueueManager WMQ network AppServer JMS Queue<SIBus queue name> app SIBus JMS SIBus config ME WMQ bus member<QMgr location> WMQ SIBus queue <WMQ queue> SIBus synchronous connectivity with WMQ • It is possible to insert the SIBus runtime between the application and the queue manager, such that the JMS application actually connects to the SIBus runtime and the bus manages the synchronous connection to a queue manager when required. • This requires an SIBus JMS ConnectionFactory, defining which bus to connect to, and an SIBus JMS Queue, defining which SIBus queue to send or receive messages from. • The configuration detailing which queues are available on which queue managers, and how to connect to those queue managers is contained in the SIBus configuration. • A connection will then be made to a messaging engine in the bus, which will then make a synchronous connection to the correct queue manager. • This allows synchronous style messaging with WMQ

31. WAS config JNDI WMQ network SIBus config SIBus synchronous connectivity with WMQ • What does this add? • All JMS resources defined are SIBus resources, referring to SIBus objects within the WAS configuration domain. • WMQ configuration is restricted to the SIBus configuration domain. • WMQ shared queue on z/OS can be fully utilised from WAS • Supports transaction recovery across multiple members of the queue sharing group, enabling full shared queue workload balancing and high availability. • What does it cost? • Extra code path, slower performance • Additional overall configuration complexity • Additional runtime management

32. WMQ direct vs. SIBus asynchronous vs. SIBus synchronous • WMQ direct Solution has the least moving parts when trying to access WMQ resources Not a seamless fit with WAS • SIBus asynchronous Fits well with the WMQ asynchronous networking principles Keeps WAS application configuration within their comfort zone Increases latency between WAS applications and WMQ • SIBus synchronous Keeps WAS application configuration within their comfort zone Increases overall complexity when accessing WMQ resources Reduces performance between WAS applications and WMQ Don’t underestimate the importance of your current skill base when choosing your messaging provider

33. Putting this into practice….

34. Using WebSphere MQ asthe JMS provider

35. WMQ Connection Factories Resources > JMS > Connection factories > [New] • Specify how an application connects to a WMQ queue manager • Typically requires: • Queue manager name • Hostname and port • Channel name • Other parameters such as: • Transport type (client or bindings) • Use of SSL • WAS V7 allows the identification of a WMQ client channel definition table (CCDT) URL • This removes the need for most of the above configuration • Prior to V7, a CCDT can be specified through custom properties • WAS V7 added the option to try ‘bindings, then client’ as a transport type

36. WMQ Destinations Resources > JMS > Queues / Topics > [New] • Defines references to the resources in WMQ that a JMS application will use • The WMQ resources must be created using WMQ administration • Queues • Identifies the actual queuein WMQ • Can be used to set propertiessuch as persistence,priority, etc. • Topics • Defines the WMQpublish/subscribe properties • Can include details of adurable subscription • Requires knowledge of WMQpubsub broker configuration

37. WMQ Activation Specifications Resources > JMS > Activation Specifications > [New] • Message-driven beans (MDBs) are a special class of EJB, they enable JEE applications to asynchronously process messages, with WAS managing the transactionality and concurrency of the application. • Activation specs are the standardised way of defining and activating an MDB • The WebSphere MQ messaging provider in WAS V7 adds support for activation specs • Activation specs combine the configuration of connectivity, the JMS destination to be processed and the runtime characteristics of the MDB itself • Activation specs can be defined at all WAS configuration scopes, as can be done for ConnectionFactories and Destinations.

38. ListenerPorts Servers > Server Types > WebSphere application servers > serverName > Message listener service > Listener ports > [New] • Prior to V7, activation specifications are not available when using WMQ. Instead, MDBs are enabled by an application server’s message listener service and its associated listener ports • Just like an activation spec, a listener port is bound to the JMS destination that an MDB wishes to receive messages from • It also identifies a ConnectionFactory to usefor connecting to WMQ • Listener ports can only be defined at server scope. • WAS V7 provides a wizard to migrate listener ports to activation specs

39. Using connectivity withWMQ via the SIBus

40. Using asynchronous connectivity withWMQ via the SIBus

41. Asynchronous connectivity to WMQ • The point where WMQ and SIBus configurationand runtime come together is within the SIBus’WebSphere MQ link • Prior to V7, two separate pieces of configuration are required • One foreign bus – defined on a bus • One WebSphere MQ link – defined on a messaging engine • V7 brings these together under the umbrella of a foreign bus connection, and adds a wizard to take you through its complete configuration • A WebSphere MQ Link can be further subdivided into an WMQ link sender and an WMQ link receiver(surprisingly similar to WMQ sender and receiver channels!), which require: • Channel names • Queue manager location (TCP/IP address and port) • Security settings • The link runs in a single messaging engine • The link has access to all resources in the bus for incoming messages, and all members of the bus have access to the link for outgoing messages

42. Configuring a foreign bus connection in WAS V7 Service integration > Buses > BusName > Foreign bus connections > [New] 1. Decide which messaging enginewill own the link 2. Provide the WMQ specific details enter the details including the send/receiver channel details 3. (or 0) Configuring the equivalent sender and receiver channels on the WMQ queue manager

43. Asynchronous connectivity to WMQ Servers > Server Types > WebSphere application servers > serverName > Transport Chain • Just as a WMQ receiver channel requires an active channel listener on the queue manager to enable it, an SIBus WMQ link receiver requires an enabled inbound transport chain on the application servers that the link may run on • The InboundBasicMQLink and InboundSecureMQLink transport chains are automatically enabled on such servers • Inbound transport chains areassociated with a port, the WMQsender channel should point tothat port • Secure inbound and outbound transport chains may be used tosecure the channels using SSL

44. The overall state of the link The current number of messages waiting to be transmitted Indoubt batches of messages can be detected and resolved The link can be started and stopped The messages can be viewed, deleted or moved to an exception destination Asynchronous connectivity to WMQ System integration > buses > BusName > Foreign bus connection > name > WebSphere MQ Links • WMQ stores messages on an XMITQ prior to sending them down a sender channel. Similarly, SIBus stores messages on a Sender channel transmitter • WAS V7 added direct administration of these transmitters, allowing viewing and management of messages currently awaiting transmission on the transmitter

45. Asynchronous connectivity to WMQTranslation guide Below is an approximate mapping between things that do similar jobs in SIBus and WMQ when connecting an SIBus to a WMQ queue manager network • There is actually considerable overlap between certain objects, for example a WAS inbound transport chain and a WMQ channel listener both listen for incoming traffic on a TCP/IP port, but the transport chain can also contain the SSL configuration for a link, where as that would be configured on a channel in WMQ.

46. Cluster bus member configured to allow MEs to failover to alternative servers Cluster Server running the ME terminates WMQ network AppServer ME Cluster WMQ network AppServer MR01 AppServer QueueManager QueueManager ME ME MR01 AppServer SIBus network MR01 configured for multiple ME locations ME Channels reconnect to the alternative location using MR01 Link fails over along with the ME SIBus network Failover of a WMQ Link • If the messaging engine is made highly available across a WAS cluster the link will also be made highly available on the WAS side • The ME, along with the link, is able to run on different application servers (with different TCP/IP addresses) • WMQ requires an additional support pack, MR01, to enable this function

47. Using synchronous connectivity withWMQ via the SIBus

48. Synchronous connectivity to WMQ • Configuring synchronous connectivity with WMQ through SIBus is a three stage process • First, a WebSphere MQ Serveris required • This defines which queue manager or queue sharing group (QSG) is to be connected to andhow to achieve it • E.g. identify the WMQ server connection channel of the queue manager • Servers > Server Types > WebSphere MQ servers > [New] • Second, that WebSphere MQ Server is made a member of the SIBus • Service integration > Buses > busName > Bus members > [Add] • Third, An SIBus queue destination is created which identifies a WMQ queue on the configured queue manager or QSG bus member. • This does not create a queue in WMQ, it simply references an existing queue from the SIBus • This enables producing or consuming applications connected to the SIBus to use the queue • Service integration > Buses > busName > Destinations > [New]

49. Summary • Why were we here • What options are available • Choosing between then • How to do it

50. Further Information • WAS product information • http://www-306.ibm.com/software/webservers/appserv/was/ • WAS Info Center • http://publib.boulder.ibm.com/infocenter/wasinfo/v6r0/index.jsp • http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp • http://publib.boulder.ibm.com/infocenter/wasinfo/v7r0/index.jsp • Using WebSphere MQ Java Interfaces in J2EE/JEE Environments • http://www-01.ibm.com/support/docview.wss?rs=171&context=SSFKSJ&dc=DB520&uid=swg21266535&loc=en_US&cs=UTF-8&lang=en&rss=ct171websphere • IBM RedBooks • http://www.redbooks.ibm.com • WebSphere Application Server V7: Messaging Administration Guide SG24-7770-00 • WebSphere Application Server V7: Concepts, Planning and Design, SG24-7708-00 • WebSphere Application Server V7: Technical Overview, REDP-4482-00 • WebSphere Application Server V6.1: JMS Problem Determination, REDP-4330-00 • WebSphere Application Server V6.1: System Management & Configuration, SG24-7304-00 • WebSphere Application Server V6 Scalability and Performance Handbook, SG24-6392-00 • WebSphere Application Server V6.1 Security Handbook, SG24-6316-01 • WebSphere Application Server V6.1: Technical Overview, REDP-4191-00 • WebSphere Application Server V6.1: Planning and Design, SG24-7305-00 • WebSphere Application Server V6.1: Installation Problem Determination, REDP-4305-00 • IBM developerWorks • http://www.ibm.com/developerworks • (Searching on “Service Integration Bus” returns a number of interesting articles) • http://www.ibm.com/developerworks/websphere/techjournal/0901_leming/0901_leming.html - WASV7 • http://www.ibm.com/developerworks/websphere/techjournal/0601_ratnasinghe/0601_ratnasinghe.html - Security • http://www.ibm.com/developerworks/websphere/techjournal/0601_smithson/0601_smithson.html - Security