Messaging Systems and Technology

1. Messaging Systems and Technology

2. Introduction • Synchronous middleware doesn’t suit all applications • Message Oriented Middleware (MOM) provides features like: • asynchronous communications between processes • store-and-forward capabilities • transactional messaging

3. Basic Asynchronous Operations • Send (dest, message) • Receive ( target, message) queue send receive

4. MOM usage • Applications often need: • deferred processing of some ‘slow’ transaction (eg printing an invoice) • support efficient 1-to-many and many-to-many communications • to send messages whether the server is available or not • to provide event notification to a dynanic user community • loosely coupled client-server systems

5. Example - Transactions

6. MOM Technologies • Broadly two (somewhat intersecting) categories • Message brokers (IBM, BEA, Microsoft, etc) • Broadcast-based publish-subscribe (TIBCO, IONA) • Both categories differ greatly in their features and capabilities

7. Some Application Examples • Stock price update notification • General workflow systems • Application integration • System management

8. Some Example Products • TIBCO/Rendezvous • IONA’s OrbixTalk • IBM’s MQ Series (Publish-Subscribe)

9. TIB/Rendezvous • Pioneers of broadcast/multicast publish-subscribe technology • Publishers and subscribers communicate using subjects Register/ Subscribe Create/ Publish Subject Sub Pub Sub Sub

10. Subjects • Hierarchical names identify a subject of interest • /CSIRO • /CSIRO/gossip • /CSIRO/gossip/ADSaT • /CSIRO/work/ADSaT • Wildcards can be used • /CSIRO/** • /CSIRO/*/ADSaT

11. Multicast/Broadcast • A published message is sent across the network only once • All subscribers receive the same message • IP Multicast or broadcast is used on LANs • Important scalability issue as bandwidth/processor usage is low

12. LAN Architecture

13. WAN Architecture Filtered, Point-to-Point

14. Quality-of-Service (QOS) • Reliable • publisher informed after time-out period if message cannot be delivered to a subscriber • Certified • guaranteed delivery or both parties informed of failure • to survive process failure messages can be logged to disk until specified time-out period expires

15. Programming Options • C/C++/Java • Self-describing message format • TIB/ObjectBus layers a standard CORBA-compliant ORB upon the TIBCO protocols. • TIBIOP provides broadcast messages in a CORBA environment.

16. Security • Security is a problem in publish-subscribe multicast technology • TIBCO Data Security (DS) product: • publisher controls secure key distribution to subscribers using digital signatures • subscribers use key to encrypt messages on a subject • choice of encryption algorithm available

17. OrbixTalk • Uses IP multicast in CORBA environment

18. Topics • Communication based on topics (read subjects, wildcards, etc) • 3 QOS levels: • otmcp: UDP IP multicast. Limits packet size to 1280 bytes and no guarantee of receipt. • otrmp: Augments IP multicast to ensure message delivery to all subscribers until configurable time-out expires • otsfp: OrbixTalk MessageStore daemon uses a store-and-forward protocol to provide guaranteed message delivery.

19. OrbixTalk Architecture

20. Architecture Issues • Daemons can have hot backups to support failure of primary daemon • Several MessageStore daemons can be used to facilitate load balancing: • each OrbixTalk process only use one MessageStore • Compaction utility must be scheduled to remove old messages

21. Security/WANs • No specific security or WAN support • security can be programmed using Orbix Transformers • WANs traffic needs appropriately configured routers, or an IIOP bridge between LANs to be written

22. MQSeries PubSub • MQSeries is probably most widely deployed MOM product • Basic technology provides a message queue architecture: • communication via shared queues managed by Queue Managers • persistent/non-persistent messages • transactional queue access with XA • broad platform support

23. MQSeries PubSub 1 PubSub Broker per QM LAN/WAN support Topics interchanged by Brokers if needed

24. Architecture • Brokers can be organized to communicate hierarchically HQ USA Asia West East HK

25. Features • Typical hierachical topic names, wildcards, etc • Streams can be used as higher level topic partitioning scheme • DEFAULT stream per broker • optional additional streams for specified topics • broker allocates a thread to handle each stream

26. Quality-of-Service • Persistent • messages written to log file • survive broker/QM failure • Non-persistent • messages lost if broker fails • fast • Message priority supported in queues

27. Other features • MQ can act as an XA transaction manager • Basic security provided, integrated with native operating system • Security exits support integration with 3rd party products • Other products, eg MQ Integrator

28. Performance Test • Publishers and subscribers run on different machines • Dual pentium 500Mhz NT 4.0 boxes, 0.5GB memory • MQ QM/broker and Orbix daemons run on same machine as publishers • Reliable, non-presistent protocols used

29. 256 Byte Message

30. 512 Byte Message

31. Interpretation • Multicast technologies are fast! • TIBCO rvd becomes bottleneck • ‘Publisher too fast’ exceptions • OrbixTalk generates large number of interrupts • one per message per listener • MQ scales well, but queue/broker architecture is inherently slower

32. Summary • MOM provides excellent solution to many business problems • Range of technologies available, with very different: • architectures/technologies • features • performance • Important to understand issues and select right one for your business