Worldwide Messaging Support for High Performance Real-time Collaboration

1. Worldwide Messaging Support for High PerformanceReal-time Collaboration Pete Burnap, Hasan Bulut, Shrideep Pallickara, Geoffrey Fox, David Walker, Ali Kaplan, Beytullah Yildiz, and Mehmet A. Nacar Community Grids Lab, Indiana University School of Computer Science, Cardiff University, UK

2. What is NaradaBrokering? • Distributed messaging infrastructure • Provides support for P2P, Centralised and Distributed interaction • Efficiently routes messages between originator and registered consumers • OBJECTIVE - To provide a recognised open-source standard for a messaging infrastructure to build SOAs and Grids upon • It is hoped that these results will influence others to use NaradaBrokering as a messaging protocol for their own applications

3. Motivation • Build distributed systems from “interoperable” services linked by messages (SOAP) – architect capabilities as services • Grids are “just” large scale sets of such services • Need to support real time streams and NOT just files (collections of messages) consistent with WS standards (P2P and “central”) • Manage messaging for • Optimize communication for bad links, firewalls etc • Collaboration (multi-cast streams) • Fault tolerance with re-transmitted messages and Replicated Services • Replay – access any message at any time • Virtual addressing with pub-sub metaphor • Performance from protocol (UDP v Parallel TCP ..) and representation • Heterogeneous Clients – filter to and from PDA’s • Open Source http://www.naradabrokering.org (4 downloads/day) is a scalable distributed pub-sub system supporting multiple standards (JMS, WS) and subscription methods • Implements “Service Internet” and Notification areas of WS-* Infrastructure • Candidate for Axis2-MOM (Message Oriented Middleware) infrastructure

4. Architecture

5. NaradaBrokering Features

6. NaradaBrokering Services

7. Buffering Service • This service is incorporated into the system to facilitate the buffering of events prior to releasing them. • Buffering service time orders events and releases event based on three metrics • Number of events in the buffer • Size of the buffer • Time spent by event in a buffer.

8. Time Differential Service • This service is essential to reduce jitters in large distributed environments. • Networks introduce unpredictable delays that increase jitter. • Service takes events released by buffering service, and ensures that it preserves time spacing between events. • TDS can provide time spacing resolution of up to 1 millisecond between events.

9. Reliable Delivery Service • Guaranteed delivery in multiple producer/ consumer settings. Guarantees hold true in the presence of • Node/Link Failures • Links can lose messages and garble message order. • Storage failures: Stores need to recover after failure. • Prolonged entity disconnects • Exactly-Once and Ordered delivery of events • Uses both positive & negative acknowledgements • Supports Replay and Fast Recovery from failures • Independent of underlying archival system. • Was used to enhance fault tolerance in Grid-FTP.

10. Dealing with large payload sizes • To cope with large payloads, the substrate incorporate 2 sets of services. • Compression/Decompression service: The substrate incorporate support for zlib based compression and decompression of payloads. • Fragmentation/Coalescing Service: These service can break up a large payload into smaller fragments. The coalescing service can take these smaller fragments and coalesce them into the original large payload. • This was used to deal with transfer of large payloads (up to 1 GB) in the NB enhanced Grid-FTP application.

11. Broker Discovery Service • Locates the nearest available broker that a client can connect to • Incorporates specialized nodes – broker discovery nodes – to maintain broker info. • Depending on load or security issues, brokers may decide to respond/ignore discovery requests. • If available the scheme can exploit IP multicast for discovery. • Nearest broker determined by ping times, loss rates and available bandwidth.

12. Topic Discovery Service • Allows publishers and subscribers to advertise topics. • Creator of topic possesses credentials to indicate ownership of the topic. • Discovery of topics takes into account credentials of client trying to discover topic. • Topic owner may restrict discovery to a limited authorized set of clients. • Discovery requests can be made using simple strings or regular expression queries.

13. Security Service [ Based on Message Level Security ] • Messages organized into topics • Each topic has a separate key; Topics can be organized into sessions

14. Web Services Support • Currently we have incorporated support for the following Web Service specifications • WS-Eventing (WSE): This is a publish/subscribe based notification framework from Microsoft and IBM. • WS-ReliableMessaging (WSRM): This is a protocol for ensuring the guaranteed delivery of SOAP messages between 2 Web Service endpoints. This specification is from IBM and Microsoft. • WS-Reliability (WSR): This is a competing specification from Oracle and Sun in the area of reliable messaging between Web Services. • These handlers are available for use in Axis1.2 or exploiting NB SOAP Intermediary support without a container • Axis1.2 version can be used inside container or as a Proxy

15. Web Services Support – cont.. • We are also working on implementing support for the WS-Notification (WSN) suite of specification that is part of the Web Services Resource Framework (WSRF). • WS-Notification explicitly adds brokers to Eventing • Note that almost all these specifications leverage the WS-Addressing (WSA) specification. • We have incorporated support for all the rules associated with WSA.

16. Parallel TCP Streaming • Data transfer across high-performance networks suffers within current TCP implementation due to tendency to hold back maximum bandwidth usage • Can be solved either by using TCP multiple streams in parallel • TCP streams consists of three basic steps: • splitting of data into sub packets at sender side • sending these sub packets over the network by using multiple Java socket streams in parallel • merging of received sub packets at the receiver side.

17. TCP Test Observations • This illustrates the capability of NaradaBrokering to maintain a high speed message delivery performance despite the scaling of message content payload. • Mean transmission delay times start to rise as the payload increase becomes increasingly steep between 1000 and 10000 bytes

18. TCP Streaming Results LAN Test Continental WAN Test Bandwidth (mbps) Bandwidth (mbps)

19. TCP Streaming Results cont… Inter-Continental WAN Test

20. Future of NaradaBrokering • Support for replicated storages within the system. • In a system with N replicas the scheme can sustain the loss of N-1 replicas. • Clarification and expansion of NB Broker to act as a WS container • Integration with Axis 2.0 as Message Oriented Middleware infrastructure • Support for High Performance transport and representation for Web Services • Needs Context catalog under development • Performance based routing • The broker network will dynamically respond to changes in the network based on metrics gathered at individual broker nodes. • Replicated publishers for fault tolerance • Pure client P2P implementation (originally we linked to JXTA) • Security Enhancements for fine-grain topic authorization, multi-cast keys, Broker attacks

21. Further Information • Web - http://www.naradabrokering.org/ • Geoffrey Fox - gcf@indiana.edu • Shrideep Pallickara - spallick@indiana.edu