XEVENTS/XMESSAGES Application Level Events and Message Service Overview

1. XEVENTS/XMESSAGESApplication Level Events and Message Service Overview Aleksander Slominski Matthew Farrellee Extreme Computing Lab, Indiana University

2. Outline • Introduction • Why We Need Events In Distributed Applications • requirements for Event System • Implemented Features • current version vs. previous • Generic Messaging API • Binding to XML Schemas and WSDL (SOAP/HTTP) • XEVENTS/XMESSAGES Java API • Small Snippets & Longer Example • Conclusions

3. Events For Distributed Computing • Domain: Scientific Applications Running Over Distributed Environment such as Grid • Problem: How to monitor complex distributed scientific applications that interconnects multiple components that were not designed to run together but must be connected and run seamlessly. • Solution: Make available events to application developers to facilitate asynchronous communication in applications and tools to monitor and permanently record events for later analysis.

4. System Requirements • Application level events and messaging • work with “legacy” code • easy to add to Java/C/C++ • Reliability layer • deal with sender errors, unsent data stored locally • deal with receive errors, keep trying in background • persistent & generic channel • Interoperability • can connect to other messaging systems • easy to create bridge (both send and receive events) • Integrate web services • most promising approach to grid app services • use WSDL/SOAP/HTTP – current best practices

5. User Requirements • Minimize impact of events • applications should not run slower • Shield user from need to deal with exceptional situations • Easy to add events to existing applications • Can be added to application parts on need basis • Persistent event storage • query past and live events • easy to analyze • Implementation must work on top of SOAP RPC • to build on web services momentum

6. Other Requirements • Network of channels • message/event sources and sinks • can build sophisticated filtering mesh • channel = source + sink • Work with directory and naming services • give names to message channels • Support Fail over • Small size of API and library • Performance • in limits of XML and SOAP/HTTP

7. Lessons from The Past • could send event but was it received? • if failed exception is thrown… • try again – but could be duplicated … • send to firewall-ed system but never got any response … • listen for events could lose events • you subscribe • but if your subscription not renewed you loose events • that is BAD! • motivation for persistent event channel • never loose any event that was sent!!!! • allow to pull persistently • so system behind firewall can pull events eventually • allow to pull in chunks! • huge historical event sets can be processed incrementally

8. Lessons from The Past • simpler API for users • to get reliability (publisher & listener API) • generalize: messages instead of events • can store any message • specialized for events • special storage (understands event type) • queries for events

9. Problem: How To Send Event • We need reliable event delivery! • (1) If Implemented as SOAP RPC • it is blocking call – overhead for application! • needs to handle exceptions – never easy! • (2) If Implemented as one-way SOAP Messaging • non blocking – good! • no information on status of event – unacceptable! • (3) If Implemented as Asynchronous SOAP RPC • requires “futures” for asynchronous sending • instead we send and forget

10. Solution: How To Send Event • option (1) available in implementation • but not recommended! • option (2) not practical • and is not SOAP RPC • option (3) - our current choice: Publisher Agent • move handling exceptions from users to the system • currently implemented as synchronous SOAP RPC • but sending is done in separate thread • exceptions recorded in log file • if can not send – store locally events in file

11. Problem: How To Receive Event • it is SOAP RPC call • if event is pushed then requires (small) web server • if behind firewall then can not receive events • failures are unpredictable • network outage • event channel died and needs restarting • event channel was migrated to other machine • we do not want to lose any events • we want to preserve application events history • lease

12. Solution: How To Receive Event • We need mechanism to deal with failures • network: retrying • channel restarted: retrying • … any other problem: retrying • be intelligent: try to find alternative channel • shield user from need to do this! • Requires persistent subscriptions • need to have persistent channels • so when finally request is sent it will be successful! • We provide Listener Agent • to do retrying and maintain lease with channels • it pulls event to deal with firewalls • can convert pull into push when delivering events to user

13. Problem: Event History • Who stores Historical Events • needs to talk to persistent channel • They can be large (ask for one year of events …) • must allow to retrieve events in chunks • Query problem • multiple query languages available • the most efficient depends on storage decision • Live and historical events may be processed differently • typically live events are typically pushed • typically historical events are pulled • should have different API? • Efficiency • for example data mining applications

14. Solution: Event History • Listener Agent allows to pull events in chunks • Query is an Object – undefined type • we currently support only one type of query: SQL • more defined later depending on channel implementations • We decided not to differentiate live and historical events • common API – less learning • we can query events from past going into future!!! • from last 2 hours to next 10 minutes • How to address efficiency • user can always opt to connect directly to storage • for example to RDBMS engine

15. Solving Client End Problem • Main focus of XEVENTS is on client side API • multiple implementation of APIs possible • the actual event infrastructure can be very complex • Reliability on client side • handled by agents • Publisher Agent API • no need for user to retry sending • Listener Agent API • no need for user to estimate time for lease

16. System Overview • Client: How to use XEVENTS/XMESSAGES • Publisher API • Listener API • Server/Service • publishers • listeners • channel: intermediary to connect publishers and listener … • can be persistent (connected to RDBMS etc.) • can be generic (accept any kind of Event) • Naming/Registry service (optional) • channels register with naming service • clients can find channels in naming service

17. Big Picture Event listeners: Event publishers: Event channels Applications publish events query and monitor events bind channel User Resources Directory service (LDAP, …) lookup channel

18. XMESSAGE: layer below events • Provides transport for messages • All required low level services are implemented here • Designed for simplicity • two interfaces • two messages • that is all! • Can be mapped to many possible infrastructures • Mapped it to WSDL PortTypes and XML Schemas • We use SOAP RPC for WSDL binding

19. Generic API: two key interfaces • XMessageSink • one way operation: push messages in public void handleMessages (XMessageBatch messages) throws Exception; • XMessageSource • two way: request messages and get messages public XMessageBatch requestMessages ( XMessageRequest req ) throws Exception;

20. XMessageBatch : wrap messages • philosophy behind is to embed explicitly headers • can be transported in any protocol • called XMessageBatch – transport more than on message • Object[] messages; • embedded messages • String currentToken; • prevents duplicate messages • String nextToken; • controls chained message retrieval (explained later) • int leaseDurationInMinutes • for how long is lease granted by message source (in minutes) • long timeToLiveForMessagesInMs • for how long messages should be retained

21. XMessageRequest: source query • Object query; • generic query interface • subtypes such as SQL query easily added • long timeToWaitForMatchInMs • control query execution if no match • int maxNumberOfMessagesToReturn • long maxTimeRangeInMsToReturn • control output for message pulling • int suggestedLeaseDurationInMinutes • how long client is eager to maintain lease

22. XMessageRequest: source query • String subscriberWsdl • allow to subscribe for push • int maxNumberOfMessagesToPush • control how many messages to push in batch • boolean takeMessages • should messages be removed from the source

23. WSDL/SOAP/XML schema binding • WSDL for • XMessageSink • XMessageSource • XML schema for • XMessageBatch • XMessageRequest

24. Easy to implement! • that is actually all for XMESSAGES • two portTypes and two complexTypes! • however we need easier to use API • take care of low level details • allows to hide and deal with network and other failures • API designed for reliability but requires • to deliver messages exactly once (deal with failures • XMESSAGES Publisher Agent • to retrieve messages for user (deal with failures) • XMESSAGES Listener Agent

25. Implementation Features • XMESSAGES Java API • Hot pluggable implementations • allows to plug in another implementation without any change to code (just add to CLASSPATH) • Easy to use API • Publisher Agent • Listener Agent • XEVENTS Java API • built directly on top of XMESSAGES API

26. Code snippets / API demonstration • no examples for low level • how to find channel – this is common first step • and do something to it • how to send events • startup publisher agent • send events • non blocking!!! • how to get events • can query (pull) • can subscribe for (push)

27. Recommendation for implementors • Channels should give long enough leases • worst case scenario: time for request to travel • recommened minimum 10 minutes • lease renew is idempotent • can repeat lease requests without side effects • lease is eventually renewed • persistent channel allows always to renew lease!

28. This is the end but …. • Visit our web page http://www.extreme.indiana.edu/xgws/xevents/ • Download source code • Subscribe to user (and dev) mailing lists • Let us know what we can do to make it better • Questions?

30. Presentation TODO … • how to know/check that channel is persistent and what queries are supported • need for metada • types of channels • stored as metadata in Directory service • keep metadata … • query generality • metadata • why not use Filter base interface + type • describe clock skew • start/end arrive timestamp • TODO relative time only !!!! • confusing for historical data • time in respect for message source and not channel • useful for monitoring vent channels