High-Performance Real-Time Messaging with Flex and LiveCycle Data Services

1. High-Performance Real-Time Messagingwith Flex and LiveCycle Data Services Mete Atamel, Computer Scientist @ Adobe. October 6, 2009 #adobemax221 (Twitter)

2. Outline • Architectural Overview. • Real-Time Channels and NIO Endpoints. • Message Filters. • Reliable Messaging. • Message Throttling. • LiveCycle Data Services Edge Server. • LiveCycle Data Services Load Testing Tool. • Q&A.

3. Architectural Overview • The obvious: LCDS acts as the integration point between RIA clients (Flex/AIR) and server-side services and resources (J2EE). • The less-obvious: This integration is based upon enterprise integration messaging patterns. Many of the core concepts and terminology found throughout LCDS is cataloged and covered in depth in “Enterprise Integration Patterns” the book: http://www.eaipatterns.com/ • Not a surprising coincidence considering that one of the original LCDS architects, Sean Neville, was a co-author, it’s well worth a read!

4. Architectural Overview – Breaking From Tradition • Current web applications rely solely on HTTP for client-server interaction. • LCDS is as protocol agnostic as possible with each supported protocol acting as a message channel between the client and server. • In addition, non-HTTP protocols such as RTMP (insecure, secure, tunneled) are supported with the possibility of adding future protocols without impacting how messages are exchanged or processed by the client or server. • In this respect LCDS is more similar to an ESB, or other message bus solutions, than to traditional web application frameworks.

5. Architectural Overview – Or Traditional? • RESTful APIs are the “new” kids on the web application block. • HTTP 1.1 certainly isn’t new - but formal attempts to build distributed APIs centered on the following tenets are: • A unique URI representing each resource. • Operations on these resources mapped to the limited set of HTTP 1.1 verbs. • Using media type tags and other HTTP headers to control the hypermedia display of a resource that the end user interacts with to drive state transitions. • From this perspective, LCDS has more in common with SOAP web services or other RPC and remoting technologies than current REST practices because when HTTP is used as the transport protocol: • HTTP requests and responses are used as envelopes for Flex messages. • Service invocations, message routing and resource state transitions depend on the internals of these Flex messages rather than HTTP protocol elements.

6. Architectural Overview – Multi-Paradigm • We don’t want to: • Limit the supported transport protocol to only HTTP, which precludes the use of real-time protocols such as RTMP. • Use resource-based APIs for everything because they don’t map correctly to traditional remoting and service-oriented (i.e. façade-style) APIs. • LCDS is a multi-paradigm solution; it uses its messaging infrastructure to provide support for all of the following: • Service-Oriented APIs – The Remoting and Proxy Services. • Publish-Subscribe Messaging APIs – the Message Service. • Resource-Oriented APIs – the Data Management and Proxy Services.

7. Architectural Overview – Comet • Comet is an umbrella term for techniques used to achieve a web application model in which a long-held HTTP request allows a web server to push data to a browser, without the browser explicitly requesting it. • Lightstreamer, Grizzly, Liberator, Jetty are some of the servers that support Comet in some fashion. • RTMP protocol support in LCDS is a true real-time, bi-directional, stateful socket connection between a client and the server. • LCDS also provides out-of-the-box Comet-style support: • Simple HTTP polling and piggybacking (neither strictly Comet, but still useful). • HTTP Long Polling. • HTTP Streaming.

8. Channels and Endpoints – Introduction • Flex client-side Channel connects to a server-side Endpoint. • Swappable, with no impact on application code for flexible deployment and seamless fallback. • Client-side ChannelSet supports automatic connect fallback. • ChannelSet is based on server config, or built in ActionScript directly. <default-channels> <channel ref=“rtmp”/> <channel ref=“long-polling-amf”/> </default-channels>

9. Channels and Endpoints – Choices, choices • AMFChannel  (NIO)AMFEndpoint. • regular, piggybacking, polling, long-polling. • StreamingAMFChannel  (NIO)StreamingAMFEndpoint. • HTTPChannel  (NIO)HTTPEndpoint. • regular, piggybacking, polling, long-polling. • StreamingHTTPChannel  (NIO)StreamingHTTPEndpoint. • RTMPChannel  RTMPEndpoint. • duplex socket. • All of the channels and endpoints have secure versions as well that use secure protocols (HTTPS, RTMPS).

10. Channels and Endpoints – Real-time • True real-time requires a duplex connection between client and server. RTMP is the only true real-time choice. • Challenges: • HTTP is not a duplex protocol. • RTMP is not HTTP. • Techniques to simulate HTTP push: • piggybacking, polling, long-polling, and streaming.

11. Channels and Endpoints – RTMP • Flash protocol for exchanging real-time data, audio and video. • Uses a single duplex TCP socket connection. • True duplex and stateful. • Available in LCDS, not BlazeDS. • Not HTTP: this can lead to deployment challenges.

12. Channels and Endpoints – HTTP piggybacking

13. Channels and Endpoints – HTTP polling

14. Channels and Endpoints – HTTP long-polling

15. Channels and Endpoints – HTTP streaming

16. Channels and Endpoints – Servlet vs. NIO endpoints • BlazeDS endpoints are Servlet-based whereas LCDS endpoints use Java NIO. • Servlet limitations: • Uses Java’s legacy blocking IO. • Dedicated thread per connection. • Does not scale to large numbers of concurrent long-polling or streaming connections. • LCDS and Java NIO advantages: • Java NIO decouples IO from threading concerns. • LCDS scales to large numbers (thousands to tens of thousands) of connections. • In either case; trade-off between # of connections and throughput.

17. Channels and Endpoints – Questions to ask • HTTP vs. non-HTTP protocol. • True real-time vs. near real-time. • Binary vs. XML data. • Secure vs. insecure protocol. • Standard ports (80, 443) vs. non-standard ports. • Scalable (NIO) vs. non-scalable (Servlet).

18. Message Filters – What is it? • Message Filters provide the opportunity to pre-process incoming messages, post-process reply messages, as well as server-pushed messages. • There are two types of message filters, sync and async, with distinct APIs due to differences between sync and async message handling. • Sync message filters work with Servlet-based endpoints. • Async message filters work with NIO-based endpoints. • Multiple message filters can be chained.

19. Message Filters – BaseSyncMessageFilter public abstract class BaseSyncMessageFilter { public AsyncMessage filterRequest(final Message message, final Endpoint endpoint, final SyncMessageFilterContext context); public void filterPush(final AsyncMessage message, final FlexClient recipient, final Endpoint endpoint, final SyncMessageFilterContext context); }

20. Message Filters – BaseAsyncMessageFilter public abstract class BaseAsyncMessageFilter { public void in(final MessageContext context); public void out(final MessageContext context); public AsyncMessage filterPush(final AsyncMessage message, final FlexClient recipient, final Endpoint endpoint); }

21. Message Filters – Configuration • Sync Message Filters can be defined in services-config.xml: <sync-message-filters> <filter id="syncFilter" class="my.custom.filters.TestSyncFilter"/> ... </sync-message-filters> • Async Message Filters can be defined in services-config.xml: <async-message-filters> <filter id="asyncFilter" class="my.custom.filters.TestAsyncFilter"/> ... </async-message-filters>

22. Advanced Messaging Support & AdvancedChannelSet • Reliable Messaging, Message Throttling, and LCDS Edge Server are part of what we call Advanced Messaging features. • Advanced Messaging support should be enabled for them to work correctly in services-config.xml: <services> <service id=“AdvancedMessagingSupport” class=“flex.messaging.services.AdvancedMessagingSupport”/> • Additionally, if ChannelSets are explicitly created in Flex apps, AdvancedChannelSet should be used when Reliable Messaging and Adaptive Message Throttling is used: var channelSet:ChannelSet = new ChannelSet(); // Existing code. var channelSet:ChannelSet = new AdvancedChannelSet(); // New code.

23. Reliable Messaging – What Is It? • Messages are exchanged correctly across the network regardless of failures as long as the client and server hosts remain running. • Correctly implies in-order, once-and-only-once delivery to the far end. • Connectivity failures are dealt with and recovered from automatically. • If connectivity fails and is restored, any undelivered messages are silently and correctly retransmitted with no risk of duplicate processing by the far end. • Both reliable and unreliable message traffic can be sent across a single channel between the client and server; no need for separate transports. • All LCDS channel and endpoint options support reliable messaging! • Reliable messaging applies to all of our services (not just the Message Service), so you can enable it for Remoting, Proxy and Data Service destinations as well.

24. Reliable Messaging – What Is It NOT? • Reliability is not “guaranteed delivery” in the face of client or server crashes or network connectivity failing indefinitely. • The Flash Player doesn’t currently support a local transactional persistent store. • LCDS doesn’t ship with an embedded data base or transactional disk store. • But when the Player supports a transactional store this could be revisited. • Once-and-only-once delivery doesn’t automatically prevent a user from repeatedly triggering a client-server interaction. • If a user clicks a button repeatedly to invoke a RPC, it’s up to the app developer to guard against that in the UI as necessary

25. Reliable Messaging – Why Is It Useful? • Lots can go wrong with network connectivity but failures are often recoverable. • Automatic recovery works in all cases were the client and server hosts continue running and network connectivity between the two is eventually restorable. • Some examples: • Failure along the current network path (e.g. a router crashes, but both hosts may still reach each other over an alternate route) • Transient connectivity issue (e.g. a router between the client and server is restarted) • Client machine switches IP addresses (e.g. undock your laptop triggering a failover from the LAN NIC to a wireless modem) • Anything else that is not a terminal failure for connectivity between the client and server.

26. Reliable Messaging – Configuration & Usage • Configuration is specified at the service or destination level.…<network> <reliable>true</reliable></network> • Tune how long a client will attempt to restore a reliable connection using a config option in services-config.xml (or programatically at the client):… <flex-client> … <reliable-reconnect-duration-millis>60000</reliable-reconnect-duration-millis></flex-client> • During configuration, keep in mind: • In order to ensure there’s something to reliably reconnect to, use a value that is shorter than the idle timeout for server-side state (e.g. FlexSession and/or FlexClient idle timeouts). • The trade-off between reliably reconnecting to the current server vs. giving up and failing over to an alternate server.

27. Message Throttling – What Is It? • Limit number of messages between Flex clients and the LCDS server. • Applies to inbound & outbound messages, and message & data management service destinations. • Supports multiple levels of static and also adaptive throttling limits. • Destination, client, consumer, and subtopic level. • Supports multiple policies to deal with messages that exceed the throttling limits. • IGNORE, ERROR, BUFFER, CONFLATE policies.

28. Message Throttling – Configuration • Specified at the destination in either messaging-config.xml or data-management-config.xml under network section. <network> <throttle-inbound policy="ERROR" max-frequency=“10” max-client frequency=“5”/> <throttle-outbound policy="IGNORE" max-frequency=“10” max-client-frequency=“5”/> … • Inbound/Outbound is with respect to the destination. • Max-frequency is destination level, max-client-frequency is client level number of messages limit per second. • There’s also Consumer level and Consumer subtopic level frequencies.

29. Message Throttling – Inbound <throttle-inbound policy="ERROR” max-frequency=”10" max-client frequency=“5"/> • Inbound throttling applies to messages sent from Flex clients (Producer) to an LCDS destination. • Policy can be NONE, IGNORE, or ERROR. • In NONE, throttling is disabled. • In IGNORE, when the limit is reached, message is silently dropped. • In ERROR, when the limit is reached, message is dropped and client gets an error message.

30. Message Throttling – Outbound <throttle-outbound policy="IGNORE" max-frequency=“10” max-client-frequency=“5”/> • Outbound throttling applies to messages sent from an LCDS destination to Flex clients (Consumer/DataService). • Policy can be NONE, IGNORE, BUFFER, or CONFLATE. • In NONE, throttling is disabled. • In IGNORE, when the limit is reached, message is silently dropped. • In BUFFER, when the limit is reached, messages are queued and drained at the rate specified by the limits. • CONFLATE applies to DMS destinations only and we’ll cover it shortly.

31. Message Throttling – Consumer level frequencies <throttle-outbound policy="IGNORE" max-frequency=“10” max-client-frequency=“5”/> • In addition to levels specified at the destination, Consumer and DataService can specify its own limit as it subscribes. consumer.maxFrequency = 3; consumer.subscribe(); • Moreover, Consumer/DataService can specifiy subscription level frequencies if they are using subtopics/selectors. consumer.addSubscription(“mysubtopic”, null, 2 /* max-frequency */); consumer.subscribe(); • Keep in mind that the following must be true: • max-frequency > max-client-frequency > Consumer.maxFrequency > consumer.addSubscrption frequency.

32. Message Throttling – Conflate Policy • Conflate policy only applies to DMS destinations because we know the format of DMS messages. • It’s similar to BUFFER policy in the sense that messages are queued once they exceed the limit but while they are sitting in the queue, they are conflated (merged). • There are two supported types of conflation: • DataMessase.UPDATE_OPERATION conflation where the most recent update replaces all the previous updates. • DataMessage.CREATE_OPERATION and DataMessage.DELETE_OPERATION conflation. • Note that update conflation is not simply the latest update but rather merge of all previous updates in a single message.

33. Message Throttling – Adaptive Frequency • The most powerful aspect of Message Throttling is adaptive frequency support. • Instead of specifying hard message frequency limits, one can configure Flex Client to have adaptive frequency support: <flex-client> <flex-client-outbound-queue-processor class="flex.messaging.client.AdvancedOutboundQueueProcessor”> <properties> <adaptive-frequency>true</adaptive-frequency> <frequency-step-size>1</frequency-step-size> <max-queue-size>0</max-queue-size> • It’s useful to keep the Message Priority feature in mind while using Message Throttling feature.

34. LCDS Edge Server – What Is It? • Allows your LCDS application to be partitioned across multiple network tiers for security with optimized server push performance. • Run service destinations that require tight security at the application tier. • Run service destinations with low-latency server push requirements and looser security constraints at the edge tier.

35. LCDS Edge Server – How Do You Want To Slice It • Unlike traditional web tier HTTP servers, the LCDS Edge Server supports real-time server push across enterprise DMZs. • Every channel & endpoint option is supported – including RTMP! • Splitting an existing application across edge and app tiers is straightforward. Simply place the subset of destinations (may be none) you want to run at the edge tier for minimized latency into the services-config.xml for your LCDS Edge Server(s). • No recoding of the client app is required when a LCDS Edge Server is introduced. • No recoding at the server is required either.

36. LCDS Edge Server – Connecting The Tiers • Register a GatewayService in the LCDS Edge Server config:…<service id=“edge-gateway" class="flex.messaging.services.GatewayService"> <properties> <gateway-endpoint> <urls> <url>amfsocket://localhost:9807</url> </urls> </gateway-endpoint> </properties></service> • Register a GatewayEndpoint in the backing LCDS App Server config:…<channel-definition id="gateway-endpoint" server-only="true"> <endpoint url="amfsocket://localhost:9807“ class="flex.messaging.endpoints.GatewayEndpoint" /></channel-definition> • The GatewayService and GatewayEndpoint open gateway connections between the tiers on behalf of connected clients to provide those clients with a unified view of the full functionality of the server-side app.

37. LCDS Edge Server – Security • Client authentication is enforced at the edge tier and authorization in the form of security constraints or programmatic checks may be applied at both tiers. • No un-authenticated message traffic is passed from the edge tier to the backing app tier. • Client authentication state is synchronized across tiers automatically. • FlexSession and FlexClient attribute state (including session-scoped remote objects and Data Service assemblers) is not synchronized across tiers automatically – that would be a serious security hole!

38. LCDS Load Testing Tool • LCDS now ships with an Java NIO based load testing tool under resources/load-testing-tool. • Can be used to simulate thousands of clients on a single machine. • Java based Channel, ChannelSet, Producer, Consumer. • No higher level RemoteObject, DataService etc. support yet. • Supports AMF polling, long-polling, streaming, and RTMP. • No AMFX (i.e. HTTPChannel) support yet. • Supports reliable messaging and adaptive throttling.

39. Conclusion • Many new features have been added to LCDS to improve the performance, reliability, deployability and reach of real-time messaging and data management. • Put them to good use! • Questions?