Building Hybrid Applications using the Azure Service Bus

1. Building Hybrid Applications using the Azure Service Bus David Ingham Senior Program Manager, Service Bus Team 3-033

2. Agenda • Service Bus overview • Recent developments with SDK 1.8 • Service Bus for Windows Server • Cross-platform hybrid apps

3. What is Service Bus? • Unified set of messaging capabilities Consistent management and observation capabilities • Service Bus Relay • Rich options for interconnecting apps across network boundaries • Service Bus Brokered Messaging • Queuing, publish/subscribe • Easily build hybrid apps

4. Service Bus Relay • Integrated w/ WCF programming model • Using Outbound Connections Only • No open inbound firewall/NAT ports • Outbound connections default to TCP • Fall back to HTTP when needed • Guarded by Datacenter Firewall • Secured with Access Control Service Bus Frontend Nodes Ctrl NLB outbound socket rendezvous outbound socket connect Ctrl Forwarder Sender Receiver

5. Audience Participation http://tinyurl.com/sbrelay DemoRemoting a WCF service • Showcasing Service Bus Relay

6. Service Bus Brokered Messaging • Message queuing (Queues) • Publish/subscribe (Topics, Subscriptions) • Enables loosely-coupling

7. Tightly Coupled Drivers Tracking Store Front End Shipping Service

8. Tightly Coupled Drivers Tracking Store Front End Shipping Service

9. Loosely Coupled Drivers Tracking Store Front End Order Queue Shipping Service

10. Loosely Coupled Drivers Tracking Store Front End Order Queue Shipping Service

11. Loosely Coupled Drivers Tracking Store Front End Order Queue Shipping Service

12. Loosely Coupled Tracking Tracking Shipping Service Store Front End Order Queue

13. Service Bus Queue S R S R Queue S R Sequential message log with shared cursor and locks Can have competing consumers Each message published is consumed by a single consumer

14. Service Bus Topic S R Subs Topic R S R R S R Sequential message log; each subscription has own lock/cursor Subscribers can filter, based on message properties Each message published is available to each subscription

15. BrokeredMessage Brokered Message Properties • Properties • Key/value pairs exposed to the broker • Subscription rules can filter based on properties • Body • Opaque payload not exposed to the broker • Can be used for encrypted data Value Key Value Key Value Key Value Key Body Body

16. DemoMartVue: Retail Insight In Real Time • Showcasing Service Bus Brokered Messaging

17. Inventory and Fulfillment Centers ? Retail Points of Sale Real Time Retail Analytics Business Operations

18. Service Bus Store Inventory Queue

19. Service Bus Store Inventory Queue Load Leveling

20. Store Service Bus Store Inventory Queue Store Inventory Load Balancing Store

21. Audit Service Bus Sub Store Inventory Data Collection Topic Sub Audit Taps

22. Audit Service Bus Sub Store Inventory Data Collection Topic Sub Sub Dashboard Real-Time Observation

23. Audit Service Bus Sub Store Inventory Data Collection Topic Sub Sub Sub MartVue Analyzer Analysis Topic Dashboard Sub Real-Time Analysis

24. Audit Service Bus Sub Store Inventory Data Collection Topic Sub Sub Sub MartVue Analyzer Scale-Out Eventing Analysis Topic Dashboard Sub Catalog Topic Sub

25. New Service Bus features inWindows Azure SDK 1.8

26. Message auto-forwarding • Link together queues & topics • Build rich messaging flows • Scale out topics • Fan-in from several queues • Reduce coupling QueueDescriptiondestinationQ = newQueueDescription("myQ2"); QueueDescriptionsourceQ = newQueueDescription("myQ1"); sourceQ.ForwardTo = “myQ2";

27. Message lock renewal • Supports long running workflows • Maximum lock duration is 5 minutes • Can renew lock repeatedly QueueClientqueueClient = QueueClient.Create("myQ"); BrokeredMessage message = queueClient.Receive(); message.RenewLock(); message.LockedUntilUtc; // check to see when you need to renew

28. Pause queues & topics • Useful in multi-tenant apps • Send ops blocked for disabled queues and topics • Receive ops blocked for disabled queues and subscriptions QueueDescriptionqd = namespaceManager.GetQueue("myQ"); qd.Status = EntityStatus.Disabled; namespaceManager.UpdateQueue(qd); qd.Status = EntityStatus.Active; namespaceManager.UpdateQueue(qd);

29. Send-time filter evaluation • Evaluate filters at the time of message send • Warn if no subscriptions match • Works with all types of filters TopicDescription td = newTopicDescription("Topic"); td.EnableFilteringMessagesBeforePublishing = true; // throwsNoMatchingSubscriptionException on send

30. Improved monitoring • View details of “sub queues” • Active, dead-letter, scheduled, transfer queues • Available as part of Description objects NamespaceManagernm = NamespaceManager.Create(); QueueDescriptionmyQ = nm.GetQueue("myQ"); varactive = myQ.MessageCountDetails.ActiveMessageCount; vardeadLettered= myQ.MessageCountDetails.DeadLetterMessageCount; varscheduled = myQ.MessageCountDetails.ScheduledMessageCount; vartransfer = myQ.MessageCountDetails.TransferMessageCount;

31. New API capabilities in SDK 1.8 • Message lock renewal • Message auto-forwarding • Pause entities • Send-time filter evaluations • Improved monitoring • Batching APIs for sending and receiving groups of messages • Consistent send of messages to multiple entities • Session browsing

32. Service Bus for Windows Server

33. On-premises messaging scenario Cloud On-premise Develop&Deploy Deploy Develop Deploy Deploy App Data

34. Cloud and on-premise symmetry • Get the connection string from the ConfigWizard or the following cmdLet: PS C:\Program Files\Service Bus\1.0> Get-SBClientConfiguration Endpoint=sb://hostName/ServiceBusDefaultNamespace; StsEndpoint=https://hostName:9355/ServiceBusDefaultNamespace; RuntimePort=9354;ManagementPort=9355 Paste it in to your configuration file: <appSettings> <addkey="Microsoft.ServiceBus.ConnectionString"value="Endpoint=sb://hostName/ServiceBusDefaultNamespace;StsEndpoint=https://hostName:9355/ServiceBusDefaultNamespace; RuntimePort=9354;ManagementPort=9355"/> </appSettings> Use it in your code: string connectionString= ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"]; MessagingFactoryfactory = MessagingFactory.CreateFromConnectionString(connectionString); NamespaceManagernamespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);

35. Cross-platform hybrid apps

36. Message-oriented middleware allows application modules to be distributed over heterogeneous platforms and reduces the complexity of developing applications that span multiple operating systems…--Wikipedia’s entry for message-oriented middleware

37. Proprietary messaging protocols • Difficult to port applications • Requires re-coding all applications • Difficult to integrate • Application level bridges to move messages and translate message formats • Restricted platform support • Limited to whatever vendor provides

38. Advanced Message Queuing Protocol 1.0 • Open, standard messaging protocol • Enables cross-platform apps to be built using brokers, libraries and frameworks from different vendors • Features • Efficient – binary connection-oriented protocol • Reliable – fire-and-forget to reliable, exactly-once delivery • Portable data representation – cross-platform, full-fidelity exchange • Flexible – client-client, client-broker, and broker-broker topologies • Broker-model independent – no requirements on broker internals

39. Announcement: OASIS AMQP 1.0 Standard released • This week OASIS announced the ratification of the AMQP 1.0 Standard • Software vendors and end-users can bet on AMQP 1.0 knowing it’s a stable, well-supported protocol standard • The culmination of several years effort by more than 20 companies • Technology vendors: Axway Software, Huawei Technologies, IIT Software, INETCO Systems, Kaazing, Microsoft, Mitre Corporation, Primeton Technologies, Progress Software, Red Hat, SITA, Software AG, Solace Systems, VMware, WSO2, Zenika. User firms: Bank of America, Credit Suisse, Deutsche Boerse, Goldman Sachs, JPMorgan Chase.

40. VMware “AMQP 1.0 is a novel addition to the growing toolkit of open protocols for transporting data between systems and virtualized application delivery. Standard transports enable lower cost business integration and messaging. AMQP 1.0 admits many use cases by defining safe message transfer between peers, without the constraint of a message broker model. With its open license, we anticipate both AMQP's wide adoption by messaging servers, and its use as a new API for database and integration products.”--Alexis Richardson, Senior Director

41. Red Hat “Red Hat is pleased to see the hard work of the Technical Committee come to fruition. We are a founding member of the AMQP Technical Committee and have been active on the specification since the early days. AMQP 1.0 represents a significant improvement in the messaging arena and we expect to continue to support it in our products to best customer needs.”-- Mark Little, Vice President, Middleware Engineering

42. SoftwareAG “A platform independent and vendor neutral protocol like AMQP removes hurdles in advancing interoperability of message-oriented middleware technologies. As a founding sponsor member of the AMQP TC and the related AMQP Steering Committee, Software AG is very pleased to see AMQP 1.0 transition to an OASIS Standard. Software AG supports numerous standards in its product suite, and AMQP has been an important addition to webMethods Nirvana, increasing interoperability and providing advanced messaging capabilities to our customers.”-- Prasad Yendluri, VP & Deputy CTO

43. Kaazing “As the enablers of the Living Web and HTML5 WebSocket technology, everyone at Kaazing is excited to support OASIS' ongoing efforts to proliferate open standards and create a superior user web experience. Standardizing AMQP and combining it with WebSocket technology is an excellent strategy when building an event driven architecture. Working alongside OASIS, Kaazing has developed a Living Web in order to create the best possible web experience for users, reduce complexity, and increase interoperability.”-- John Fallows, CTO and Co-Founder

44. Microsoft “Microsoft congratulates the AMQP community on approval of AMQP version 1.0 as an OASIS Standard. As an open and interoperable messaging protocol that can scale from mobile clients to the cloud, AMQP has benefitted from the participation of technical experts from around the world, and the achievement of this important milestone will lead to continued growth in the AMQP ecosystem. We look forward to working with the community to promote AMQP-based interoperability and innovation.”-- Scott Guthrie, Corporate VP, Microsoft's Server and Tools Business Division

45. Announcement: Service Bus AMQP 1.0 preview • AMQP 1.0 preview available today in Azure Service Bus • Updated Service Bus client library available • Support for a range of third-party client libraries • More platforms will be supported as libraries become available • General availability in H1 2013

46. Support for multiple protocols .NET, Node.js, Java, PHP, Python “SBMP” high perf. .NET only HTTP high reach lower perf. AMQP 1.0 high perf high reach App ( .NET ) App ( any language ) App ( any language ) SB .NET Lib SB Wrappers Community Libs “SBMP” HTTP AMQP 1.0 Windows Any OS Any OS

47. AMQP 1.0 client libraries

48. DemoPictureMagic • Showcasing support for cross-platform hybrid apps using Service Bus and AMQP 1.0

49. MessageId = 1234 ReplyTo = RQ Subs Transform Requests Topic Image Transformer App ClientApp MessageId = 5678 CorrelationId = 1234 RQ

50. Linux (Azure) SwiftMQ Broker Monitor Service Bus Transform Requests Topic Transformer Windows (on-prem) Archiver Twitter