Message Queuing and Asynchronous Inter Process Communication

1. Message Queuing andAsynchronous Inter Process Communication An exposition on why and how to use .Net to introduce Message Queuing into your applications Message Queuing - a walk on the wild side

2. Sub-title: • How to make your applications appear to be really snappy when they’re actually dog-slow. Message Queuing - a walk on the wild side

3. What is Message Queuing? • Message Queuing is a technique for implementing Asynchronous Inter Process Communication • It’s an abstraction of a file-based persistence and network communication layer. Message Queuing - a walk on the wild side

4. What is Inter Process Communication? • IPC is communication between two discrete processes, typically one of which provides some service to the other (but they could be equal partners, servicing each other). Message Queuing - a walk on the wild side

5. So what then is Synchronous Inter Process Communication? • Synchronous IPC is what we usually refer to as a Remote Procedure Call (RPC), examples being a call to a dll function or an MTS object, or executing a SQL query. Message Queuing - a walk on the wild side

6. Okay, what then is Asynchronous Inter Process Communication? • Asynchronous IPC means that the process requesting the service doesn’t wait for confirmation that the service has been carried out. Message Queuing - a walk on the wild side

7. Why would I want to use this malarky when I can use good old RPC’s? • Lots of reasons, but mainly: • Your calling process wants to get on with things and doesn’t care when the service is carried out, as long as it’s assured that it will be eventually, guaranteed, no questions, no excuses. Message Queuing - a walk on the wild side

8. Like when?? Example: an order processing system When a customer places an order, the Order Placing component doesn’t want to wait for the Credit Check, Stock Control and Order Despatch components to deal with the order. MQ is one way to accomplish this, in a way which allows parts of an application to continue functioning even if other parts are out of service. Message Queuing - a walk on the wild side

9. How does it work? • One process acts as a client and posts a message to a named queue. • The other process acts as a server and retrieves the message from the named queue and processes it. The message contains everything it needs to perform the required service. • The Message Queue system takes care of everything in between Message Queuing - a walk on the wild side

10. If required, notification of completion of the service could be posted back to the client as another message in a separate queue Message Queuing - a walk on the wild side

11. What exactly is a “message”? • A message is a container for whatever you like. • It can contain some ASCII text, or a large chunk of binary data to be stored in a database, a spreadsheet, or anything in between. The MQ system doesn’t care about the contents, or impose any formats. • Both processes (ie the programmers) must have agreed beforehand on the format of the message body – no XML-type dtd’s, WSDL or any of that stuff.) Message Queuing - a walk on the wild side

12. Typically there will be a single queue for each IPC, ie between each client and the server providing the service. A ‘componentised’ application could, overall, use many queues. Queues are managed by the MQ Service. The MQ Service guarantees that the messages are persisted, delivered once only, and retrieved in FIFO order. Messages can be grouped as a transaction (so that they are all successfully processed, or none are.) Message Queuing - a walk on the wild side

13. This demo uses MSMQ2, which is part of Windows2000+, but there are other cross-platform MQ systems available. • It’s called MSMQ2 because it’s the first version that works properly. Message Queuing - a walk on the wild side

14. What’s it look like? • Single server model (ie single-machine model) • Multi-server model Message Queuing - a walk on the wild side

15. Single server model (ie single-machine model) • Simple to setup and administer • Has the advantages of AIPC without much complexity • BUT • Is limited to processes running on the same machine. Message Queuing - a walk on the wild side

16. Multi-server model • More complex to setup. • Can be across the motherboard, across the room or Across The Universe. • (over MSII – the Microsoft Intergalactic Internet – real soon now) • Can provide multiple routes for messages, and is therefore very robust. Message Queuing - a walk on the wild side

17. Has lots of TLA’s (MQIS, PEC, BSC, PSC, RS, IC, DC and the list goes on…. it’s TLA Heaven. eg “a BSC may also be a RS, but you must install a PEC or PSC before you install a BSC, whether you use an IC or a DC…”) • These all work together to guarantee that a message is delivered once, and once only, wherever it is sent. Message Queuing - a walk on the wild side

18. A Primary Enterprise Controller (PEC) manages the entire MQ system across servers. It is the foundation of the MQ infrastructure. • The Primary Site Controller (PSC) is the No. 2IC, controlling one site in the infrastructure. The PEC also serves as the PSC for its own site. Message Queuing - a walk on the wild side

19. A Backup Server Controller (BSC) provides load balancing and failure recovery support. It takes over control if the PEC or PSC fail. • The Routing Server (RS) supports dynamic routing, load balancing and intermediate store-and-forward message processing. RS’s allow multiple routes to be set up between source to destination, with cost-based routing policies.PEC, PSC and BSC all act as RS’s. Message Queuing - a walk on the wild side

20. Either way, to process the messages you can use…. • Sequential server:   The server process waits for a message to arrive at the top of the named queue. It pops the message off the queue and processes it. This is repeated, one message at a time. Each message has to wait for the previous message to be processed before it is dealt with. Message Queuing - a walk on the wild side

21. Alternatively the process can respond to OnMessage (?) events, using separate threads to process each message. Message Queuing - a walk on the wild side

22. Message Queuing is complementary to binary remoting. Message Queuing - a walk on the wild side

23. Here’s One I Did Earlier: Message Queuing - a walk on the wild side

24. Now for some Code examples…. Message Queuing - a walk on the wild side

25. .Net provides classes that tidy up a lot of the detail (System.Messaging namespace) • but not all – so I did my own to finish the job… Message Queuing - a walk on the wild side

26. MSMQ.cs: contains a class for opening a local queue for reading or writing, first creating it if necessary, and posting or retrieving messages to and from it. This class abstracts most of the details of using the message queue, including serializing the message object.MSMQSender and ProcCall code which posts messages to the queue using the above class.MSMQReceiver for an application client which pops messages from the queue, using the above class.ProcCallHandler a processing server, which retrieves procedure call messages from the queue in sequence and executes the requested procedure. A sort of disconnected RPC processor. Message Queuing - a walk on the wild side

27. References:MSMQ from Scratch (Crane, Crummer, Miller)Visual Studio.Net Help MSDN Message Queuing - a walk on the wild side