COMP60611 Fundamentals of Parallel and Distributed Systems

1. COMP60611Fundamentals of Parallel and Distributed Systems Lecture 16 Deadlock + Channels in Promela John Gurd, Graham Riley Centre for Novel Computing School of Computer Science University of Manchester

2. Introduction • A closer look at deadlock • Channels, modelling distributed systems in Promela • Conclusion

3. Deadlock - Overview • Deadlock means no further progress is possible. • four necessary & sufficient conditions for deadlock • Deadlock implies there are no eligible transitions from the state a computation has reached. • In real codes, deadlock implies that there are blocked threads or processes. • Our aim is deadlock avoidance, that is to design systems where deadlock cannot occur.

4. Source in the Literature • Deadlock or ‘deadly embrace’ was addressed as a problem arising in operating systems in 1971. • The following paper identified four necessary and sufficient conditions for the occurrence of deadlock and discussed methods to avoid deadlock: • E.G. Coffman, jr., M.J. Elphick, A. Shoshani, “System Deadlocks”. Computing Surveys, Vol. 3, No. 2, June 1971.

5. Four necessary and sufficient conditions • Serially reusable resources: • the processes involved share resources which they use under mutual exclusion, repeatedly. • Incremental acquisition: • processes hold on to resources already allocated to them while waiting to acquire additional resources. • No pre-emption: • once acquired by a process, resources cannot be pre-empted (forcibly withdrawn) but are only released voluntarily. • Wait-for cycle: • a circular chain (or cycle) of processes exists such that each process holds a resource which its successor in the cycle is waiting to acquire.

6. Wait-for cycle (think dining Philosophers) Has A awaits B A Has E awaits A Has B awaits C E B C D Has C awaits D Has D awaits E Based on Magee/Kramer

7. Avoiding deadlock • Ensure that at least one of the four conditions does not hold! • Serially reusable resources • Often cannot avoid this! E.g. a scanner and a printer • Incremental acquisition • If processes have to hold more than one resource at a time, make sure they all acquire the resources in the same order. • No pre-emption • Consider timeouts or the immediate release of a resource a process holds if it cannot get the following resource. • Wait-for cylce • Try asymmetric behaviour.

8. Good practice summary • Good practice when acquiring shared resources: • All processes acquire the resources in the same order. • This was not the case with our dining philosophers. • Issue with release/acquire… • One process could always immediately re-acquire a resource released due, for example, to a timeout. • … a progress (fairness) issue. • Consider FIFO queue of processes waiting to acquire a resource. • Use libraries that provide this (or provide control over the behaviour), such as the pthread library. • Check their behaviour is what you require!

9. Channels in Promela • So far we have considered systems with global shared variables. • Promela can also be used to model distributed systems where, for example, computers are geographically distributed and some form of explicit message passing is required to exchange data. • E.g. internet-based client-server systems using web protocols and scientific distributed computing using MPI or Grid computing. • Promela supports Channels for this purpose. • We abstract away from the details of the network and its protocols to focus on the interaction between processes.

10. Channels support a variety of mechanisms to exchange data between processes explicitly. • We will see an implementation of a simple tuple space in the next lab exercise using channels.

11. Channels - basics • In Promela, a channel is a global entity so any process can send and receive on it. • In some systems channels are restricted to work between a specific pair of processes. • In Promela a channel is a datatype supporting two basic operations • Send (chan1 ! x, y, z) and Receive (chan1 ? p, q, r) • Channels have a message type • Once a channel is initialised it can only send and receive messages of its message type • At most 255 channels can be created (keep models small!)

12. Channel initialisation • Channels have a capacity and a message type. • These are declared in a channel initializer (i.e. in a channel declaration): • chan ch = [capacity] of {typename, typename,…} • E.g. a channel could have a message type: byte, byte, bool… • Think of a structure declaration in C. • There are two types of channel • A channel with a capacity of zero is known as a rendezvous channel. • A channel with a capacity greater than zero is known as a buffered channel.

13. Simple client-server example chan request = [0] of {byte} active proctype Server() { byte client; end: do :: request ? client -> printf (“Client %d\n”, client); od } active proctype Client0() { request ! 0; } active proctype Client1() { request ! 1; }

14. Notes on semantics of channels • The send statement consists of a channel variable followed by an exclamation point and then a sequence of expressions whose number and types match the message type of the channel. • The receive statement consists of a channel variable followed by a question mark and a sequence of variables. • Semantically, the expressions in the send statement are evaluated and their values transferred through the channel; the receive statement assigns these values to the variables listed. • A receive statement cannot be executed unless there is a message in the channel. • It will block until this is the case – hence, receive statements often appear as guards in an if or do statement. • What happens on a send depends on the type of the channel • Rendezvous (capacity 0) or buffered (capacity > 0).

15. Notes on channel example… • Note the use of the end: label in the server… • Often Client processes will have a limited lifetime. They may come into being, send and receive some messages and then terminate. • A Server often doesn’t know when it will be required to respond to messages sent to it by clients, so it should never terminate (or terminate only when told).

16. Notes continued… • The end: label ensures that an end state with the Server blocked on a receive is not considered as an invalid end state by Spin! • This type of end is expected (by the designer) in client-server systems. • Any label beginning with ‘end’ serves this purpose.

17. Rendezvous channels • For a channel with zero capacity, the exchange of data is synchronous. • That is, the exchange takes place when both the sender and receiver are ready. • If one isn’t ready, the other will block until it is. • The transfer then takes place in one atomic operation. • This is the only operation in Promela where the control pointers of two processes are incremented at the same time! • The two processes are said to engage in a rendezvous.

18. Some other useful things • Promela supports the type mtype which can be used to name the type of message supported by a channel: • Channels also support the use of an anonymous variable (_): • (useful for rendezvous when you just want to know a rendezvous has occurred but don’t care about the value of a message – i.e. in a ‘handshake’: reply ? _ mtype {open, close, reset} chan ch [0] of {mtype, byte} byte id; ch ! open, id; ch ! close id;

19. Receiver Sender (green, 10, true) (green, 20, false) (colour, time, flash) (red, 10, false) Buffered channels – capacity non zero • The capacity defines how many messages can be held in the channel. • By default the send and receive statements treat the channel as a FIFO (first-in-first-out). For example: Chan ch = [3] of {mtype, byte, bool} Colour: red, yellow or green. e.g. controlling a traffic light…

20. Random receive (??) and matching • With buffered channels it is possible for a process to receive from any position in the channel, rather than from the ‘head’ of the FIFO queue. • Use random receive (?? Instead of ?) to receive an appropriate message from anywhere in the channel. • If multiple messages are available, the first one found is returned. • It is also possible to receive only a message which matches (specific values of some variables in) a request…

21. Random receive and eval() • By default, a receive statement removes a message from the channel regardless of its contents and assigns the values to the variables in the receive statement. • There is a mechanism to allow a receive statement to specify values for some fields instead of variables. • The first message found in the channel with values that match the values specified in the receive statement will be returned. • For example, a receiver may request a ‘red’ message with a ‘time=20’ and will use whatever value of flash is in the message. • When a matched receive completes the variables in the receive statement are assigned the appropriate values in the message, as usual. • The eval() function can be used to return the current value of a variable in the receiver process for use in the matching…

22. Example chan ch • In the random receive, the message matching ‘red’ and ’20’ will be returned. • eval(time) returns the value of the variable time. • The variable flash in the receiver will be set to ‘true’. Receiver (green, 20, false) Sender (red, 20, true) (green, 5, true) mtype {red, green} byte time=20; bool flash; ch ?? red, eval(time), flash (red, 10, false)

23. Copying vs removal of a message • A receive can take a copy of the values of a message and leave the message in the channel, rather than removing it from the channel. • Use angled brackets around the message: • ch ? <colour, time, flash> • ch ?? <colour, time, flash>

24. Polling or Non-blocking read • Buffered channels can be polled to see if a channel contains a message of a certain ‘type’. • A polling receive uses square brackets around the message, returns true or false and does not block. • No variables are changed (side-effect free). • Suitable for use in guard statements: do :: ch ?? [green, time, false] -> ch ?? green, time, false :: else -> /* do something else */ od

25. Other channel functions • Check the number of messages in a channel using: • len (ch) • Check whether a channel is full or empty: • full(ch), empty(ch), nfull(ch), nempty(ch) • (use nfull and nempty for not-full, not-empty, rather than !full() etc.

26. Conclusion • Examined the four necessary conditions for deadlock to occur and how to avoid it. • Reviewed the use of channels in Promela for modelling distributed systems. • We will see channels in lab exercise 5 where we consider termination algorithms. A channel-based implementation of a tuplespace will be compared with a pthread-based coded version. • Finally, we will re-examine some attempted solutions to the critical section problem using the support for verification in jSpin.