On the Complexity of Buffer Allocation in Message Passing Systems

On the Complexity of Buffer Allocation in Message Passing Systems Joint work with Jan B. Pedersen & Alan Wagner Alex Brodsky University of British Columbia

Outline • Motivation • Definitions • Buffer Allocation Problem • Buffer Sufficiency Problem • Nonblocking Buffer Allocation Problem • Other Models and Related Problems • Conclusion

Motivation

What is the Problem? send(p2,...) send(p1,...) recv(p2,...) recv(p1,...) Unless there is somewhere to put the message the senders will deadlock. So....

What is the Problem? send(p2,...) send(p1,...) recv(p2,...) recv(p1,...) Unless there is somewhere to put the message the senders will deadlock. So, buffers are used.

What is the Problem? send(p2,...) send(p1,...) send(p2,...) send(p1,...) send(p2,...) send(p1,...) recv(p1,...) recv(p2,...)

Problem Statement • Not all systems have unrestricted amounts of buffers. • e.g., clusters that offload message passing functionality to the network interface card (NIC). • Hence, we must determine the number of buffers needed for a safe program execution. • This is the Buffer Allocation Problem (BAP). • Question: What is the complexity of BAP?

Assumptions

Assumptions • Processes are asynchronous.

Assumptions • Processes are asynchronous. • The communication pattern is static. • i.e., doesn't change from execution to execution.

Assumptions • Processes are asynchronous. • The communication pattern is static. • i.e., doesn't change from execution to execution. • Send/recv calls are explicitly matched. send(p2,...) recv(p1,...)

Assumptions • Processes are asynchronous. • The communication pattern is static. • i.e., doesn't change from execution to execution. • Send/recv calls are explicitly matched. • Buffers are allocated on the receiver.

Assumptions • Processes are asynchronous. • The communication pattern is static. • i.e., doesn't change from execution to execution. • Send/recv calls are explicitly matched. • Buffers are allocated on the receiver. • Sends block if • no buffers are available • & receiver is not ready.

Problem Input

Problem Input • What is the invariant across executions of a program?

Problem Input • What is the invariant across executions of a program? • The static communication pattern

Problem Input • What is the invariant across executions of a program? • The static communication pattern. • Use communication graphs to represent communication patterns. • The communication graph becomes the problem input.

Communication Graph P0 P1 P2 P3 P4 P5 Time Process component Processes are denoted by vertical process arcs (up to down).

Communication Graph P0 P1 start Event send recv end Events (start, end, send, receive) are denoted by vertices.

Communication Graph P0 P1 send Communication arcs denote sends from one process to another.

Communication Graph P0 P1 P2 P3 P4 P5 Examples of communication graphs.

Arrival != Receipt P0 P1 P2 P3 Arrival interval Receive event occurs when message is received.

Arrival != Receipt P0 P1 P2 P3 Arrival interval Messages can arrive before receive events.

Dependencies P0 P1 All events depend on start events.

Dependencies P0 P1 Receive events depend on send events.

Dependencies P0 P1 If there are NO buffers, send events depend on receive events.

Dependencies P2 P3 A send event depends on the preceding event.

Dependencies P2 P3 The arrival interval is defined by a receive event and its dependency within the same process.

Circular Dependency & Deadlock P4 P5 With no buffers, the send/receive events depend on each other.

Circular Dependency & Deadlock P4 P5 A circular dependency (with no buffers) represents deadlock.

The t-ring P0 P1 P2 P3 P4 P5 We call such a circular dependency a t-ring, e.g., t=6.

Solving Deadlock P0 P1 P2 P3 P4 P5 0 1 0 2 0 0 To solve deadlock, we use buffers.

Buffer Assignment P0 P1 P2 P3 P4 P5 0 1 0 2 0 0 Each process is assigned 0 or more buffers.

Solving Deadlock P0 P1 0 1 Initially, neither process can complete a send.

Solving Deadlock P0 P1 0 1 Message from process 0 is buffered by process 1.

Solving Deadlock P0 P1 0 1 Process 0 can proceed to receive from process 1.

Solving Deadlock P0 P1 0 1 Finally, process 1 receives from process 0.

Solving Deadlock P2 P3 P4 P5 0 2 0 0 Initially, none of the sends can complete.

Solving Deadlock P2 P3 P4 P5 0 2 0 0 Since message arrival is nondeterministic, 2 buffers are needed.

Solving Deadlock P2 P3 P4 P5 0 2 0 0 Process P4 can complete its receives.

Solving Deadlock P2 P3 P4 P5 0 2 0 0 Process P3 can complete its receives.

A program is k-safe if k buffers are sufficient to guarantee deadlock free execution. Safety

Informal Question: How many buffers does a program need to avoid deadlock? Formal Question: Given a communication graph, how many buffers are needed to avoid deadlock in the corresponding program? Decision Question (BAP): Given a communication graph and integer k, is the corresponding program k-safe? Buffer Allocation Problem (BAP)

Thm: BAP is NP-hard • Proof by reduction from 3SAT • 3SAT Decision Problem: Does the formula of the form ∧i (ai ∨ bi∨ ci) have a satisfying assignment where each ai, bi, ci, is a either a variable xj or its negation, (n variables). • Idea: For any 3SAT formula we show how to construct a corresponding communication graph to test n-safety (requires n buffers). • 2 widgets: fix assignment and check clauses

The Construction x0 ~x0 x1 ~x1 x2 ~x2 x3 ~x3 For a formula over n variables create a graph with 2n processes.

Fixing the assignment x0 ~x0 x1 ~x1 x2 ~x2 x3 ~x3 The 2-rings are used to fix a variable assignment.

Fixing the assignment x0 ~x0 x1 ~x1 x2 ~x2 x3 ~x3 0 1 1 0 1 0 0 1 A buffer assignment fixes the variables, e.g., ~x0, x1, x2, ~x3. No more than n buffers may be selected, (testing for n-safety).

Use a 3-ring for each Clause x0 ~x0 x1 ~x1 x2 ~x2 x3 ~x3 0 1 1 0 1 0 0 1 (x0+x1+x3) (x0+~x2+x3) Each clause is represented by a 3-ring. Which will not deadlock only if one of the processes has a buffer.

On the Complexity of Buffer Allocation in Message Passing Systems