Introduction to Distributed Systems: Algorithms & Models (Spring 2014)

1. Several sets of slides by Prof. Jennifer Welch will be used in this course. The slides are mostly identical to her slides, with some minor changes. Set 1: Introduction

2. Set 1: Introduction DISTRIBUTED ALGORITHMS Spring 2014 Prof. Jennifer Welch

3. Distributed Systems • Distributed systems have become ubiquitous: • share resources • communicate • increase performance • speed • fault tolerance • Characterized by • independent activities (concurrency) • loosely coupled parallelism (heterogeneity) • inherent uncertainty Set 1: Introduction

4. Uncertainty in Distributed Systems • Uncertainty comes from • differing processor speeds • varying communication delays • (partial) failures • multiple input streams and interactive behavior Set 1: Introduction

5. Reasoning about Distributed Systems • Uncertainty makes it hard to be confident that system is correct • To address this difficulty: • identify and abstract fundamental problems • state problems precisely • design algorithms to solve problems • prove correctness of algorithms • analyze complexity of algorithms (e.g., time, space, messages) • prove impossibility results and lower bounds Set 1: Introduction

6. Potential Payoff of Theoretical Paradigm • careful specifications clarify intent • increased confidence in correctness • if abstracted well then results are relevant in multiple situations • indicate inherent limitations • cf. NP-completeness Set 1: Introduction

7. Application Areas • These areas have provided classic problems in distributed/concurrent computing: • operating systems • (distributed) database systems • software fault-tolerance • communication networks • multiprocessor architectures • Newer application areas: • cloud computing • mobile computing, … Set 1: Introduction

8. Basic Models • Introduce two basic communication models: • message passing • shared memory • and two basic timing models: • synchronous • asynchronous Set 1: Introduction

9. Basic Models Message passing Shared memory synchronous Yes No asynchronous Yes Yes (Synchronous shared memory model is PRAM) Set 1: Introduction

10. Relationship of Theory to Practice • time-shared operating systems: issues relating to (virtual) concurrency of processes such as • mutual exclusion • deadlock also arise in distributed systems • MIMD multiprocessors: • no common clock => asynchronous model • common clock => synchronous model • loosely coupled networks, such as Internet, => asynchronous model Set 1: Introduction

11. Relationship of Theory to Practice • Failure models: • crash: faulty processor just stops. Idealization of reality. • Byzantine (arbitrary): conservative assumption, fits when failure model is unknown or malicious • self-stabilization: algorithm automatically recovers from transient corruption of state; appropriate for long-running applications Set 1: Introduction

12. Message-Passing Model • processors are p0, p1, …, pn-1 (nodes of graph) • bidirectional point-to-point channels (undirected edges of graph) • each processor labels its incident channels 1, 2, 3,…; might not know who is at other end Set 1: Introduction

13. Message-Passing Model 1 1 p3 p0 3 2 2 2 p2 p1 1 1 Set 1: Introduction

14. Modeling Processors and Channels • Processor is a state machine including • local state of the processor • mechanisms for modeling channels • Channel directed from processor pi to processor pj is modeled in two pieces: • outbuf variable of pi and • inbuf variable of pj • Outbuf corresponds to physical channel, inbuf to incoming message queue Set 1: Introduction

15. inbuf[1] outbuf[2] p1's local variables p2's local variables outbuf[1] inbuf[2] Modeling Processors and Channels Pink area (local vars + inbuf) is accessible state for a processor. Set 1: Introduction

16. Configuration • Vector of processor states (including outbufs, i.e., channels), one per processor, is a configuration of the system • Captures current snapshot of entire system: accessible processor states (local vars + incoming msg queues) as well as communication channels. Set 1: Introduction

17. p1 m3 m2 m1 p2 Deliver Event • Moves a message from sender's outbuf to receiver's inbuf; message will be available next time receiver takes a step p1 m3 m2 m1 p2 Set 1: Introduction

18. Computation Event • Occurs at one processor • Start with old accessible state (local vars + incoming messages) • Apply transition function of processor's state machine; handles all incoming messages • End with new accessible state with empty inbufs, and new outgoing messages Set 1: Introduction

19. Computation Event b a old local state new local state c d e pink indicates accessible state: local vars and incoming msgs white indicates outgoing msg buffers Set 1: Introduction

20. Execution • Format is config, event, config, event, config, … • in first config: each processor is in initial state and all inbufs are empty • for each consecutive (config, event, config), new config is same as old config except: • if delivery event: specified msg is transferred from sender's outbuf to receiver's inbuf • if computation event: specified processor's state (including outbufs) change according to transition function Set 1: Introduction

21. Admissibility • Definition of execution gives some basic "syntactic" conditions. • usually safety conditions (true in every finite prefix) • Sometimes we want to impose additional constraints • usually liveness conditions (eventually something happens) • Executions satisfying the additional constraints are admissible. These are the executions that must solve the problem of interest. • Definition of “admissible” can change from context to context, depending on details of what we are modeling Set 1: Introduction

22. Asynchronous Executions • An execution is admissible for the asynchronous model if • every message in an outbuf is eventually delivered • every processor takes an infinite number of steps • No constraints on when these events take place: arbitrary message delays and relative processor speeds are not ruled out • Models reliable system (no message is lost and no processor stops working) Set 1: Introduction

23. Example: Flooding • Describe a simple flooding algorithm as a collection of interacting state machines. • Each processor's local state consists of variable color, either red or green • Initially: • p0: color = green, all outbufs contain M • others: color = red, all outbufs empty • Transition: If M is in an inbuf and color = red, then change color to green and send M on all outbufs Set 1: Introduction

24. p0 p0 M M M M p2 p1 deliver event at p1from p0 p2 p1 computation event by p1 p0 p0 M M M M computation event by p2 deliver event at p2from p1 p2 p1 p2 p1 M M Example: Flooding Set 1: Introduction

25. p0 p0 M M M M M M p2 p1 deliver event at p1from p2 p2 p1 computation event by p1 M M p0 p0 M M M M M M etc. to deliver rest of msgs p2 p1 deliver event at p0from p1 p2 p1 Example: Flooding (cont'd) Set 1: Introduction

26. Nondeterminism • The previous execution is not the only admissible execution of the Flooding algorithm on that triangle. • There are several, depending on the order in which messages are delivered. • For instance, the message from p0 could arrive at p2 before the message from p1 does. Set 1: Introduction

27. Termination • For technical reasons, admissible executions are defined as infinite. • But often algorithms terminate. • To model algorithm termination, identify terminated states of processors: states which, once entered, are never left • Execution has terminated when all processors are terminated and no messages are in transit (in inbufs or outbufs) Set 1: Introduction

28. Termination of Flooding Algorithm • Define terminated processor states as those in which color = green. Set 1: Introduction

29. Synchronous Message PassingSystems • An execution is admissible for the synchronous model if it is an infinite sequence of "rounds" • What is a "round"? • It is a sequence of deliver events that move all messages in transit into inbuf's, followed by a sequence of computation events, one for each processor. Set 1: Introduction

30. Synchronous Message Passing Systems • The new definition of admissible captures lockstep unison feature of synchronous model. • This definition also implies • every message sent is delivered • every processor takes an infinite number of steps. • Time is measured as number of rounds until termination. Set 1: Introduction

31. Example of Synchronous Model • Suppose flooding algorithm is executed in synchronous model on the triangle. • Round 1: • deliver M to p1 from p0 • deliver M to p2 from p0 • p0 does nothing (as it has no incoming messages) • p1 receives M, turns green and sends M to p0 and p1 • p2 receives M, turns green and sends M to p0 and p1 Set 1: Introduction

32. Example of Synchronous Model • Round 2: • deliver M to p0 from p1 • deliver M to p0 from p2 • deliver M to p1 from p2 • deliver M to p2 from p1 • p0 does nothing since its color variable is already green • p1 does nothing since its color variable is already green • p2 does nothing since its color variable is already green Set 1: Introduction

33. p0 p0 M M p2 p1 round 1 events p2 p1 round 2 events p0 M M p2 p1 M M Example of Synchronous Model Set 1: Introduction