1 / 35

Distributed Systems Course Distributed transactions

Distributed Systems Course Distributed transactions. 13.1 Introduction 13.2 Flat and nested distributed transactions 13.3 Atomic commit protocols 13.4 Concurrency control in distributed transactions 13.5 Distributed deadlocks 13.6 Transaction recovery.

maxine
Download Presentation

Distributed Systems Course Distributed transactions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Systems CourseDistributed transactions 13.1 Introduction 13.2 Flat and nested distributed transactions 13.3 Atomic commit protocols 13.4 Concurrency control in distributed transactions 13.5 Distributed deadlocks 13.6 Transaction recovery

  2. Commitment of distributed transactions - introduction • a distributed transaction refers to a flat or nested transaction that accesses objects managed by multiple servers • When a distributed transaction comes to an end • the either all of the servers commit the transaction • or all of them abort the transaction. • one of the servers is coordinator, it must ensure the same outcome at all of the servers. • the ‘two-phase commit protocol’ is the most commonly used protocol for achieving this 2 •

  3. (a) Flat transaction (b) Nested transactions M X T 11 X T Client N 1 T T Y 12 T T T 21 T 2 Client Y P Z Figure 13.1 T 22 Distributed transactions In a nested transaction, the top-level transaction can open subtransactions, and each subtransaction can open further subtransactions down to any depth of nesting In the nested case, subtransactions at the same level can run concurrently, so T1 and T2 are concurrent, and as they invoke objects in different servers, they can run in parallel. A flat client transaction completes each of its requests before going on to the next one. Therefore, each transaction accesses servers’ objects sequentially 3 •

  4. X Client a.withdraw(10) A T 1 T Y Figure 13.2 T = openTransaction openSubTransaction T B b.withdraw(20) 2 a.withdraw(10); openSubTransaction b.withdraw(20); Z openSubTransaction . T c deposit(10) C c.deposit(10); 3 openSubTransaction d.deposit(20) D d.deposit(20); T 4 closeTransaction Nested banking transaction • client transfers $10 from A to C and then transfers $20 from B to requests can be run in parallel - with several servers, the nested transaction is more efficient 4 •

  5. Why might a participant abort a transaction? The coordinator of a flat distributed transaction • Servers execute requests in a distributed transaction • when it commits they must communicate with one another to coordinate their actions • a client starts a transaction by sending an openTransaction request to a coordinator in any server (next slide) • it returns a TID unique in the distributed system(e.g. server ID + local transaction number) • at the end, it will be responsible for committing or aborting it • each server managing an object accessed by the transaction is a participant - it joins the transaction (next slide) • a participant keeps track of objects involved in the transaction • at the end it cooperates with the coordinator in carrying out the commit protocol • note that a participant can call abortTransaction in coordinator 5 •

  6. openTransaction goes to the coordinator join openTransaction participant closeTransaction A a.withdraw(4); . . join BranchX T participant b.withdraw(T, 3); Client B b.withdraw(3); T = openTransaction BranchY join a.withdraw(4); c.deposit(4); participant b.withdraw(3); c.deposit(4); d.deposit(3); C closeTransaction Figure 13.3 D d.deposit(3); Note: the coordinator is in one of the servers, e.g. BranchX BranchZ A flat distributed banking transaction a client’s (flat) banking transaction involves accounts A, B, C and D at servers BranchX, BranchY and BranchZ • Note that the TID (T) is passed with each request e.g. withdraw(T,3) Each server is shown with a participant, which joins the transaction by invoking the join method in the coordinator 6 •

  7. The join operation • The interface for Coordinator is shown in Figure 12.3 • it has openTransaction, closeTransaction and abortTransaction • openTransaction returns a TID which is passed with each operation so that servers know which transaction is accessing its objects • The Coordinator interface provides an additional method, join, which is used whenever a new participant joins the transaction: • join(Trans, reference to participant) • informs a coordinator that a new participant has joined the transaction Trans. • the coordinator records the new participant in its participant list. • the fact that the coordinator knows all the participants and each participant knows the coordinator will enable them to collect the information that will be needed at commit time. 7 •

  8. Atomic commit protocols • transaction atomicity requires that at the end, • either all of its operations are carried out or none of them. • in a distributed transaction, the client has requested the operations at more than one server • one-phase atomic commit protocol • the coordinator tells the participants whether to commit or abort • what is the problem with that? • this does not allow one of the servers to decide to abort – it may have discovered a deadlock or it may have crashed and been restarted • two-phase atomic commit protocol • is designed to allow any participant to choose to abort a transaction • phase 1 - each participant votes. If it votes to commit, it is prepared. It cannot change its mind. In case it crashes, it must save updates in permanent store • phase 2 - the participants carry out the joint decision The decision could be commit or abort - participants record it in permanent store • 8

  9. The two-phase commit protocol Why does participant record updates in permanent storage at this stage? How many messages are sent between the coordinator and each participant? • During the progress of a transaction, the only communication between coordinator and participant is the join request • The client request to commit or abort goes to the coordinator • if client or participant request abort, the coordinator informs the participants immediately • if the client asks to commit, the 2PC comes into use • 2PC • voting phase: coordinator asks all participants if they can commit • if yes, participant records updates in permanent storage and then votes • completion phase: coordinator tells all participants to commit or abort • the next slide shows the operations used in carrying out the protocol • 9

  10. canCommit?(trans)-> Yes / No • Call from coordinator to participant to ask whether it can commit a transaction. Participant replies with its vote. • doCommit(trans) • Call from coordinator to participant to tell participant to commit its part of a transaction. • doAbort(trans) • Call from coordinator to participant to tell participant to abort its part of a transaction. • haveCommitted(trans, participant) • Call from participant to coordinator to confirm that it has committed the transaction. • getDecision(trans) -> Yes / No • Call from participant to coordinator to ask for the decision on a transaction after it has voted Yes but has still had no reply after some delay. Used to recover from server crash or delayed messages. These are asynchronous requests to avoid delays Figure 13.4 Operations for two-phase commit protocol This is a request with a reply • participant interface- canCommit?, doCommit, doAbortcoordinator interface- haveCommitted, getDecision Asynchronous request • 10

  11. Figure 13.5 The two-phase commit protocol • Phase 1 (voting phase): • 1. The coordinator sends a canCommit? request to each of the participants in the transaction. • 2. When a participant receives a canCommit? request it replies with its vote (Yes or No) to the coordinator. Before voting Yes, it prepares to commit by saving objects in permanent storage. If the vote is No the participant aborts immediately. • Phase 2 (completion according to outcome of vote): • 3. The coordinator collects the votes (including its own). • (a)If there are no failures and all the votes are Yes the coordinator decides to commit the transaction and sends a doCommit request to each of the participants. • (b)Otherwise the coordinator decides to abort the transaction and sends doAbort requests to all participants that voted Yes. • 4. Participants that voted Yes are waiting for a doCommit or doAbort request from the coordinator. When a participant receives one of these messages it acts accordingly and in the case of commit, makes a haveCommitted call as confirmation to the coordinator. 11 •

  12. Coordinator Participant step status step status canCommit? prepared to commit 1 Yes (waiting for votes) 2 prepared to commit (uncertain) doCommit 3 committed haveCommitted 4 committed Figure 13.6 done Communication in two-phase commit protocol Think about the coordinator in step 1 - what is the problem? Think about step 2 - what is the problem for the participant? Think about participant before step 2 - what is the problem? • Time-out actions in the 2PC • to avoid blocking forever when a process crashes or a message is lost Step 2 - participant is uncertain. E.g. coordinator may have crashed Before step 2. Maybe coordinator has crashed In step 1 maybe some participants have crashed • 12

  13. Performance of the two-phase commit protocol • if there are no failures, the 2PC involving N participants requires • NcanCommit? messages and replies, followed by NdoCommit messages. • the cost in messages is proportional to 3N, and the cost in time is three rounds of messages. • The haveCommitted messages are not counted • there may be arbitrarily many server and communication failures • 2PC is is guaranteed to complete eventually, but it is not possible to specify a time limit within which it will be completed • delays to participants in uncertain state • some 3PCs designed to alleviate such delays • they require more messages and more rounds for the normal case • 13

  14. 13.3.2 Two-phase commit protocol for nested transactions • Recall Fig 13.1b, top-level transaction T and subtransactions T1, T2, T11, T12, T21, T22 • A subtransaction starts after its parent and finishes before it • When a subtransaction completes, it makes an independent decision either to commit provisionally or to abort. • A provisional commit is not the same as being prepared: it is a local decision and is not backed up on permanent storage. • If the server crashes subsequently, its replacement will not be able to carry out a provisional commit. • A two-phase commit protocol is needed for nested transactions • it allows servers of provisionally committed transactions that have crashed to abort them when they recover. • 14

  15. Figure 13.7Operations in coordinator for nested transactions The TID of a subtransaction is an extension of its parent's TID, so that a subtransaction can work out the TID of the top-level transaction. The client finishes a set of nested transactions by calling closeTransaction or abortTransacation in the top-level transaction. • openSubTransaction(trans) -> subTrans • Opens a new subtransaction whose parent is trans and returns a unique subtransaction identifier. • getStatus(trans)-> committed, aborted, provisional • Asks the coordinator to report on the status of the transaction trans. Returns values representing one of the following: committed, aborted, provisional. • This is the interface of the coordinator of a subtransaction. • It allows it to open further subtransactions • It allows its subtransactions to enquire about its status • Client starts by using OpenTransaction to open a top-level transaction. • This returns a TID for the top-level transaction • The TID can be used to open a subtransaction • The subtransaction automatically joins the parent and a TID is returned. • 15

  16. T abort (at M) 11 T provisional commit (at X) 1 T provisional commit (at N) T 12 provisional commit (at N) T 21 aborted (at Y) T 2 T provisional commit (at P) 22 Transaction T decides whether to commit T12 has provisionally committed and T11 has aborted, but the fate of T12 depends on its parent T1 and eventually on the top-level transaction, T. Although T21 and T22 have both provisionally committed, T2 has aborted and this means that T21 and T22 must also abort. Suppose that T decides to commit although T2 has aborted, also that T1 decides to commit although T11 has aborted • Recall that • A parent can commit even if a subtransaction aborts • If a parent aborts, then its subtransactions must abort • In the figure, each subtransaction has either provisionally committed or aborted Figure 13.8 • 16

  17. 2PC formal algorithm: Phase 1: Obtaining a Decision • Coordinator asks all participants to prepareto commit transaction T. • Ci adds the records <prepare T> to the log and forces log to stable storage. • sends prepare T messages to all sites at which T executed. • Upon receiving message, transaction manager at site determines if it can commit the transaction. • if not, add a record <no T> to the log and send abort T message to Ci • if the transaction can be committed, then: • add the record <ready T> to the log • force all records for T to stable storage • send readyT message to Ci

  18. Phase 2: Recording the Decision T can be committed if Cireceived a ready T message from all the participating sites: otherwise T must be aborted. Coordinator adds a decision record, <commit T> or <abort T>, to the log and forces record onto stable storage. Once the record is in stable storage it is irrevocable (even if failures occur). Coordinator sends a message to each participant informing it of the decision (commit or abort). Participants take appropriate action locally.

  19. Handling of Failures – Site Failure (Sk) When site Skrecovers, it examines its log to determine the fate of transactions active at the time of the failure. • Log contain <commit T> record: site executes redo (T) • Log contains <abort T> record: site executes undo (T) • Log contains <ready T> record: site must consult Ci to determine the fate of T. • If T committed, redo (T) • If T aborted, undo (T) • The log contains no control records concerning T replies that Sk failed before responding to the prepare T message from Ci • since the failure of Skprecludes the sending of such a response Cimust abort T • Skmust execute undo (T)

  20. Handling of Failures – Coordinator’s Failure (Ci) • If coordinator Ci fails while the commit protocol for T is executing then participating sites must decide on T’s fate: • If an active site contains a <commit T> record in its log, then T must be committed. • If an active site contains an <abort T> record in its log, then T must be aborted. • If some active participating site does not contain a <ready T> record in its log, then the failed coordinator Ci cannot have decided to commit T. Can therefore abort T. • If none of the above cases holds, then all active sites must have a <ready T> record in their logs, but no additional control records (such as <abort T> of <commit T>). In this case active sites must wait for Cito recover, to find decision. • Blocking problem : active sites may have to wait for failed coordinator to recover.

  21. Concurrency Control • Modify concurrency control schemes for use in distributed environment. • We assume that each site participates in the execution of a commit protocol to ensure global transaction atomicity. • We assume all replicas of any item are updated immediately. • Concurrency Control techniques: • Locking.

  22. Locking in a Distributed Environment Sk X Sj Si Network X T T Begin Xlock(x) : : Sm X • Same rules as in centralized. • Shared and Exclusive locks. • Each transaction follows 2PL. • Problem when requesting locks on replicated data. • Solutions: • Single Lock Manager. • Distributed Lock Manager. • Primary Copy Protocol. • Majority Protocol.

  23. Single Lock Manager • System maintains a single lock manager that resides in a single chosen site, say Si . • When a transaction needs to lock a data item, it sends a lock request to Si and lock manager determines whether the lock can be granted immediately: • If yes, lock manager sends a message to the site which initiated the request. • If no, request is delayed until it can be granted, at which time a message is sent to the initiating site.

  24. Single Lock Manager • Once it gets the lock, the transaction can read the data item from any one of the sites at which a replica of the data item resides. • Writes must be performed on all replicas of a data item. • Advantages of scheme: • Simple implementation. • Simple deadlock handling. • Disadvantages of scheme are: • Bottleneck: lock manager site becomes a bottleneck. • Vulnerability: system is vulnerable to lock manager site failure.

  25. Distributed Lock Manager • In this approach, functionality of locking is implemented by lock managers at each site. • Lock managers control access to local data items. • But special protocols may be used for replicas. • Advantage: work is distributed and can be made robust to failures. • Disadvantage: deadlock detection is more complicated • Lock managers cooperate for deadlock detection. • More on this later. • Two variants of this approach: • Primary copy. • Majority protocol.

  26. Primary Copy • Choose one replica of data item to be the primary copy. • Site containing the replica is called the primary site for that data item. • Different data items can have different primary sites. • When a transaction needs to lock a data item Q, it requests a lock at the primary site of Q. • Implicitly gets lock on all replicas of the data item. • After it gets the lock from primary site, it can read from any of the replicas. • Advantage: Concurrency control for replicated data handled similarly to unreplicated data - simple implementation. • Disadvantage: If the primary site of Q fails, Q is inaccessible even though other sites containing a replica may be accessible.

  27. Majority Protocol • Local lock manager at each site administers lock and unlock requests for data items stored at that site. • When a transaction wishes to lock an unreplicated data item Q residing at site Si, a message is sent to Si ‘s lock manager. • If Q is locked in an incompatible mode, then the request is delayed until it can be granted. • When the lock request can be granted, the lock manager sends a message back to the initiator indicating that the lock request has been granted.

  28. Majority Protocol • In case of replicated data • If Q is replicated at n sites, then a lock request message must be sent to more than half of the n sites in which Q is stored. • The transaction does not operate on Q until it has obtained a lock on a majority of the replicas of Q. • When writing the data item, transaction performs writes on all replicas. • Advantage: • Can be used even when some sites are unavailable • details on how handle writes in the presence of site failure later • Disadvantage: • Requires 2(n/2 + 1) messages for handling lock requests, and (n/2 + 1) messages for handling unlock requests. • Potential for deadlock even with single item - e.g., each of 3 transactions may have locks on 1/3rd of the replicas of a data.

  29. Consider the following two transactions and history, with item X and transaction T1 at site 1, and item Y and transaction T2 at site 2: Deadlock Handling T2: write(Y) write(X) T1: write(X) write(Y) T2 at site 2 T1 at site 1 XLock(X) write(X) Xlock(Y) Wait Xlock(Y) write(Y) Xlock(X) Wait Result: deadlock which cannot be detected locally at either site.

  30. Deadlock Handling Coordinator Site Global WFG T1 T2 Local WFG Local WFG S1 S2 Network T1 T2 T2 T1 • How to detect deadlock from last slide? • Global Wait-for Graph + Local Wait-for Graphs. • Global WFG: • Constructed and maintained in a single site; the deadlock-detection coordinator. • 2 Graphs: • Real graph: Real, but unknown, state of the system. • Constructed graph: Approximation generated by the coordinator through combining local WFGs.

  31. Deadlock Handling • The global wait-for graph can be constructed when: • a new edge is inserted in or removed from one of the local wait-for graphs. OR • a number of changes have occurred in a local wait-for graph. OR • the coordinator needs to invoke cycle-detection. • If the coordinator finds a cycle: • It selects a victim and notifies all sites. • The sites roll back the victim transaction.

  32. Local and Global Wait-For Graphs Local Global

  33. Example Wait-For Graph for False Cycles Initial state:

  34. Suppose that starting from the state shown in figure, 1. T2 releases resources at S1 resulting in a message remove T1 T2 message from the Transaction Manager at site S1 to the coordinator) 2. And then T2 requests a resource held by T3 at site S2 resulting in a message insert T2 T3 from S2 to the coordinator Suppose further that the insert message reaches before the delete message this can happen due to network delays. The coordinator would then find a false cycle T1  T2  T3  T1 The false cycle above never existed in reality. False Cycles

  35. Unnecessary rollbacks may result when deadlock has indeed occurred and a victim has been picked, and meanwhile one of the transactions was aborted for reasons unrelated to the deadlock. Unnecessary rollbacks can result from false cycles in the global wait-for graph. Unnecessary Rollbacks

More Related