Introduction to Atomic Transactions

1. Introduction to Atomic Transactions • Low level synchronization use semaphores for mutual exclusion, critical region management, deadlock prevention, and crash recovery. • We want to have a high level abstraction for synchronization such that all of the above are hidden. • We do have such abstraction and is called atomic transaction, or transaction, or atomic action. • Atomic transaction = all-or-nothing • One process wants to begin a transaction with one or more processes. • They can negotiate various options, create and delete objects, and perform operations. • Then the initiator announces that it want all the others to commit themselves to the work done so far. • If all of them agree, the results are made permanent. • If one or more processes refuse (or crash before agreement), the situation reverts to exactly the state it had before the transaction began, with all side effects on objects, files, databases, magically wiped out.

2. Introduction to Atomic Transaction (continue) • The idea of atomic transaction came from magnetic tapes (60s) • If a run failed, all the tapes could be rewound & the job restarted with no harm done --- all-or-nothing property of an atomic transaction. • Modern Banking example: • Group the two operations together and form an atomic transaction • Withdraw(amount, account1) • Deposit(amount, account2) • Crash between the steps: account1 debited, but account2 not credited • So we want either both would be completed, or neither would be completed. The key is rolling back to the initial state if transaction fails.

3. a b d c a b’ d c a b d c Disk1 a b d c a b d c a b d X Disk2 The Transaction Model • The system has some independent processes which can fail at random. • Communication errors are handled transparently by underlying software. • Stable Storage • RAM is volatile, Disk survives CPU failures but not for disk head crashes. • Stable storage is designed to survive anything except major disasters such as floods and earthquakes. • Stable storage can be implemented with a pair of ordinary disks. (Figure 3-15) • When two corresponding blocks differ, Disk1 is the correct one (Disk1 is updated first, then Disk2) • When bad checksum is observed, copy from the corresponding block. bad checksum

4. Transaction Primitives • Programming using transactions requires special primitives that must either be supplied by the OS or the language run-time system. • 1. BEGIN_TRANSACTION: Mark the start of a transaction. • 2. END_TRANSACTION: Terminate the transaction & try to commit. • 3. ABORT_TRANSACTION: Kill the transaction; restore the old values. • 4. READ: Read data from a file (or other object). • 5. WRITE: Write data to a file (or other object). • Different kinds of objects have different primitives: • Mail: SEND, RECEIVE, FORWARD. • Accounting: READ, WRITE, Procedure calls, sub-transactions. • BEGIN_TRANSACTION and END_TRANSACTION are used to delimit the scope of a transaction. • The operations between them form the body of the transaction. • Either all of them are executed or none are executed. • These may be system calls, library procedures, or bracketing statements in a language, depending on the implementation.

5. Transaction Properties • Transactions have FOUR essential properties. Transactions are: • Atomic: To the outside world, the transaction happens indivisibly. Each transaction happens either completely, or not at all. While a transaction is in progress, other processes (whether or not they are themselves involved in transactions) cannot see any of the intermediate states. • Consistent: The transaction does not violate system invariants. E.G.. in a banking system, a key invariant is the law of conservation of money. After an internal transfer, the amount of money in the bank must be the same as it was before the transfer, but for a brief moment during the transaction, this invariant may be violated, but is not visible outside the transaction. • Isolated: Same as serializable. Concurrent transaction do not interfere with each other. The final result should looks as though all transactions ran sequentially in some (system dependent) order. • Durable: Once a transaction commits, the changes are permanent. No failure after the commit can undo the results or cause them to be lost.

6. Transaction Properties (more on serializable) • We have 3 transactions by 3 different processes at the same time. • If they were to be run sequentially, the final value of x would be 1, 2, or 3. • We see various orders, called schedules, in which they might be interleaved. • It is up to the system to ensure that individual operations are interleaved correctly. • Begin_Transaction Begin_Transaction Begin_Transaction x = 0; x = 0; x = 0; x = x + 1; x = x + 2; x = x + 3; End_Transaction End_Transaction End_Transaction • Time Schedule 1 x = 0; x = x + 1; x = 0; x = x + 2; x = 0; x = x + 3; Legal & Serializable Schedule 2 x = 0; x = 0; x = x + 1; x = x + 2; x = 0; x = x + 3; Legal Schedule 3 x = 0; x = 0; x = x + 1; x = 0; x = x + 2; x = x + 3; Illegal

7. Nested Transactions • Transactions may contain sub-transactions, often called nested transactions. • The top-level transaction may fork off children that run in parallel with one another, on different processors, to gain performance or simplify programming. • Each of these children may execute one or more sub-transactions, or fork off its own children. • Sub-transactions give rise to a subtle, but important, problem. • If a transaction starts several sub-transactions in parallel, and one of these commits, making its results visible to the parent transaction. • After further computations, the parent aborts, restoring the entire system to the state it had before the top-level transaction started. • Consequently, the results of the sub-transaction that committed must be undone. Thus the permanence referred to above applies only to top-level transactions. • Since transactions can be nested arbitrarily deeply, considerable administration is needed to get everything right. Transaction and sub-transaction have to keep a lot of private copies for rollback if necessary.

8. Atomic Transaction Implementation • Transactions sounds like a good idea, but how are they implemented? • It should be clear that if each process executing a transaction just updates the objects (data, files, records) in place, transaction will not be atomic and changes will not vanish magically if the transaction aborts. • The results of running multiple transaction will not be serializable. • So we have the following implementations for atomic transaction • Private Workspace • When a process starts a transaction, it is given a private workspace containing all the files (and other objects) to which it has access. • Until the transaction either commits or abort, all of its reads and writes go to the private workspace instead of the real one. • Writeahead Log • Files are actually updates in place. • But before the updates, a record is written to the writeahead log on stable storage telling which transaction is making the change, which objects are changed, and their old and new values are.

9. Private Workspace • Each process is given a private workspace containing all the objects. • Problems: the cost of copying everything to a private workspace is expensive. • Optimization: • Do not need to copy Read-only files. • Copy on demand • Don’t need to copy everything at first, start the workspace as empty (with a back pointer) and copy the file to private workspace for writing. • Don’t even need to copy the whole file, just copy the index to the workspace. • If the process need to write a block, a copy of the block is made and the address of the copy inserted into the index. • The block can then be modified -- without affecting the original. • The new blocks are sometimes called shadow blocks.

10. Index Private Workspace New Index Original Index 0’ 1 2 3’ 0 1 2 0 1 2 0 1 2 3 1 2 0 1 2 0 0’ 3’ 1 2 0 Private Workspace (continue) • For more complex transaction, the private workspace might contain a large number of files. • If transaction aborts, the private workspace is simply deleted and all the private blocks that it points to ate put back on the free list. • If transaction commits, the private indices are moved into the parent’s workspace atomically. The blocks that are no longer reachable are put onto the free list.

11. x = 0; y = 0; BEGIN_TRANSACTION x = x + 1; y = y + 2; x = y * y; END_TRANSACTION Log: x = 0 / 1 Log: x = 0 / 1 y = 0 / 2 Log: x = 0 / 1 y = 0 / 2 x = 1 / 4 Writeahead Log • Writeahead Log = intentions list. • Before each updates, a record is written to the stable storage. • The record specifies: • which transaction is making the change; • which file and block is being changed; • what the old and new values are. • If the transaction succeeds and is committed, a commit record is written to the log, but the data structures do not have to be changed, as they have already been updated. • If the transaction aborts, the log can be used to back up to the original state. • Starting at the end and going backward, each log record is read and the change described in it undone. The action is called a rollback. • The log can also be used for recovering from crashes.

12. Two-Phase Commit Protocol • The action of committing a transaction must be done atomically --- instantaneously and indivisibly. • In a distributed system, the commit may require the cooperation of multiple processes on different machines, each of which holds some of the variables, files, and databases, and other objects changed by the transaction. • The basic idea: • One process functions as the coordinator -- usually it is the one executing the transaction. • The commit protocol begins when the coordinator writes a log entry saying that it is starting the commit protocol, followed by sending each of the other processes involved (the subordinates) a message telling them to prepare to commit. • When a subordinate gets the message it checks to see if it is ready to commit, make a log entry, and send back its decision. • When the coordinator has received all the responses, it knows whether to commit or abort. • If all the processes are prepared to commit, the transaction is committed. • If one or more are unable to commit (or do not respond), the transaction is aborted. • Either way, the coordinator writes a log entry and then sends a message to each subordinate informing it of the decision. • It is this write to the log that actually commits the transaction and makes it go forward no matter what happens afterward.

13. Coordinator Subordinate Write “Prepare” in the log Send “Prepare” message Write “Ready” in the log Send “Ready” message Collect all replies Write log record Send “Commit” message Write “Commit” in the log Commit Send “Finished” message Phase I Phase II Two-Phase Commit Protocol (continue) • Due to the use of the log on stable storage, this protocol is highly resilient in the face of (multiple) crashes. • If coordinator crashes after having written the initial log record, upon recovery it can just continue where it left off, repeating the initial message if need be. • If it crashes after having written the result of the vote to the log, upon recovery it can just re-inform all the subordinates of the result. • If a subordinate crashes before having replied to the first message, the coordinator will keep sending it messages, until it gives up. • If a subordinate crashes later, it can see from the log where it was, and thus do what it must do.