Operating Systems Design Principles

1. COT 5611 Operating Systems Design PrinciplesSpring 2012 Dan C. Marinescu Office: HEC 304 Office hours: M-Wd 3:30 – 5:30 PM

2. Lecture 23 • Reading assignment: • Chapter 9 from the on-line text • Last time – • ADVANCE • SEQUENCE • TICKET • Events • Coordination with events • Virtual memory and multi-level memory management Lecture 23

3. Today • Atomic actions • All-or nothing and Before-or-after atomicity • Applications of atomicity Lecture 23

4. Lecture 23

5. Atomicity • Atomicity  ability to carry out an action involving multiple steps as an indivisible action; hide the structure of the action from an external observer. • All-or-nothing atomicity (AONA) • To an external observer (e.g., the invoker) an atomic action appears as if it either completes or it has never taken place. • Before-or-after atomicity (BOAA) • Allows several actions operating on the same resources (e.g., shared data) to act without interfering with one another • To an external observer (e.g., the invoker) the atomic actions appear as if they completed either before or after each other. • Atomicity • simplifies the description of the possible states of the system as it hides the structure of a possible complex atomic action • allows us to treat systematically and using the same strategy two critical problems in system design and implementation (a) recovery from failures and (b) coordination of concurrent activities Lecture 23

6. Atomicity in computer systems • Hardware: interrupt and exception handling (AONA) + register renaming (BOAA) • OS: SVCs (AONA) + non-sharable device (e.g., printer) queues (BOAA) • Applications: layered design (AONA) + process coordination (BOAA) • Database: updating records (AONA) + sharing records (BOAA) • Example: exception handling when one of the following events occur • Hardware faults • External events • Program exception • Fair-share scheduling • Preemptive scheduling when priorities are involved • Process termination to avoid deadlock • User-initiated process termination • Register renaming avoid unnecessary serialization of program operations imposed by the reuse of registers by those operations. High performance CPUs have more physical registers than may be named directly in the instruction set, so they rename registers in hardware to achieve additional parallelism. r1  m(1000) r1  r1+5 m(1000)  r1 r1  m(2000) r2m(2000) r1  r1+8 r2  r2 + 8 m(2000)  r1 m(2000)  r2 Lecture 23

7. Atomicity in databases and application software • Recovery from system failures and coordination of multiple activities is not possible if actions are not atomic. • Database example: a procedure to transfer from a debit account (A) to a credit account (B) Procedure TRANSFER (debit_account, credit_account, amount) GET (temp, A) temp  temp – amount PUT (temp, A) GET (temp, B) temp  temp + amount PUT (temp, B) What if: (a) the system fails after the first PUT; (b) multiple transactions on the same account take place. • Layered application software example: a calendar program with three layers of interpreters: • Calendar program • JVM • Physical layer Lecture 23

8. All-or-nothing atomicity • The AONA is required to - (1) handle interrupts (e.g., a page fault in the middle of a pipelined instruction). Need to retrofit the AONA at the machine language interface if every machine instruction is an AONA then the OS could save as the next instruction the one where the page fault occurs. Additional complications with a user-supplied exception handler. - (2) handle supervisor calls (SVCs); an SVC requires a kernel action to change the PC, the mode bit (from user to kernel) and the code to carry out the required function. The SVC should appear as an extension of the hardware. • Design solutions a typewriter driver activated by a user issued SVC, READ. • Implement the “nothing” option  blocking read; when no input is present reissue the READ as the next instruction. This solution allows a user to supply its own exception handler. • Implement the “all” option  non-blocking read; return control to the user program if no input available with a zero length input. Lecture 23

9. Before-or-after atomicity • Two approaches to concurrent action coordination: • Sequence coordination e.g., “action A should occur before B”  strict ordering • BOAA, the effect of A and B is the same whether A occurs before B or B before A  non-strict ordering. • BOAA is more general than sequence coordination. • Example: two transactions operating on account A each performs GET and PUT • Six possible sequences of actions: (G1,P1, G2, P2), (G2,P2,G1,P1), (G1,G2, P1, P2), (G1,G2, P2, P1), (G2,G1, P1,P2), (G2, G1, P2, P1). • Only the first two lead to correct results. • Solution the sequence Ri  Pi should be atomic. • Correctness condition for coordination  if every possible result is guaranteed to be the same as if the actions were applied in one after another in some order. • Before-or-after atomicity guarantees the correctness of coordination  indeed it serializes the actions. • Stronger correctness requirements are sometimes necessary: • External time consistency  e.g., in banking the transaction should be processed in the order they are issued. • Sequential consistency  e.g., instruction reordering should not affect the result Lecture 23

10. Common strategy and side-effects of atomicity • The common strategy for BOAA and AONA  hide the internal structure of a complex action; prevent an external observer to discover the structure and the implementation of the atomic action. • Atomic actions could have “good” (benevolent) side-effects: • An audit log records the cause of a failure and the recovery steps for later analysis • Performance optimization: when adding a record to a file the data management may restructure/reorganize the file to improve the access time Lecture 23

11. Bootstrapping for the development atomic actions • Bootstrapping  informally means to build complex actions based on simple ones • Bootstrapping steps • Find a systematic way to reduce a general problem to a particular one • Solve the problem for the particular problem • See how this solution can be generalized Lecture 23

12. All-or nothing disk storage • The commit point in CAREFUL_PUT will be discussed later. • The three sectors, S1, S2, and S3 form a virtual sector. • Writing and reading are sequential, one waits for the previous to complete. • We consider only system crashes, the disk does not fail. • If the system crashes during a PUT only one of the sectors is affected • On a GET we compare the contents of the sectors. Lecture 23

13. Is there anything wrong with this implementation? We assumed that the three sectors have the same contents at the beginning of an operation. A previous failure would invalidate this assumption. Example: - the running thread is interrupted while writing S3; data3 is garbage - the next GET will find data1=data2  data1 will be used - a new thread calls call PUT but it is interrupted while writing S2 - the next call to GET will find that data1 ≠ data 2 so it will use data3 The fix: guarantee that the three sectors are identical before updating. Lecture 23

14. Lecture 23

15. The implementation of the all-or-nothing disk Assumes that only one thread at a time attempts to use the PUT and GET Does not implement before-or-after atomicity. The CHECK_AND_REPAIR is idempotent  does not have any side effects, can be interrupted and restarted. The commit point occurs after writing to S2 is finished; after that point the new data is available to a GET (before writing to S2 starts) the old data is available to a new GET. Writing to the three sectors is not done to improve durability; it is important that the writing is done sequentially Lecture 23

16. Generalization  the golden rule of atomicity • An all-or-nothing action should consist of • A pre-commit phase  it should be possible to back up from it without leaving any trace • Post-commit phase  this phase should be able to run to completion • The semantics for programming all-or nothing actions Lecture 23

17. Do and don’t for pre-commit and post-commit phases Carry out all steps necessary to prepare the post-commit phase (which should run to completion), e.g., check permissions, bring in all pages that may be needed, mount removable media, allocate stack space, do not expose any results or carry out actions that are irreversible. Shared resources allocated during the pre-commit cannot be released until after the commit point. The commit step should be the last step of an all-or nothing action. Lecture 23

18. Shadow copies  a technique to implement AONA Used by programs that modify existing files e.g., text editors, calendar management programs, compilers, etc. Pre-commit  create a duplicate working copy of the file and make changes to the working copy Commit point exchange the working copy with the original; use atomic actions supported by the OS e.g., RENAME Post-commit –clean up, release the space of the working copy Note that the new copy is not available before the RENAME Lecture 23

19. Version histories • Maintain the history of a variable, do not delete older versions. • Accept tentative values but ignore them until they are committed. • A journal storage manager • Allows the creation of a more complex storage model that the cell storage (an item fits into one cell). • Provides atomic actions to the end-user. Lecture 23

20. Lecture 23

21. New procedures for a journal storage Lecture 23

22. The state transitions of a journal storage system Lecture 23

23. Read and write procedures: caller_id is the action identifier returned by NEW_ACTION Lecture 23

24. The attributes of the read and write If the current all-or nothing action fails before the COMMIT then the new version is not visible to the next read. They support writing of an entire record with multiple fields. If say a record has 13 fields and if action 1234 fails then the entire record is not available. A read will access the last committed action say action 1232. Lecture 23

25. Example: a thread has created a new record with: data_id=A, new_value=75, and client_id=1794. The procedure READ_CURRENT_VALUE will return value 24 for A and ignore versions aborted or pending. Lecture 23

26. An all-or-nothing transfer using journal storage Note: the transaction is not before-or-after It checks if there are enough funds in the credit_account. The order of steps is unconstrained Problems: updates to version history and changes to the outcome must be all-or-nothing. But these can be done by overriding a single cell. Lecture 23

27. Atomicity logs and journal storage • Log  An interleaved version of all variables; the information about the update of each data forms a record appended at the end of the log. • Easy access to a log, only the pointer to the last record is needed • Combine all-or-nothing atomicity of journal storage with the speed of cell storage. Two steps • Logcarry out the change in the journal storage • Install change the cell storage by overriding the previous version of each record • The log is the authoritative record of the outcome of an action; the cell storage can be reconstructed using the log. • The log should reside in non-volatile memory. Lecture 23

28. Types of logs Atomicity log. Allows a crash recovery procedure to undo all-or-nothing actions that didn’t complete, or finish all-or-nothing actions that committed but that didn’t record all of their effects. Archive log. Many uses for archive information: watching for failure patterns, reviewing the actions of the system preceding and during a security breach, recovery from application-layer mistakes fraud control, and compliance with record-keeping requirements. Performance log. Most mechanical storage media have much higher performance for sequential access than for random access. Since logs are written sequentially, they are ideally suited to such storage media. When combined with a cache that eliminates most disk reads, a performance log can provide a significant speed-up. 4. Durability log. If the log is stored on a non-volatile medium (e.g., magnetic tape) that fails in ways and at times that are independent from the failures of the cell storage medium (e.g., magnetic disk) then the copies of data in the log are replicas that can be used as backup in case of damage to the copies of the data in cell storage. Any log that uses a non-volatile medium also helps support durability. Lecture 23

29. Logging configurations Lecture 23

30. Logging protocols • Reason for write-ahead-log  log append while install overwrites • Log record • Id of the all-or-nothing action performing the update • The do or redo action  component action that can perform the install if the system crashes before the install. • The undo action  component action that can reverse the effects if the system crashes during the install. • Four types of log records • BEGIN  NEW_ACTION writes this record and records the action id • CHANGE  written by the pre-commit phase • OUTCOME  written by the COMMIT or by the ABORT procedures • END  the final step of an action Lecture 23

31. Example of log records Lecture 23

32. Example: all-or-nothing TRANSFER with logging Lecture 23

33. Recovery procedures • We need recovery procedures in case of a system crash • Assume an in-core database  • The log is not affected as it resides on non-volatile memory • Abandon the in-core database and all all-or-nothing actions in progress • Two steps: • Backwards scan the log and identify all actions with an OUTCOME record showing that the action has been COMMITTED, call them winner actions • Forward scan the log and identify REDO actions of every winner whose OUTCOME record shoe COMMITTED Lecture 23

34. Lecture 23

35. Lecture 23