Ensuring System Integrity Through Advanced OS Synchronization

1. CSC 660: Advanced OS Synchronization CSC 660: Advanced Operating Systems

2. Topics • Race Conditions • Critical Sections • Kernel Concurrency • What needs Protection? • Atomic Operations • Spin Locks • Semaphores • Deadlock CSC 660: Advanced Operating Systems

3. Race Conditions out = 4 abc.pdf 4 prog.c 5 Process A as2.txt 6 in = 7 7 Process B CSC 660: Advanced Operating Systems

4. Race Conditions • Process A read in, stores in local var slot. • Timer interrupt. • Scheduler switches to process B. • Process B reads in, stores in local var slot. • Process B stores filename in slot 7 (slot). • Process B updates in to be 8. • Scheduler eventually switches to process A. • Process A writes filename in slot 7 (slot). • Process A computes in = slot + 1. • Process A updates in to be 8. CSC 660: Advanced Operating Systems

5. Critical Sections • How can we prevent race conditions? • Prohibit multiple processes from accessing shared resource at the same time. • Critical section is code that accesses shared resource. CSC 660: Advanced Operating Systems

6. Critical Sections • No two processes may be simultaneously inside their critical sections. • No assumptions may be made about speed or number of CPUs. • No process running outside its critical section may block other processes. • No process should have to wait forever to enter its critical section. CSC 660: Advanced Operating Systems

7. Critical Sections CSC 660: Advanced Operating Systems

8. Kernel Concurrency • Interrupts • Softirqs and tasklets • Kernel preemption • Sleeping • SMP CSC 660: Advanced Operating Systems

9. What needs Protection? • What needs protection? • Global kernel data structures. • Shared data between process/interrupt context. • Shared data between interrupt handlers. • What doesn’t? • Data local to executing thread (i.e., stack.) • Local variables. • Dynamically allocated data w/ only local ptrs. • Per-CPU data doesn’t need SMP protection. CSC 660: Advanced Operating Systems

10. Process A read i(7) incr i(7 -> 8) - write i(8) Process B read i(7) - incr i(7 -> 8) - write i(8) Why do we need atomicity? Problem: Two processes incrementing i. A: read i(7) A: incr i(7 -> 8) B: read i(8) B: incr i(8 -> 9) Uniprocessor Version CSC 660: Advanced Operating Systems

11. Process A atomic_inc i (7->8) Process B - atomic_inc i (8->9) Atomic Operations Atomic operations are indivisible. Provided by atomic_t in the kernel. Operations: atomic_{read,set,add,sub,inc,dec,dec_and_test) x86 assembly: lock byte preceding opcode makes atomic. CSC 660: Advanced Operating Systems

12. Process A atomic_inc i (7->8) Process B - atomic_inc i (8->9) Atomicity doesn’t provide Ordering One atomic order of operations: Another atomic order of operations: Process A - atomic_inc i (8->9) Process B atomic_inc i (7->8) CSC 660: Advanced Operating Systems

13. Barriers provide Ordering • Optimization Barriers • Prevent compiler from re-ordering instructions. • Compiler doesn’t know when interrupts or other processors may read/write your data. • Kernel provides barrier() macro. • Memory Barriers • Read/write barriers prevent loads/stores from being re-ordered across barrier. • Kernel provides rmb(), wmb() macros. CSC 660: Advanced Operating Systems

14. Spin Locks • If lock “open” • Sets lock bit with atomic test-and-set. • Continues into critical region. • else lock “closed” • Code “spins” in busy wait loop until available. • Kernel-preemption or SMP will interrupt. CSC 660: Advanced Operating Systems

15. Spin Lock Functions spin_lock_init(spinlock_t *lock) Initialize spin lock to 1 (unlocked). spin_lock(spinlock_t *lock) Spin until lock becomes 1, then set to 0 (locked). spin_lock_irqsave(spinlock_t *l, u flags) Like spin_lock() but disables and saves interrupts. Always use an IRQ disabling variant in interrupt context. spin_unlock(spinlock_t *lock) Set spin lock to 1 (unlocked). spin_lock_irqrestore(spinlock_t *l, u flags) Like spin_lock(), but restores interrupt status. spin_trylock(spinlock_t *lock) Set lock to 0 if unlocked and return 1; return 0 if locked. CSC 660: Advanced Operating Systems

16. Spin Lock • Disables kernel pre-emption. • Atomic test-and-sets lock. • If old value positive Lock acquired. • Else Enables pre-emption. If break_lock is 0, sets to 1 to indicate a task is waiting. Busy wait loop while (spin_is_locked(lock)) cpu_relax(); # pause instruction on P4 Goto 1. CSC 660: Advanced Operating Systems

17. Semaphores Integer value with atomic access. If S>0, semaphore prevents access. Using a semaphore for mutual exclusion: down(S); /* critical section */ up(S); CSC 660: Advanced Operating Systems

18. Semaphores Down (P): Request to enter critical region. If S > 0, decrements S, enters region. Else process sleeps until semaphore is released. Up (V): Request to exit critical region. Increments S. If S > 0, wakes sleeping processes. CSC 660: Advanced Operating Systems

19. Linux Semaphores DECLARE_MUTEX(sem); Static declares a mutex semaphore. void init_MUTEX(struct semaphore *sem); Dynamic declaration of a mutex semaphore. void down(struct semaphore *sem); Decrements semaphore and sleeps. int down_interruptible(struct semaphore *sem); Same as down() but returns on user interrupt. int down_trylock(struct semaphore *sem); Same as down() but returns immediately if not available. void up(struct semaphore *sem); Releases semaphore. CSC 660: Advanced Operating Systems

20. Linux Semaphores #include <asm/semaphore.h> struct semaphore sem; init_MUTEX(&sem); if (down_interruptible(&sem)) return –ERESTARTSYS; /* user interrupt */ /* * critical section */ up(&sem); CSC 660: Advanced Operating Systems

21. Tradeoffs Spin Locks Busy waits waste CPU cycles. Semaphores Context switch on sleep is expensive. CSC 660: Advanced Operating Systems

22. Process A down(A) down(B) Process B down(B) down(A) Deadlocks Single process deadlock: A: spin_lock(A) A: spin_lock(A) Two process deadlock: CSC 660: Advanced Operating Systems

23. Deadlock Prevention Lock Ordering If you acquire multiple locks, you must always acquire them in the same order. Don’t double acquire a lock. Always use interrupt-disabling variants of spin_lock() in interrupt context. Always release locks. Be sure that your code will always make appropriate calls to release locks. CSC 660: Advanced Operating Systems

24. Key Points • Synchronization is essential for accessing shared kernel data structures. • Kernel provides two major types: • Spin Locks • Semaphores • and many variants. • System performance depends strongly on the type of synchronization used. • Synchronization must be used carefully and consistently to avoid deadlocks. CSC 660: Advanced Operating Systems

25. References • Daniel P. Bovet and Marco Cesati, Understanding the Linux Kernel, 3rd edition, O’Reilly, 2005. • Johnathan Corbet et. al., Linux Device Drivers, 3rd edition, O’Reilly, 2005. • Robert Love, Linux Kernel Development, 2nd edition, Prentice-Hall, 2005. • Claudia Rodriguez et al, The Linux Kernel Primer, Prentice-Hall, 2005. • Peter Salzman et. al., Linux Kernel Module Programming Guide, version 2.6.1, 2005. • Andrew S. Tanenbaum, Modern Operating Systems, 3rd edition, Prentice-Hall, 2005. CSC 660: Advanced Operating Systems