CS 6560 Operating System Design

1. CS 6560 Operating System Design Lecture 7: Kernel Synchronization Kernel Time Management

2. References • Our textbook: Robert Love, Linux Kernel Development, 2nd edition, Novell Press, 2005. • Understanding the LINUX Kernel, 3rd. edition, O’Reilly, 2005. (covers 2.6) • The kernel code and its own documentation.

3. Plan • Chap 9: Concurrency methods in Linux 2.6 • Chap 10: Timers and Time Management

4. Recall: Linux 2.6 Sync Methods • Per CPU copies of variables • Preemption disabling • Interrupt disabling • atomic operations • spinlocks • Semaphores • Completion variables • Seq locks • Barriers

5. Reader-Writer Spin Locks • Reader-writer spin locks allow several readers, but only one instance of execution can write. Readers can only read, writers can do both. • Use: • Define a RW_LOCK which is referenced by lock and unlock operations • bracket critical section of reader with read_lock/read_unlock • Bracket critical section of writer with write_lock/write_unlock

6. Reader-Writer Spin Locks - cont’d • Methods • A variety of variations are available for disabling interrupts and saving interrupt state • Also methods for initialization and testing • Downside • Favor readers over writers

7. Kernel Semaphores • Linux kernel semaphores are locks that cause sleeping - uses waitques • Must be in process context - current is valid • Good for holding a longer time, but not a very short time. • Do not mix semaphores and spin locks.

8. Kernel Semaphores • Semaphore Use • Initialize • Bracket critical sections with down/up operations • Methods • Initialization • Variations of down with interruptible and non-interruptible sleep, and just testing • One form of up • Has readers and writers version

9. Completion Variables • One task waits for another • To wait, call wait_for_completion • To notitfy completion, call complete() • Defined in kernel/sched.c • Used in fork, exit, workqueues and quite a few other places.

10. Seq Locks • How used: • Readers loop while reading (read_seqbegin) until assured that read was uninterrupted (read_seqretry). • Writers simply bracket critical section (write_seqlock/write_sequnlock) • Advantage: favors writers over readers. • Example: see kernel/timer.c, line 1105. • Implemented in kernel/sched.c (uses spin locks and wait queues)

11. Barriers • Problem: Compilers and SMB cause operations to be executed out of order. • Solution: Barriers provides places (barriers) in the code for synchronization among readers and writers.

12. Barrier Operations • rmb() is a read barrier. All loads from memory before this line must be executed before proceeding and all loads from memory after this line must be executed after this line is reached. • wmb() is a write barrier. All stores to memory before this line must be executed before proceeding and all stores to memory after this line must be executed after this line is reached. • mb() is a read and write barrier. All stores and loads before this line must be executed before proceeding and all stores and loads after this line must be executed after this line is reached.

13. Example of Barriers • Assume that A and B are memory locations and x and y are registers. Assume initially A has a value of 1 and B has a value of 2. • Thread 1: A = 3; // store to A mb(); // barrier B = 4; // store to B • Thread 2: x = B; // load from B rmb(); // barrier y = A; // load from A

14. Reference on Barriers • See Documents/memory-barriers.txt by David Howells.

15. Notes on Barriers • A processor may reorder instructions or collapse instructions to optimize code. • A compiler may reorder instructions because of their caches. • These should not result in wrong values on a single processor, but may lead to wrong results when more than one processor or a device controller is involved. • There is a difference between how a CPU and how the external system views operation order. • Memory barriers can partially order loads and stores in a single processor. • Memory barriers can be paired access among several processors to partially order execution. • IO also needs ordering.

16. More on Interrupt Context • Several macros to determine if in hard or soft interrupt context. See include/kernel/hardirq.h • Notes • They depend upon having preemption off. • There is still a current variable, but it is not for the process that caused the interrupt.

17. Chapter 10: Timers and Time Management

18. Time related Tasks • System time • Wall clock time • Interval timers

19. System Time • Stored as jiffies. • Originally an unsigned 32-bit integer updated 100 times a second • Now an unsigned 64-bit integer updated 1000 times a second. • Problem with 32-bits: wrap around: 2^32= 4,294,967,296. At 100 per second, = 497 days)

20. HZ • Frequency is called HZ • Granularity: • varies according to architecture (ranges from 24 to 1024) • Higher = faster = more responsive, but more overhead

21. Implementation of Jiffies • Jiffies in a 32-bit overlayed with a 64-bit number. • Initialized as -300 seconds to cause any wrap around in 5 minutes. • Comparisons are carefully crafted: time_after, time_before, time_after_eq, time_before_eq.

22. Hardware clocks • PIT = Programmable Interrupt Timer (arch/i386/i8253.c) • Other choices are now available, for example APIC (see arch/i386/apic.c), but normally used for other purposes.

23. Timer Interrupt Handler • Arch dependent, but some effort to make it generic. • Eventually calls do_timer. • Has bottom half implemented by a softirq.

24. Time of Day • Stored in xtime as a struct: 12 struct timespec { 13time_t tv_sec; /* seconds */ 14 long tv_nsec; /* nanoseconds */ 15 };

25. Timers • Local timers - see book and code • Timer structures: • Specify: expires (in jiffies), data, and function to run • Timer methods: init_timer, add_timer, mod_timer, del_timer, del_timer_sync

26. More • Bogomips • Small delays • Schedule_timeout()