Exploring Advanced Operating System Concepts

1. Advanced Operating Systems - Spring 2009Lecture 19 – Monday March 30, 2009 • Dan C. Marinescu • Email: dcm@cs.ucf.edu • Office: HEC 439 B. • Office hours: M, Wd 3 – 4:30 PM. • TA: Chen Yu • Email: yuchen@cs.ucf.edu • Office: HEC 354. • Office hours: M, Wd 1.00 – 3:00 PM.

2. Last, Current, Next Lecture • Last time: • Caching • Introduction to I/O Subsystem • Today • I/O subsystem • File System Implementation • Next time: • File System Interface and Mass Storage Structure

3. Communication among asynchronous sub-systems: polling versus interrupts • Polling periodically checking the status of an I/O device • Interrupt  deliver data or status information when status information immediately . • Intel Pentium Vector Table

4. Interrupts: used for I/O and for exceptions • CPU Interrupt-request line  triggered by I/O device • Interrupt handler receives interrupts • To mask an interrupt  ignore or delay some interrupts • Interrupt vector to dispatch interrupt to correct handler • Based on priority • Some non-maskable

5. DMA Bypasses CPU to transfer data directly between I/O device and memory; it allows subsystems within the computer to access system memory for reading and/or writing independently of CPU: disk controller, graphics cards, network cards, sound cards, GPUs (graphics processors),   also used for intra-chip data transfer in multi-core processors,.  Avoids programmed I/O for large data movement Requires DMA controller Direct Memory Access (DMA)

6. DMA Transfer

7. Device drivers and I/O system calls • Multitude of I/O devices • Character-stream or block • Sequential or random-access • Sharable or dedicated • Speed of operation • Read-write, read only, or write only • Device-driver layer hides differences among I/O controllers from kernel: • I/O system calls encapsulate device behaviors in generic classes

8. Block and Character Devices • Block devices (e.g., disk drives, tapes) • Commands e.g., read, write, seek • Raw I/O or file-system access • Memory-mapped file access possible • Character devices (e.g., keyboards, mice, serial ports) • Commands e.g., get, put • Libraries allow line editing

9. Network Devices and Timers • Network devices • Own interface different from bloc or character devices • Unix and Windows NT/9x/2000 include socket interface • Separates network protocol from network operation • Includes select functionality • Approaches vary widely (pipes, FIFOs, streams, queues, mailboxes) • Timers • Provide current time, elapsed time, timer • Programmable interval timer for timings, periodic interrupts • ioctl (on UNIX) covers odd aspects of I/O such as clocks and timers

10. Blocking and Nonblocking I/O • Blocking  process suspended until I/O completed • Easy to use and understand • Insufficient for some needs • NonblockingI/O call returns control to the process immediately • User interface, data copy (buffered I/O) • Implemented via multi-threading • Returns quickly with count of bytes read or written • Asynchronous process runs while I/O executes • I/O subsystem signals process when I/O completed

11. Synchronous/Asynchronous I/O Synchronous Asynchronous

12. Kernel I/O Subsystem • Scheduling • Some I/O request ordering using per-device queue • Some OSs try fairness • Buffering – store data in memory while transferring to I/O device. • To cope with device speed mismatch or transfer size mismatch • To maintain “copy semantics”

13. Sun Enterprise 6000 Device-Transfer Rates

14. Kernel I/O Subsystem and Error Handling • Caching  fast memory holding copy of data • Always just a copy • Key to performance • Spooling  holds output for a device that can serve only one request at a time (e.g., printer). • Device reservation  provides exclusive access to a device • System calls for allocation and de-allocation • Possibility of deadlock • Error handling: • OS can recover from disk read, device unavailable, transient write failures • When I/O request fails error code. • System error logs hold problem reports

15. I/O Protection • I/O instructions are priviledged • Users make system calls

16. Kernel Data Structures • Kernel keeps state info for I/O components, including open file tables, network connections, device control blocs • Complex data structures to track buffers, memory allocation, “dirty” blocks • Some use object-oriented methods and message passing to implement I/O

17. UNIX I/O Kernel Structure

18. Hardware Operations • Operation for reading a file: • Determine device holding file • Translate name to device representation • Physically read data from disk into buffer • Make data available to the process • Return control to process

19. STREAMS in Unix • STREAM  a full-duplex communication channel between a user-level process and a device in Unix System V and beyond • A STREAM consists of: - STREAM head interfaces with the user process - driver end interfaces with the device- zero or more STREAM modules between them. • Each module contains a read queue and a write queue • Message passing is used to communicate between queues

20. STREAMS

21. I/O major factor in system performance: • Execute • device driver, • kernel I/O code • Context switches • Data copying • Network traffic stressful

22. Improving Performance • Reduce number of context switches • Reduce data copying • Reduce interrupts by using large transfers, smart controllers, polling • Use DMA • Balance CPU, memory, bus, and I/O performance for highest throughput

23. File System • Basic Concepts • Allocation Methods • Free-Space Management • Efficiency and Performance • Fault tolerance and recovery • NFS

24. Basic concepts • File Collection of related information residing on secondary storage. • File control block  data structure summarizing information about a file

25. Open and Read operations

26. Virtual File Systems (VFS) • Support an object-oriented implementation of a file systems. • The VFS API used for different types of file systems.

27. Directory Implementation • Directory  provides information about files on a system or on a physical device. • Linear list of file names with pointer to the data blocks. • simple to program • time-consuming to execute • Hash Table – linear list with hash data structure. • decreases directory search time • collisions – situations where two file names hash to the same location • fixed size

28. Physical layout of a file • An allocation method refers to how disk blocks are allocated for files: • Contiguous allocation  Each file occupies a set of contiguous blocks on the disk • Simple – only starting location (block #) and length (number of blocks) required • Random access • Wasteful of space (dynamic storage-allocation problem) • Files cannot grow • Linked allocation  Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. • Simple – need only starting address • Free-space management system – no waste of space • No random access • Mapping • Indexed allocation  Brings all pointers together into the index block. pointer block =

29. Contiguous Allocation • Mapping from logical to physical Q LA/512 R • Block to be accessed = ! + starting address • Displacement into block = R

30. Extent-Based Systems • Extent a contiguous block of disks. • Extent-based file systems  allocate disk blocks in extents. A file consists of one or more extents. • Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme

31. Linked Allocation • Maping Q LA/511 R Block to be accessed is the Qth block in the linked chain of blocks representing the file. Displacement into block = R + 1 File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2.

32. Indexed Allocation- bring all pointers together index table

33. Indexed Allocation • Need index table • Random access • Dynamic access without external fragmentation, but have overhead of index block. • Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words. We need only 1 block for index table. Q LA/512 R Q = displacement into index table R = displacement into block

34. Indexed Allocation – Mapping • Mapping from logical to physical in a file of unbounded length (block size of 512 words). • Linked scheme – Link blocks of index table (no limit on size). Q1 LA / (512 x 511) R1 Q1= block of index table R1is used as follows: Q2 R1 / 512 R2 Q2 = displacement into block of index table R2 displacement into block of file:

35. Indexed Allocation – Mapping (Cont.) • Two-level index (maximum file size is 5123) Q1 LA / (512 x 512) R1 Q1 = displacement into outer-index R1 is used as follows: Q2 R1 / 512 R2 Q2 = displacement into block of index table R2 displacement into block of file:

36. Indexed Allocation – Mapping (Cont.)  outer-index file index table

37. Combined Allocation Scheme: UNIX (4K bytes per block)

38. Free-Space Management 0 1 2 n-1 • Bit vector (n blocks) … 0  block[i] free 1  block[i] occupied bit[i] =  Block number calculation (number of bits per word) * (number of 0-value words) + offset of first 1 bit

39. Free-Space Management (Cont.) • Bit map requires extra space • Example: block size = 212 bytes disk size = 230 bytes (1 gigabyte) n = 230/212 = 218 bits (or 32K bytes) • Easy to get contiguous files • Linked list (free list) • Cannot get contiguous space easily • No waste of space • Grouping • Counting

40. Free-Space Management (Cont.) • Need to protect: • Pointer to free list • Bit map • Must be kept on disk • Copy in memory and disk may differ • Cannot allow for block[i] to have a situation where bit[i] = 1 in memory and bit[i] = 0 on disk • Solution: • Set bit[i] = 1 in disk • Allocate block[i] • Set bit[i] = 1 in memory

41. Efficiency and Performance • Efficiency dependent on: • disk allocation and directory algorithms • types of data kept in file’s directory entry • Performance • disk cache – separate section of main memory for frequently used blocks • free-behind and read-ahead – techniques to optimize sequential access • improve PC performance by dedicating section of memory as virtual disk, or RAM disk

42. Caching for regular I/O operations and memory mapped I/O A page cache caches pages rather than disk blocks using virtual memory techniques: used by memory-mapped I/O Routine I/O through the file system uses the buffer (disk) cache • A unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O I/O with a unified buffer cache I/O without a unified buffer cache

43. Fault -tolerance • Recovery • Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies • Backup/Restore Use system programs to back up data from disk to another storage device • Log Structured File Systems • All transactions are written to a log • A transaction is considered committed once it is written to the log • However, the file system may not yet be updated • The transactions in the log are asynchronously written to the file system • When the file system is modified, the transaction is removed from the log • If the file system crashes, all remaining transactions in the log must still be performed