I/O and Filesystems

1. I/O and Filesystems

2. How to provide interfaces Rough reading guide (no exam guarantee): Tanenbaum Ch. 5.1 – 5.5 & 6.1-3 Silberschatz Ch. 13 & 10.1-4, 11.1-8 Also: Patterson Ch. 8 (I/O)

3. I/O • A computer without peripheral devices is pretty much useless • Old mainframes required kernel-recompile to install new devices • Now: PCs have millions of possible devices

4. Goals for I/O Handling • Enable use of peripheral devices • Present a uniform interface for • Users (files etc.) • Devices drivers • Hide the details of devices from users (and OS)

5. Device Controllers • I/O devices have components: • mechanical component • electronic component • The electronic component is the device controller • may be able to handle multiple devices • OS deals with controllers not devices • Controller's tasks • convert bit stream to block of bytes • perform error correction as necessary • make blocks available to main memory

6. Relevant Device Properties (1) • The mechanical/electronic properties of devices impact software • timing (seek times etc.) • interrupts • data rates

7. Relevant Device Properties (2) Some typical device, network, and bus data rates

8. Accessing Devices (1) • Most devices provide • buffers (in / out) • control registers • status registers • These are accessed from the OS/Apps • I/O ports • memory-mapped • hybrid

9. Accessing Devices (2) • Separate I/O and memory space • Memory-mapped I/O • Hybrid

10. Memory-Mapped I/O • No special instruction in assembler needed • No special protection needed (remember VM) • Testing control registers directly Not load and test! • Caching of control registers? • One bus?

11. Memory-Mapped I/O (a) A single-bus architecture (b) A dual-bus memory architecture

12. Memory-Mapped I/O

13. Shuffling Data • Data needs to be sent • one byte at a time • several bytes at a time • Using the CPU to shuffle small amounts of data may be inefficient  Direct Memory Access (DMA)

14. Direct Memory Access (DMA) Operation of a DMA transfer

15. Interrupts Revisited Connections between devices and interrupt controller actually use interrupt lines on the bus rather than dedicated wires

16. Goals of I/O Software (1) • Device independence • programs can access any I/O device • without specifying device in advance • (file on floppy, hard drive, or CD-ROM) • Uniform naming • name a file or device as a string or an integer • not depending on which machine • Error handling • handle as close to the HW as possible

17. Goals of I/O Software (2) • Synchronous vs. asynchronous transfers • blocking transfers vs. interrupt-driven • OS may make interrupt-driven operations look like blocking to the user • Buffering • data coming off a device cannot be stored in final destination • OS should buffer for pre-processing/RT.. • Sharable vs. dedicated devices • disks are sharable • tape drives would not be • OS should be able to support both

18. I/O There are three kinds of I/O handling • Programmed I/O • “Do it all yourself” • Interrupt-driven I/O • “Here you are, now tell me when its done” • DMA-based I/O • “Let someone else do it”

19. Programmed I/O

20. Writing a string to the printer using interrupt-driven I/O Interrupt-Driven I/O Code for system call Code for interrupt handler

21. I/O Using DMA • Printing a string using DMA a) code executed when the print system call is made b) interrupt service procedure

22. I/O Software Layers Layers of the I/O Software System

23. Interrupt Handlers (1) • Interrupt handlers are best hidden • have driver starting an I/O operation block until interrupt notifies of completion • Interrupt procedure does its task • then unblocks driver that started it

24. Interrupt Service Routine Steps that must be performed in software when interrupt occurs (no complete list) Step 1: Preparation • Save regs not already saved by interrupt hardware • Set up context for interrupt service procedure • Ack interrupt controller, reenable interrupts () Step 2: Actual task execution • Copy registers from where saved • Run service procedure • Schedule and run a new process (+ new context)

25. Interrupt Handlers (3) • Interrupt handlers should be fast (why?) • Sometimes there is a lot of things to do • Copying buffers, waking up processes, starting new I/O ops, etc. • Solution: split in two parts • Top half: disabled interrupts, only essential work • Bottom half: enabled interrupts, does most of the work

26. Device Drivers (1) • Logical position of device drivers is shown here • Communication between drivers and device controllers over the bus

27. Device Drivers (2) • Classically we distinguish two types • Block devices (disks..) • Character devices (keyboards, printers..) • Handles (typically) • Abstract requests “reads” and “writes” • Communication with device controller • Initialization • Power management

28. Device-Independent I/O Software (1) Generic I/O management in the kernel Functions of the device-independent I/O software

29. Uniform Interfacing for Device Drivers (a) Without a standard driver interface (b) With a standard driver interface

30. Device-Independent I/O Software (3) • How are devices addressed? • UNIX/Linux • Major number • Identifying the device driver • Minor number • Identifying the virtual device

31. Linux Driver Registration • Drivers (modules) need to register with the kernel • Kernel keeps table of drivers • Special kernel routines for adding/deleting entries

32. Buffering • Where to put the buffers • User • Kernel • What if the buffer is full? • How many buffers? • When to tell the user app?

33. Device-Independent I/O Software (a) Unbuffered input (b) Buffering in user space (c) Buffering in the kernel followed by copying to user space (d) Double buffering in the kernel

34. I/O Buffering • Where to store data to be sent? • User buffer • Locking? • When finished? • Kernel buffer • No locking • Device controller

35. Device-Independent I/O Software Networking may involve many copies

36. User-Space I/O Software filesystem Layers of the I/O system and the main functions of each layer

37. More I/O • Read more about disks, clocks, terminals etc. in Tanenbaum • Disks: • Slow • Block size (transfer unit) • Read about RAID 0-5! (Tanenbaum 302-306) • Covered in the exercise

38. Redundant Array of Independent Disks (RAID)

39. File Systems • UNIX: Everything is a file! • Main memory is not enough • Persistency • Not large enough • Need a structuring of “data”

40. Long-term Information Storage • Must store large amounts of data • Information stored must survive the termination of the process using it • Multiple processes must be able to access the information concurrently • Store information on disks and other external media in units called files • Filesystem: OS Part that manages files

41. File Naming Typical file extensions

42. File Structure Three common ways how OS structures files a) byte sequence b) record sequence c) Tree (records of varied length, key field)

43. File Types • Regular Files • ASCII • binary (a) executable (b) archive • Directories • Character special files • Block special files

44. File Access • Sequential access (early OS) • read all bytes/records from the beginning • cannot jump around, could rewind or back up • convenient when medium was magnetic tape • Random access (modern OS) • bytes/records read in any order • essential for many app., e.g., database • read can be … • move file marker (seek) and then read, or • read and then move file marker

45. File Attributes Possible file attributes (also metadata)

46. Create Delete Open Close Read Write Append Seek Get attributes Set attributes Rename Lock File Operations

47. An Example Program Using File System Calls (1/2)

48. An Example Program Using File System Calls (2/2)

49. Memory-Mapped Files • Accessing files using read/writes is sometimes cumbersome • Alternative: map the whole file in memory • Access the memory directly

50. Memory-Mapped Files (a) Segmented process before mapping files into its address space (b) Process after mapping existing file abc into one segment creating new segment for xyz