1 / 53

Review – I/O subsystems

Review – I/O subsystems. OS responsibilities to device drivers Device API, buffering, naming, interrupt handling Installing device drivers Static vs. dynamic Discovering devices that are present Polling and identification Bitmap display device Bitblt to “paint” characters on screen

tompkinsa
Download Presentation

Review – I/O subsystems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Review – I/O subsystems • OS responsibilities to device drivers • Device API, buffering, naming, interrupt handling • Installing device drivers • Static vs. dynamic • Discovering devices that are present • Polling and identification • Bitmap display device • Bitblt to “paint” characters on screen • Window management File Systems

  2. Network I/O & Project #3 • Socket abstraction • An I/O stream to a network connection • Connection: • A serial conversation between processes on separate machines • Separate from all other connections • Project #3: simple web server • Listen on a port; open a socket; make a connection File Systems

  3. Review – Disks • Implementer of File abstraction • Storage of large amounts of data for very long times • Persistence, reliability • Controlled like I/O devices, but integral part of information storage subsystem • Rapidly increasing capacities, dropping prices • $0.5–$6.0 per gigabyte • Slowly improving transfer rates, seek performance • Only a factor of 5-10 in three decades! File Systems

  4. Review – Disks (continued) • Organized into cylinders, tracks, sectors • Random access • Any sector can be read & written independently of any other • Very high bandwidth for consecutive reads or writes • Seek time is (often) dominating factor in performance • Bad blocks are a fact of life • Most detected during formatting • Some occur during operation • Controller or OS must step around them • Seek optimization algorithms • Popular study topics, less popular in real systems • Long seek queues  system is out of balance File Systems

  5. Files and File Systems CS-3013 & CS-502Operating Systems File Systems

  6. File(an abstraction) • A (potentially) large amount of information or data that lives a (potentially) very long time • Often much larger than the memory of the computer • Often much longer than any computation • Sometimes longer than life of machine itself • (Usually) organized as a linear array of bytes or blocks • Internal structure is imposed by application • (Occasionally) blocks may be variable length • (Often) requiring concurrent access by multiple processes • Even by processes on different machines! File Systems

  7. File Systems and Disks • User view • File is a named,persistent collection of data • OS & file system view • File is collection of disk blocks • File System maps file names and offsets to disk blocks • Outline for this section • Files • Directories • Implementation of Files • Implementation of Directories File Systems

  8. Fundamental ambiguity • Is the file the “container of the information” or the “information” itself? • Almost all systems confuse the two. • Almost all people confuse the two. File Systems

  9. Example – Suppose that you e-mail me a document • Later, how do either of us know that we are using the same version of the document? • Windows/Outlook/Exchange/MacOS: • Time-stamp is a pretty good indication that they are • Time-stamps preserved on copy, drag and drop, transmission via e-mail, etc. • Unix/Linux • By default, time-stamps not preserved on copy, ftp, e-mail, etc. • Time-stamp associated with container, not with information File Systems

  10. Rule of Thumb • Almost always, application thinks in terms of the information • Most systems think in terms of containers Professional Guidance: Be aware of the distinction, even when the system is not File Systems

  11. Name: Although the name is not always what you think it is! Type: May be encoded in the name (e.g., .cpp, .txt) Dates: Creation, updated, last accessed, etc. (Usually) associated with container Better if associated with content Size: Length in number of bytes; occasionally rounded up Protection: Owner, group, etc. Authority to read, update, extend, etc. Locks: For managing concurrent access … Attributes of Files File Systems

  12. File Metadata • Information about a file • Maintained by the file system • Separate from file itself • Possibly attached to the file • E.g., in block #-1 • Used by OS in variety of ways • Occasionally used by application File Systems

  13. File Types File Systems

  14. Non-attribute of files – Location • Example 1: mv ~lauer/project1.doc ~cs502/public_html/S06 • Example 2: • System moves file from disk block 10,000 to disk block 20,000 • System restores a file from backup • May or may not be reflected in metadata File Systems

  15. Attribute anomaly – Unix File Naming ln ~lauer/project1.doc ~cs502/public_html/S06 ln ~lauer/project1.doc NewProject1.doc • Unix hard links allow one file to have more than one name and/or location • The real name of a Unix file is its i-node – an internal name known only to the OS; points to metadata • Hard links are not used very often in modern Unix practice • Usually safe to regard last element of path as name File Systems

  16. Operations on Files • Open, Close • Gain or relinquish access to a file • OS returns a file handle – an internal data structure letting it cache internal information needed for efficient file access • Read, Write, Truncate • Read: return a sequence of n bytes from file • Write: replace n bytes in file, and/or append to end • Truncate: throw away all but the first n bytes of file • Seek, Tell • Seek: reposition file pointer for subsequent reads and writes • Tell: get current file pointer • Create, Delete: • Conjure up a new file; or blow away an existing one File Systems

  17. File – a very powerful abstraction • Documents, code • Databases • Very large, possibly spanning multiple disks • Streams • Input, output, keyboard, display • Pipes, network connections, … • Virtual memory backing store • … File Systems

  18. File Access Methods • Sequential access • Read all bytes/records from the beginning • Cannot jump around, could possibly rewind or back up • Convenient when medium was magnetic tape or punched cards • Random access • Bytes/records read in any order • Essential for data base systems • Read can be … • move file pointer (seek), then read or … • read and then move file marker • Keyed or indexed access – access items in file based on the contents of an (part of an) item in the file • Provided in older commercial operating systems (IBM ISAM) • (Usually) handled separately by database applications in modern systems File Systems

  19. Questions? File Systems

  20. Directory – A Special Kind of File • A tool for users & applications to organize and find files • User-friendly names • Names that are meaningful over long periods of time • The data structure for OS to locate files (i.e., containers) on disk File Systems

  21. Directory structures • Single level • One directory per system, one entry pointing to each file • Small, single-user or single-use systems • PDA, cell phone, etc. • Two-level • Single “master” directory per system • Each entry points to one single-level directory per user • Uncommon in modern operating systems • Hierarchical • Any directory entry may point to • Individual file • Another directory • Used in most modern operating systems File Systems

  22. Directory Considerations • Efficiency – locating a file quickly. • Naming – convenient to users. • Separate users can use same name for separate files. • The same file can have different names for different users. • Names need only be unique within a directory • Grouping – logical grouping of files by properties • e.g., all Java programs, all games, … File Systems

  23. Directory Organization – Hierarchical • Most systems support idea of current (working) directory • Absolute names – fully qualified from root of file system • /usr/group/foo.c • Relative names – specified with respect to working directory • foo.c • A special name – the working directory itself • “.” • Modified Hierarchical – Acyclic Graph (no loops) and General Graph • Allow directories and files to have multiple names • Links are file names (directory entries) that point to existing (source) files File Systems

  24. Links • Hard links: bi-directional relationship between file names and file • A hard link is directory entry that points to a source file’s metadata • Metadata maintains reference count of the number of hard links pointing to it – link reference count • Link reference count is decremented when a hard link is deleted • File data is deleted and space freed when the link reference count goes to zero • Symbolic (soft) links: uni-directional relationship between a file name and the file • Directory entry points to new metadata • Metadata points to the source file • If the source file is deleted, the link pointer is invalid File Systems

  25. Path Name Translation • Assume that I want to open “/home/lauer/foo.c” fd = open(“/home/lauer/foo.c”, O_RDWR); • File System does the following • Opens directory “/” – the root directory is in a known place on disk • Search root directory for the directory home and get its location • Open home and search for the directory lauer and get its location • Open lauer and search for the file foo.c and get its location • Open the file foo.c • Note that the process needs the appropriate permissions • File Systems spend a lot of time walking down directory paths • This is why open calls are separate from other file operations • File System attempts to cache prefix lookups to speed up common searches File Systems

  26. Directory Operations • Create: • Make a new directory • Add, Delete entry: • Invoked by file create & destroy, directory create & destroy • Find, List: • Search or enumerate directory entries • Rename: • Change name of an entry without changing anything else about it • Link, Unlink: • Add or remove entry pointing to another entry elsewhere • Introduces possibility of loops in directory graph • Destroy: • Removes directory; must be empty File Systems

  27. More on Directories • Fundamental mechanism for interpreting file names in an operating system • Orphan: a file not named in any directory • Cannot be opened by any application (or even OS) • (Often) does not even have name! • Tools • FSCK – check & repair file system, find orphans • Delete_on_close attribute • Special directory entry: “..”  parent in hierarchy • Essential for maintaining integrity of directory system • Useful for relative naming File Systems

  28. Break File Systems

  29. Implementation of Files • Map file abstraction to physical disk blocks • Some goals • Efficient in time, space, use of disk resources • Fast enough for application requirements • Effective for a wide variety of file sizes • Many small files (< 1 page) • Huge files (100’s of gigabytes, terabytes, spanning disks) • Everything in between File Systems

  30. File Allocation Schemes • Contiguous • Blocks of file stored in consecutive disk sectors • Directory points to first entry • Linked • Blocks of file scattered across disk, as linked list • Directory points to first entry • Indexed • Separate index block contains pointers to file blocks • Directory points to index block File Systems

  31. Contiguous Allocation • Ideal for large, static files • Databases, fixed system structures, OS code • CD-ROM, DVD • Simple address calculation • Block address  sector address • Fast multi-block reads and writes • Minimize seeks between blocks • Prone to fragmentation when … • Files come and go • Files change size • Similar to unpaged virtual memory File Systems

  32. Bad Block Management –Contiguous Allocation • Bad blocks must be concealed • Foul up the block-to-sector calculation • Methods • Spare sectors in each track, remapped by formatting • Look-aside list of bad sectors • Check each sector request against hash table • If present, substitute a replacement sector behind the scene • Handling • Disk controller, invisible to OS • Lower levels of OS file system; invisible to application File Systems

  33. Contiguous Allocation – Extents • Extent: a contiguously allocated subset of a file • Directory entry contains • (For file with one extent) the extent itself • (For file with multiple extents) pointer to an extent block describing multiple extents • Advantages • Speed, ease of address calculation of contiguous file • Avoids (some of) the fragmentation issues • Can be extended to support files across multiple disks • Disadvantages • Degenerates to indexed allocation in Unix-like systems File Systems

  34. Blocks scattered across disk Each block contains pointer to next block Directory points to first and last blocks Sector header: Pointer to next block ID and block number of file Linked Allocation File Systems

  35. Linked Allocation • Advantages • No space fragmentation! • Easy to create, extend files • Ideal for lots of small files • Disadvantages • Lots of disk arm movement • Space taken up by links • Sequential access only! File Systems

  36. Variation on Linked Allocation – File Allocation Table • Instead of a link oneach block, put alllinks in one table • the FAT (File Allocation Table) • One entry per physical block in disk • Directory points to first & last blocks of file • Each block points to next block (or EOF) File Systems

  37. FAT File Systems • Advantages • Advantages of Linked File System • FAT can be cached in memory • Searchable at CPU speeds, pseudo-random access • Disadvantages • Limited size, not suitable for very large disks • FAT cache describes entire disk, not just open files! • Not fast enough for large databases • Used in MS-DOS, early Windows systems File Systems

  38. Disk Defragmentation • Re-organize blocks in disk so that file is (mostly) contiguous • Link or FAT organization preserved • Minimizes disk arm movement during sequential accesses File Systems

  39. Bad Block Management –Linked and FAT Systems • In OS:– format all sectors of disk • Don’t reserve any spare sectors • Allocate bad blocks to a hidden file for the purpose • If a block becomes bad, append to the hidden file • Advantages • Very simple • No look-aside or sector remapping needed • Totally transparent without any hidden mechanism File Systems

  40. Indexed Allocation • i-node: • Part of file metadata • Data structure lists the sector address of each block of file • Advantages • True random access • Only i-nodes of open files need to be cached • Supports small and large files File Systems

  41. Unix i-nodes • Direct blocks: • Pointers to first n sectors • Single indirect table: • Extra block containing pointers to blocks n+1 .. n+m • Double indirect table: • Extra block containing single indirect blocks • … File Systems

  42. Indexed Allocation • Access to every block of file is via i-node • Bad block management • Same as Linked/FAT systems • Disadvantage • Not as fast as contiguous allocation for large databases • Requires reference to i-node for every access vs. • Simple calculation of block to sector address File Systems

  43. Questions? File Systems

  44. Implementation of Directories • A list of [name, information] pairs • Must be scalable from very few entries to very many • Name: • User-friendly, variable length • Any language • Fast access by name • Information: • File metadata (itself) • Pointer to file metadata block (or i-node) on disk • Pointer to first & last blocks of file • Pointer to extent block(s) • … File Systems

  45. name1 attributes name2 attributes name3 attributes name4 attributes … … Very Simple Directory • Short, fixed length names • Attribute & disk addresses contained in directory • MS-DOS, etc. File Systems

  46. name1 name2 name3 name4 … Simple Directory i-node i-node i-node i-node Data structurescontaining attributes • Short, fixed length names • Attributes in separate blocks (e.g., i-nodes) • Attribute pointers are disk addresses (or i-node numbers) • Older Unix versions File Systems

  47. attributes attributes attributes attributes … … name1 longer_name3 very_long_name4 name2 … More Interesting Directory • Variable length file names • Stored in heap at end • Modern Unix, Windows • Linear or logarithmic search for name • Compaction needed after • Deletion, Rename File Systems

  48. Very Large Directories • Hash-table implementation • Each hash chain like a small directory with variable-length names • Must be sorted for listing File Systems

  49. Free Block Management • Bitmap • Very compact on disk • Expensive to search • Supports contiguous allocation • Free list • Linked list of free blocks • Each block contains pointer to next free block • Only head of list needs to be cached in memory • Very fast to search and allocate • Contiguous allocation vary difficult File Systems

  50. Free Block ManagementBit Vector 0 1 2 n-1 … 0  block[i] free 1  block[i] occupied bit[i] =  Free block number calculation (number of bits per word) * (number of 0-value words) + offset of first 1 bit File Systems

More Related