File-System Interface and Implementation

1. File-System Interface and Implementation

2. Chapter 9: File-System Interface • File Concept • Access Methods • Disk and Directory Structure • File-System Mounting • File Sharing • Protection

3. Objectives • To explain the function of file systems • To describe the interfaces to file systems • To discuss file-system design tradeoffs, including access methods, file sharing, file locking, and directory structures • To explore file-system protection

4. File Concept • Uniform logical view of information storage (no matter the medium) • OS abstracts from physical properties into a logical storage unit, the file • Files mapped onto physical devices, usually nonvolatile • File is a collection of related information • Smallest allotment of nameable storage • Contiguous logical address space • Types: • Data • numeric • character • binary • Program • May be free form or rigidly formed (structured)

5. File Structure • None - sequence of words, bytes • Simple record structure • Lines • Fixed length • Variable length • Complex Structures • Formatted document • Relocatable load file • Can simulate last two with first method by inserting appropriate control characters • Who decides: • Operating system • Program / programmer

6. File Attributes • Name – only information kept in human-readable form • Identifier – unique tag (number) identifies file within file system; it is the non-human -readable name for the file. • Type – needed for systems that support different types • Location – pointer to file location on device • Size – current file size(in bytes, words, or blocks)and possibly the maximum allowed size are included in this attribute. • Protection – controls who can do reading, writing, executing • Time, date, and user identification –This information may be kept for creation,last modification,and last use. These data can be useful for protection, security, and usage monitoring • Information about files are kept in the directory structure, which is maintained on the disk • Typically,a directory entry consists of the file’s name and identifier • Identifier locates other file attributes • Attributes may be > 1KB to record this information for each file • Directory structures may be > 1MB

7. File Operations • File is an abstract data type • Operations include the following (and usually more) • Create – find space, add entry to directory • Write – write data at current file position pointer location and update pointer • Read – read file contents at pointer location, update pointer • Reposition within file (seek) – change pointer location • Delete – free space and remove entry from directory • Truncate – delete data starting at pointer

8. File Types • Most operating systems recognize file types • Filename extension • I.e. resume.doc, server.java, readerthread.c • Most support them • Automatically open a type of file via a specific application (.doc) • Only execute files of a given extension (.exe, .com) • Run files of a given type via a scripting language (.bat) • Can get more advanced • If source code modified since executable compiled, if attempt made to execute, recompile and then execute (TOPS-20) • Mac OS encodes creating program’s name in file attributes • Double clicking on file passes the file name to appropriate application • Unix has magic number stored in file at first byte indicating file type

9. File Types – Name, Extension

10. File Structure • Types can indicate internal file structure • Some Oses enforce, some use as hints, some ignore • But some most conform to OS-required format • I.e. executable file • Some support more formats • DEC VMS supported 3 • The more that are supported, the more kernel code, etc • Some enforce access methods • Others allow arbitrary access • Unix supports directory files, executable files • But all files are strings of bytes • Can open a directory file via a text editor • Files stored in fixed-size disk blocks • Can have internal fragmentation

11. Access Methods • Sequential Access – tape model of a file read next write next reset no read after last write (rewrite) • Direct Access – random access, relative access read n write n position to n read next write next rewrite n n = relative block number • Can accommodate structured data in file by mapping record number to block number • Oses usually support both kinds, sometimes require access method declaration during create()

12. Sequential-access File

13. Simulation of Sequential Access on Direct-access File

14. Example of Index and Relative Files

15. Disk Structure • Disk can be subdivided into partitions • Disks or partitions can be RAID protected against failure • Disk or partition can be used raw– without a file system, or formattedwith a file system • Partitions also known as minidisks, slices • Entity containing file system known as a volume • Each volume containing file system also tracks that file system’s info in device directoryor volume table of contents or directory) • Records information for all files on the volume • As well as general-purpose file systemsthere are many special-purpose file systems, frequently all within the same operating system or computer

16. A Typical File-system Organization

17. File System Types • Operating systems have multiple file system types • One or more general-purpose (for storing user files) • One or more special-purpose, i.e. • tmpfs—“temporary” file system in volatile main memory, contents erased if the system reboots or crashes • objfs—a “virtual” file system (essentially an interface to the kernel that looks like a file system) that gives debuggers access to kernel symbols • ctfs— a virtual file system that maintains “contract” information to manage which processes start when the system boots and must continue to run during operation • lofs—a “loop back” file system that allows one file system to be accessed in place of another one • procfs—a virtual file system that presents information on all processes as a file system

18. Directory Overview • Directory similar to symbol table translating file names to their directory entries • Can be organized in many ways • Organization needs to support operations including: • Search for a file or multiple files • Create a file • Delete a file • List a directory • Rename a file • Traverse the file system

19. Directory Organization • Should have the features • Efficiency – locating a file quickly • Naming – convenient to users • Two users can have same name for different files • The same file can have several different names • Grouping – logical grouping of files by properties, (e.g., all Java programs, all games, …) or arbitrarily

20. Single-Level Directory • A single directory for all users Naming problem Grouping problem

21. Two-Level Directory • Separate directory for each user • Path name • Can have the same file name for different users • Efficient searching • No grouping capability

22. Added Directory Concepts • Many variations, but some components essential • Idea of current directory – default location for activities • Now need a path specification • If file is in current directory, just name it • If in another directory, must specify by more detailed name • Also need way to specify different filesystems • MS-DOS gives letter to each volume, “\” separates directory name from file name – C:\userb\test • VMS uses letter for volume and “[]” for directory specification – u:[sst.jdeck]login.com;1 • Note the support for versions via the trailing number • Unix treats volume name as part of directory name - /u/pbg/test • Many Oses search a set of paths for command names • “ls” might search in current directory then in system directories

23. Tree-Structured Directories

24. Tree-Structured Directories (Cont.) • Most common • For example, allows users to can create directories within their directory • Directory can then contain files or other directories • Directory can be another file with defined formatting and attribute indicating its type • Separate system calls to manage directory actions • Absolute path is full specification of file local - /foo/bar/baz • Relative path is location relative to current directory - ../baz • Efficient searching • Search path • Grouping Capability • Current directory (working directory) • cd /spell/mail/prog • type list

25. Tree-Structured Directories (Cont) mail • Creating a new file is done in current directory • Delete a file rm <file-name> • Creating a new subdirectory is done in current directory mkdir <dir-name> Example: if in current directory /mail mkdir count prog copy prt exp count Deleting “mail”  deleting the entire subtree rooted by “mail”? • Make users manually delete contents (and subcontents) first (MS-DOS) • Provide an option to delete all contents (Unix)

26. Protection • File owner/creator should be able to manage controlled access: • What can be done • By whom • But never forget physical security • Types of access • Read • Write • Execute • Append • Delete • List • Others can include renaming, copying, editing, etc • System calls then check for valid rights before allowing operations • Another reason for open() • Many solutions proposed and implemented

27. Access Lists and Groups • Mode of access: read, write, execute • Three classes of users RWX a) owner access 7  1 1 1 RWX b) group access 6  1 1 0 RWX c) public access 1  0 0 1 • Ask manager to create a group (unique name), say G, and add some users to the group. • For a particular file (say game) or subdirectory, define an appropriate access. owner group public chmod 761 game Attach a group to a file chgrp G game

28. Access Control • More generally solved via access control lists • For a given entity, keep list of user-ids allowed to access and what access methods • Constructing such as list can be tedious and unrewarding • Data structure must be stored somewhere • Variable size

29. Windows XP Access-Control List Management

30. A Sample UNIX Directory Listing

31. Directory Implementation • Linear list of file names with pointer to the data blocks • Simple to program • Time-consuming to execute • Linear search time • Could keep ordered alphabetically via linked list or use B+ tree • Hash Table – linear list with hash data structure • Decreases directory search time • Collisions– situations where two file names hash to the same location • Only good if entries are fixed size, or use chained-overflow method

32. Allocation Methods - Contiguous • An allocation method refers to how disk blocks are allocated for files: • Contiguous allocation – each file occupies set of contiguous blocks • Best performance in most cases • Simple – only starting location (block #) and length (number of blocks) are required • Problems include finding space for file, knowing file size, external fragmentation, need for compaction off-line (downtime) or on-line

33. Contiguous Allocation • Mapping from logical to physical Q LA/512 R Block to be accessed = Q + starting address Displacement into block = R

34. Contiguous Allocation of Disk Space

35. Extent-Based Systems • Many newer file systems (i.e., Veritas File System) use a modified contiguous allocation scheme • Extent-based file systems allocate disk blocks in extents • An extentis a contiguous block of disks • Extents are allocated for file allocation • A file consists of one or more extents

36. Allocation Methods - Linked • Linked allocation – each file a linked list of blocks • File ends at nil pointer • No external fragmentation • Each block contains pointer to next block • No compaction, external fragmentation • Free space management system called when new block needed • Improve efficiency by clustering blocks into groups but increases internal fragmentation • Reliability can be a problem • Locating a block can take many I/Os and disk seeks • FAT (File Allocation Table) variation • Beginning of volume has table, indexed by block number • Much like a linked list, but faster on disk and cacheable • New block allocation simple

37. pointer block = Linked Allocation • Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk

38. Linked Allocation • Mapping Q LA/511 R Block to be accessed is the Qth block in the linked chain of blocks representing the file. Displacement into block = R + 1

39. Linked Allocation

40. File-Allocation Table

41. Allocation Methods - Indexed • Indexed allocation • Each file has its own index block(s) of pointers to its data blocks • Logical view index table

42. Example of Indexed Allocation

43. Indexed Allocation (Cont.) • Need index table • Random access • Dynamic access without external fragmentation, but have overhead of index block • Mapping from logical to physical in a file of maximum size of 256K bytes and block size of 512 bytes. We need only 1 block for index table Q LA/512 R Q = displacement into index table R = displacement into block

44. Indexed Allocation – Mapping (Cont.) • Mapping from logical to physical in a file of unbounded length (block size of 512 words) • Linked scheme – Link blocks of index table (no limit on size) Q1 LA / (512 x 511) R1 Q1= block of index table R1is used as follows: Q2 R1 / 512 R2 Q2 = displacement into block of index table R2 displacement into block of file:

45. Indexed Allocation – Mapping (Cont.) • Two-level index (4K blocks could store 1,024 four-byte pointers in outer index -> 1,048,567 data blocks and file size of up to 4GB) Q1 LA / (512 x 512) R1 Q1 = displacement into outer-index R1 is used as follows: Q2 R1 / 512 R2 Q2 = displacement into block of index table R2 displacement into block of file:

46. Indexed Allocation – Mapping (Cont.)  outer-index file index table

47. Combined Scheme: UNIX UFS (4K bytes per block, 32-bit addresses) Note: More index blocks than can be addressed with 32-bit file pointer

48. Performance • Best method depends on file access type • Contiguous great for sequential and random • Linked good for sequential, not random • Declare access type at creation -> select either contiguous or linked • Indexed more complex • Single block access could require 2 index block reads then data block read • Clustering can help improve throughput, reduce CPU overhead

49. Performance (Cont.) • Adding instructions to the execution path to save one disk I/O is reasonable • Intel Core i7 Extreme Edition 990x (2011) at 3.46Ghz = 159,000 MIPS • http://en.wikipedia.org/wiki/Instructions_per_second • Typical disk drive at 250 I/Os per second • 159,000 MIPS / 250 = 630 million instructions during one disk I/O • Fast SSD drives provide 60,000 IOPS • 159,000 MIPS / 60,000 = 2.65 millions instructions during one disk I/O

50. Free-Space Management • File system maintains free-space list to track available blocks/clusters • (Using term “block” for simplicity) • Bit vector or bit map (n blocks) 0 1 2 n-1 …  1  block[i] free 0  block[i] occupied bit[i] = Block number calculation (number of bits per word) * (number of 0-value words) + offset of first 1 bit CPUs have instructions to return offset within word of first “1” bit