320 likes | 456 Views
More on File Management. Chapter 12. File Management. provide file abstraction for data storage guarantee, to the extend possible, that data in the file is valid performance: throughput and response time minimize the potential for lost or destroyed data: reliability provide protection
E N D
More on File Management Chapter 12
File Management • provide file abstraction for data storage • guarantee, to the extend possible, that data in the file is valid • performance: throughput and response time • minimize the potential for lost or destroyed data: reliability • provide protection • API: create, delete, read, write files
File Naming • files must be referable by unique names • external names: symbolic • in a hierarchical file system (UNIX) external names are given as pathnames (path from the root to the file) • internal names: i-node in UNIX (an index into an array of file descriptors/headers for a volume) • directory: translation from external to internal names (more than one external name for an internal name is allowed) • information about file is split between the directory and the file descriptor (in UNIX all of it is stored in the file descriptor): size, location on disk, owner, permissions, date created, date last modified, date last access, link count (in UNIX)
Protection Mechanisms • files are OS objects: unique names and a finite set of operations that processes can perform on them • protection domain is a set of {object,rights} where right is the permission to perform one of the operations • at every instant in time, each process runs in some protection domain • in Unix, a protection domain is {uid, gid} • protection domain in Unix is switched when running a program with SETUID/SETGID set or when the process enters the kernel mode by issuing a system call • how to store all the protection domains ?
Protection Mechanisms (cont’d) • Access Control List (ACL): associate with each object a list of all the protection domains that may access the object and how • in Unix ACL is reduced to three protection domains: owner, group and others • Capability List (C-list): associate with each process a list of objects that may be accessed along with the operations • C-list implementation issues: where/how to store them (hardware, kernel, encrypted in user space) and how to revoke them
Secondary Storage Management • Space must be allocated to files • Must keep track of the space available for allocation
Preallocation • Need the maximum size for the file at the time of creation • Difficult to reliably estimate the maximum potential size of the file • Tend to overestimated file size so as not to run out of space
Methods of File Allocation • Contiguous allocation • Single set of blocks is allocated to a file at the time of creation • Only a single entry in the file allocation table • Starting block and length of the file • External fragmentation will occur
Methods of File Allocation • Chained allocation • Allocation on basis of individual block • Each block contains a pointer to the next block in the chain • Only single entry in the file allocation table • Starting block and length of file • No external fragmentation • Best for sequential files • No accommodation of the principle of locality
Methods of File Allocation • Indexed allocation • File allocation table contains a separate one-level index for each file • The index has one entry for each portion allocated to the file • The file allocation table contains block number for the index
File Allocation • contiguous: a contiguous set of blocks is allocated to a file at the time of file creation • good for sequential files • file size must be known at the time of file creation • external fragmentation • chained allocation: each block contains a pointer to the next one in the chain • consolidation to improve locality • indexed allocation: good both for sequential and direct access (UNIX)
Free Space Management • bitmap: one bit for each block on the disk • good to find a contiguous group of free blocks • small enough to be kept in memory • chained free portions: {pointer to the next one, length} • index: treats free space as a file
UNIX File System • Naming • External/Internal names, Directories • Lookup • File blocks Disk blocks • Protection • Free Space Management
File Naming • External names (used by the application) • Pathname: /usr/users/file1 • Internal names (used by the OS kernel) • I-node: file number/index on disk File system on disk 0 1 superblock I-node area ( one I-node per file) File-block area
Directories • Files which store translation tables (external names to internal names) usr usr users usr 23 users 41 file1 87 Root directory (always I-node 2) /usr/users/file1 corresponds to I-node 87
File Content Lookup • address table used to translate logical file blocks into disk blocks • address table stored in the I-node 0 1 2 45 File with i-node 87 65 Address Table 85 File System disk 45 65 85
File Protection • ACL with three protection domains (file owner, file owner group, others) • Access rights: read/write/execute • Stored in the I-node
Free Space Management • Free I-nodes • Marked as free on disk • An array of 50 free I-nodes stored in the superblock • Free file blocks • Stored as a list of 50- free block arrays • First array stored in the superblock
In-Kernel File System Data Structures fd=open(pathname,mode); /* fd = index in Per-Proc OFT */ for (..) read(fd,buf,size); close(fd); Application PCBs Per-process Open File Table OS Kernel I-node cache Per-OS Open File Table (offset in file, ptr to I-node) Buffer cache File system on disk 0 1
File System Consistency • a file system uses the buffer cache for performance reasons • two copies of a disk block (buffer cache, disk) -> consistency problem if the system crashes before all the modified blocks are written back to disk • the problem is critical especially for the blocks that contain control information (meta-data): directory blocks, i-node, free-list • Solution: • write through meta-data blocks (expensive) or order of write-back is important • ordinary file data blocks written back periodically (sync) • utility programs for checking block and directory consistency after crash
More on File System Consistency • Example 1: create a new file • Two updates: (1) allocate a free I-node; (2) create an entry in the directory • (1) and (2) must be write-through (expensive) or (1) must be written-back before (2) • If (2) is written back first and a crash occurs before (1) is written back the directory structure is inconsistent and cannot be recovered • Example 2: write a new block to a file • Two updates: (1) allocate a free block; (2) update the address table of the I-node • (1) and (2) must be write-through or (1) must be written-back before (2) • If (2) is written back first and a crash occurs before (1) is written back the I-node structure is inconsistent and cannot be recovered
Log-Structured File System (LFS) • as memory gets larger, buffer cache size increases -> increase the fraction of read requests which can be satisfied from the buffer cache with no disk access • conclusion: in the future most disk accesses will be writes • but writes are usually done in small chunks in most file systems (meta data for instance) which makes the file system highly inefficient • LFS idea (Berkeley): to structure the entire disk as a log • periodically, or when required, all the pending writes (data and metadata together) being buffered in memory are collected and written as a single contiguous segment at the end of the log
LFS segment • contain i-nodes, directory blocks and data blocks, all mixed together • each segment starts with a segment summary • segment size: 512 KB - 1MB • two key issues: • how to retrieve information from the log • how to manage the free space on disk
File location in LFS • the i-node contains the disk addresses of the file block as in the standard UNIX • but there is no fixed location for the i-node • an i-node map is used to maintain the current location of each i-node • i-node map blocks can also be scattered but a fixed checkpoint region on the disk identifies the location of all the i-node map blocks • usually i-node map blocks are cached in main memory most of the time, thus disk accesses for them are rare
Segment cleaning in LFS • LFS disk is divided in segments which are written sequentially • live data must be copied out of a segment before the segment can be re-written • the process of copying data out of a segment: cleaning • a separate cleaner thread moves along the log, removes old segments from the end and puts live data into memory for rewriting in the next segment • as a result a LFS disk appears like a big circular buffer with the writer thread adding new segments to the front and the cleaner thread removing old segments from the end • book-keeping is not trivial: i-node must be updated when blocks are moved to the current segment