1 / 32

More on File Management

More on File Management. Chapter 12. File Management. provide file abstraction for data storage guarantee, to the extend possible, that data in the file is valid performance: throughput and response time minimize the potential for lost or destroyed data: reliability provide protection

christina
Download Presentation

More on File Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. More on File Management Chapter 12

  2. File Management • provide file abstraction for data storage • guarantee, to the extend possible, that data in the file is valid • performance: throughput and response time • minimize the potential for lost or destroyed data: reliability • provide protection • API: create, delete, read, write files

  3. File Naming • files must be referable by unique names • external names: symbolic • in a hierarchical file system (UNIX) external names are given as pathnames (path from the root to the file) • internal names: i-node in UNIX (an index into an array of file descriptors/headers for a volume) • directory: translation from external to internal names (more than one external name for an internal name is allowed) • information about file is split between the directory and the file descriptor (in UNIX all of it is stored in the file descriptor): size, location on disk, owner, permissions, date created, date last modified, date last access, link count (in UNIX)

  4. Protection Mechanisms • files are OS objects: unique names and a finite set of operations that processes can perform on them • protection domain is a set of {object,rights} where right is the permission to perform one of the operations • at every instant in time, each process runs in some protection domain • in Unix, a protection domain is {uid, gid} • protection domain in Unix is switched when running a program with SETUID/SETGID set or when the process enters the kernel mode by issuing a system call • how to store all the protection domains ?

  5. Protection Mechanisms (cont’d) • Access Control List (ACL): associate with each object a list of all the protection domains that may access the object and how • in Unix ACL is reduced to three protection domains: owner, group and others • Capability List (C-list): associate with each process a list of objects that may be accessed along with the operations • C-list implementation issues: where/how to store them (hardware, kernel, encrypted in user space) and how to revoke them

  6. Secondary Storage Management • Space must be allocated to files • Must keep track of the space available for allocation

  7. Preallocation • Need the maximum size for the file at the time of creation • Difficult to reliably estimate the maximum potential size of the file • Tend to overestimated file size so as not to run out of space

  8. Methods of File Allocation • Contiguous allocation • Single set of blocks is allocated to a file at the time of creation • Only a single entry in the file allocation table • Starting block and length of the file • External fragmentation will occur

  9. Methods of File Allocation • Chained allocation • Allocation on basis of individual block • Each block contains a pointer to the next block in the chain • Only single entry in the file allocation table • Starting block and length of file • No external fragmentation • Best for sequential files • No accommodation of the principle of locality

  10. Methods of File Allocation • Indexed allocation • File allocation table contains a separate one-level index for each file • The index has one entry for each portion allocated to the file • The file allocation table contains block number for the index

  11. File Allocation • contiguous: a contiguous set of blocks is allocated to a file at the time of file creation • good for sequential files • file size must be known at the time of file creation • external fragmentation • chained allocation: each block contains a pointer to the next one in the chain • consolidation to improve locality • indexed allocation: good both for sequential and direct access (UNIX)

  12. Free Space Management • bitmap: one bit for each block on the disk • good to find a contiguous group of free blocks • small enough to be kept in memory • chained free portions: {pointer to the next one, length} • index: treats free space as a file

  13. UNIX File System • Naming • External/Internal names, Directories • Lookup • File blocks  Disk blocks • Protection • Free Space Management

  14. File Naming • External names (used by the application) • Pathname: /usr/users/file1 • Internal names (used by the OS kernel) • I-node: file number/index on disk File system on disk 0 1 superblock I-node area ( one I-node per file) File-block area

  15. Directories • Files which store translation tables (external names to internal names) usr usr users usr 23 users 41 file1 87 Root directory (always I-node 2) /usr/users/file1 corresponds to I-node 87

  16. File Content Lookup • address table used to translate logical file blocks into disk blocks • address table stored in the I-node 0 1 2 45 File with i-node 87 65 Address Table 85 File System disk 45 65 85

  17. File Protection • ACL with three protection domains (file owner, file owner group, others) • Access rights: read/write/execute • Stored in the I-node

  18. Free Space Management • Free I-nodes • Marked as free on disk • An array of 50 free I-nodes stored in the superblock • Free file blocks • Stored as a list of 50- free block arrays • First array stored in the superblock

  19. In-Kernel File System Data Structures fd=open(pathname,mode); /* fd = index in Per-Proc OFT */ for (..) read(fd,buf,size); close(fd); Application PCBs Per-process Open File Table OS Kernel I-node cache Per-OS Open File Table (offset in file, ptr to I-node) Buffer cache File system on disk 0 1

  20. File System Consistency • a file system uses the buffer cache for performance reasons • two copies of a disk block (buffer cache, disk) -> consistency problem if the system crashes before all the modified blocks are written back to disk • the problem is critical especially for the blocks that contain control information (meta-data): directory blocks, i-node, free-list • Solution: • write through meta-data blocks (expensive) or order of write-back is important • ordinary file data blocks written back periodically (sync) • utility programs for checking block and directory consistency after crash

  21. More on File System Consistency • Example 1: create a new file • Two updates: (1) allocate a free I-node; (2) create an entry in the directory • (1) and (2) must be write-through (expensive) or (1) must be written-back before (2) • If (2) is written back first and a crash occurs before (1) is written back the directory structure is inconsistent and cannot be recovered • Example 2: write a new block to a file • Two updates: (1) allocate a free block; (2) update the address table of the I-node • (1) and (2) must be write-through or (1) must be written-back before (2) • If (2) is written back first and a crash occurs before (1) is written back the I-node structure is inconsistent and cannot be recovered

  22. Log-Structured File System (LFS) • as memory gets larger, buffer cache size increases -> increase the fraction of read requests which can be satisfied from the buffer cache with no disk access • conclusion: in the future most disk accesses will be writes • but writes are usually done in small chunks in most file systems (meta data for instance) which makes the file system highly inefficient • LFS idea (Berkeley): to structure the entire disk as a log • periodically, or when required, all the pending writes (data and metadata together) being buffered in memory are collected and written as a single contiguous segment at the end of the log

  23. LFS segment • contain i-nodes, directory blocks and data blocks, all mixed together • each segment starts with a segment summary • segment size: 512 KB - 1MB • two key issues: • how to retrieve information from the log • how to manage the free space on disk

  24. File location in LFS • the i-node contains the disk addresses of the file block as in the standard UNIX • but there is no fixed location for the i-node • an i-node map is used to maintain the current location of each i-node • i-node map blocks can also be scattered but a fixed checkpoint region on the disk identifies the location of all the i-node map blocks • usually i-node map blocks are cached in main memory most of the time, thus disk accesses for them are rare

  25. Segment cleaning in LFS • LFS disk is divided in segments which are written sequentially • live data must be copied out of a segment before the segment can be re-written • the process of copying data out of a segment: cleaning • a separate cleaner thread moves along the log, removes old segments from the end and puts live data into memory for rewriting in the next segment • as a result a LFS disk appears like a big circular buffer with the writer thread adding new segments to the front and the cleaner thread removing old segments from the end • book-keeping is not trivial: i-node must be updated when blocks are moved to the current segment

More Related