File-System Interface

2. File-System Interface • File Concept • File Operations • Access Methods • Directory Structure • Access control GMU – CS 571

3. Files • A file is a named collection of related information that is recorded on secondary storage • Several information storage media (magnetic/optical disks) • The operating system provides a uniform logical view of information storage GMU – CS 571

4. Files • Files • are mapped onto physical storage devices. • represent programs (both source and object forms) and data. • have a certain structure that may be considered as sequence of bits, bytes, lines, records… • meaning defined by file’s creator • have attributes that are recorded by the O.S. (name, size, type, location, protection info, time info, etc.) • logically contiguous • Information about files are kept in the directory structure, which is also maintained on the secondary storage. GMU – CS 571

5. Basic File Operations • Create • Write • Read • Delete • Others • reposition within the file, append, rename, truncate, ... • For write/read operations, the operating system needs to keep a file position pointer for each process • Need to update it dynamically and properly GMU – CS 571

6. File Operations • To avoid searching the directory entries repeatedly, many systems require that an open() system call be issued before that file is first used actively. • Operating System keeps • a system-wide open-file table containing information about all open files • per-process open-file tables containing information about all open files of each process • The open operation takes a file name and searches the directory, copying the directory entry into the open-file table. It returns a pointer to the entry in the open file table. GMU – CS 571

7. File Operations • The per-process open table contains info about • Position pointer (current location within file) • Access rights • Accounting • Pointer to the system-wide open-file table entry • The system-wide open table includes info about • File location on the disk • File size • File open count (the number of processes using this file) • A process that completes its operations on a given file will issue a close() system call. GMU – CS 571

8. Process A’s Open-File Table . . . . . . . . . . . File Operations (Cont.) . . . . . . . . . System-Wide Open-File Table Process B’s Open-File Table GMU – CS 571

9. An Example Program Using File System Calls (1/3) /* File copy program. Error checking and reporting is minimal. */ /* “myfilecopy oldfile newfile” will copy the contents of “oldfile” to “newfile” */ /* The program will read blocks of 4K from the “oldfile” to a buffer, and store them to “newfile” sequentially */ #include <sys/types.h> /* include necessary header files */ #include <fcntl.h> #include <stdlib.h> #include <unistd.h> int main(int argc, char *argv[]); /* ANSI prototype */ #define BUF_SIZE 4096 /* use a buffer size of 4096 bytes */ #define OUTPUT_MODE 0700 /* protection bits for output file */ GMU – CS 571

10. An Example Program Using File System Calls (2/3) int main(int argc, char *argv[]) { int in_fd, out_fd, rd_count, wt_count; char buffer[BUF_SIZE]; if (argc != 3) exit(1); /* error if argc is not 3 */ /* Open the input file and create the output file */ in_fd = open(argv[1], O_RDONLY); /* open the source file */ if (in_fd < 0) exit(2); /* if it cannot be opened, exit */ out_fd = creat(argv[2], OUTPUT_MODE); /* create the destination file */ if (out_fd < 0) exit(3); /* if it cannot be created, exit */ GMU – CS 571

11. An Example Program Using File System Calls (3/3) /* Copy loop */ while (TRUE) { rd_count = read(in_fd, buffer, BUF_SIZE); /* read a block of data */ if (rd_count <= 0) break; /* if end of file or error, exit loop */ wt_count = write(out _fd, buffer, rd_count); /* write data */ if (wt_count <= 0) exit(4); /* wt_count <= 0 is an error */ } /* Close the files */ close(in_fd); close(out_fd); if (rd_count == 0) /* no error on last read */ exit(0); else exit(5); /* error on last read */ } GMU – CS 571

12. File Types • Most operating systems associate a type with a file • File type can be used to operate on files in reasonable ways • ex: Windows – file type (i.e. suffix) used to determine what program to open a file with • ex: Unix – info stored in file (‘magic number’) can be used for differentiation – suffix not always used GMU – CS 571

13. File Types – Name, Extension GMU – CS 571

14. File Structure • None - sequence of words, bytes • Simple record structure • Lines • Fixed length • Variable length • Complex Structures • Formatted document • Relocatable load file • Can simulate last two methods with first method by inserting appropriate control characters • Who decides: • Operating system • Program GMU – CS 571

15. Internal File Structure • Disk systems have a well-defined block size determined by the size of a sector. • All disk I/O is performed in units of one block (physical record). • Each block is one or more sectors • A sector can hold 32 – 4096 bytes • Files are made of logical records. • Often, a number of logical records will be packed into physical records. • Operating System will perform translation from logical records to physical records. • Internal fragmentation GMU – CS 571

16. File Access Methods • Sequential Access • Information is processed in order, one record after the other (tape model) • Example: editors and compilers read next write next reset (rewind) GMU – CS 571

17. File Access Methods • Direct Access • The file is made up fixed-length logical records that allow programs to read and write records rapidly in any order read n write n or alternatively: position to n read next write next n = relative block number • request to read block N translated into physical address B*N + start (for block size B) • ex: database • Other access methods often built on top of direct access GMU – CS 571

18. Directory Structure • The directory acts as a symbol table that translates file names into their directory entries. • Operations on a directory • Search for a file • Create a file • Delete a file • List a directory • Rename a file • … GMU – CS 571

19. Organize the Directory (Logically) to Obtain • Efficiency – locating a file quickly • Naming – convenient to users • Two users can have same name for different files • The same file can have several different names • Grouping – logical grouping of files by properties, (e.g., all Java programs, all games, …) GMU – CS 571

20. Single-Level Directory • A single directory for all users Naming problem Grouping problem GMU – CS 571

21. Two-Level Directory • Separate directory for each user • Path name • Can have the same file name for different user • Efficient searching • No grouping capability GMU – CS 571

22. Tree Directory Structure • Tree-structureddirectories extend the structure to a tree of arbitrary height • User-imposed structure • Relative paths vs. absolute paths • Directory deletion policy • Concept of a ‘current directory’ GMU – CS 571

23. Acyclic-Graph Directories • Allows shared subdirectories and files. • A shared file will “exist” in multiple directories at once. GMU – CS 571

24. Achieving File Sharing • Option 1: Duplicate all information about the shared file in both directories (Problem?) • Option 2: Create a new directory entry called link • The link is effectively a pointer to another file or directory • When the directory entry of a referred file is a link, we resolve the link by using the path name (symbolic link in Unix) • “ln –s reports/report1.txt myreport” GMU – CS 571

25. report1.txt “reports” Directory Achieving File Sharing (Cont.) • Option 3: Each entry in a directory can point to a little data structure (File Control Block [FCB], or “i-node”) that keeps information about the file • The directory entries corresponding to a shared file will all point to the same file control block • Non-symbolic or “hard” links in Unix • “ln reports/report1.txt myreport” FCB of the file “root“ Directory myreport GMU – CS 571

26. Achieving File Sharing (Cont.) • What to do when a shared file is deleted by a user? • The deletion of a link should not affect the original file • If the original file is deleted, we may be left with dangling pointers. • Solutions • Using backpointers, delete also all links. The search may be expensive. • Alternatively, leave the links intact until an attempt is made to use them (Unix symbolic links). May lead to infrequent but subtle problems. • In case of non-symbolic (or in Unix, “hard”) links: Preserve the file until all references are deleted. Keep the count of the number of the references, delete the file when the count reaches zero. GMU – CS 571

27. File Protection • File owner/creator should be able to control: • what can be done • by whom • Types of access • Read • Write • Execute • Append • Delete • List GMU – CS 571

28. Access Lists and Groups • Mode of access: read, write, execute • Three classes of users RWX a) owner access 7  1 1 1 RWX b) group access 6  1 1 0 RWX c) public access 1  0 0 1 • Ask manager to create a group (unique name), say G, and add some users to the group. • For a particular file (say game) or subdirectory, define an appropriate access. GMU – CS 571

29. Windows XP Access-control List Management GMU – CS 571

30. A Sample UNIX Directory Listing GMU – CS 571

31. File System Implementation • File System Structure • File System Implementation • Allocation Methods • File System Performance GMU – CS 571

32. File System Structure • An operating system may allow multiple file systems. • Once the user interface is determined, the file system must be implemented to map the logical file system to the physical secondary-storage devices. • File control block – storage structure that keeps information about a given file (Unix “i-nodes”). • Ownership, size, permissions, access date info, location of data blocks GMU – CS 571

33. Schematic View of Virtual File System

34. Layered File System • File system is organized into layers • Logical File System Layer manages the file-system structure (through directories and FCBs). • File-Organization Module performs mapping between logical blocks and physical blocks. It also includes free-space manager and block allocation manager. • Basic File System Layer issues generic commands to the appropriate device driver (I/O Control Layer) to read and write physical blocks on the disk GMU – CS 571

35. Storage Structure • A disk is a physical memory storage device that can be used for: • a single file system (in its entirety) • multiple file systems • in part for file systems, in part for other purposes (e.g. for swap space or unformatted (raw) disk space) • These parts are known as partitions, slices or minidisks. GMU – CS 571

36. Storage Structure (Cont.) • Each partition can be either “raw” (containing no file system), or “cooked” (with a file system) • Raw disk • contains a large sequential array of logical blocks, without any file-system data • can be used as swap space • can be used for special (e.g. database) applications GMU – CS 571

37. Storage Structure (Cont.) • Each partition that contains a file system has a device directory • The device directory keeps information (name, location, size, type, owner) for files on that partition. GMU – CS 571

38. Accessing Disk Sub-system • Disks allow direct access to stored data • Disk access time has two components • Random access time (positioning) that includes seek time and rotational latency (5-10 ms) • Transfer time (10 MB/s) • Compare to the memory access time of 10-100 nanoseconds GMU – CS 571

39. Accessing Disk Sub-system • When a process needs I/O, it issues a system call to the OS • input or output • from what disk address • to what memory address • how many sectors • At any point in time, the disk may have several pending requests that must be scheduled: • FCFS • SSTF (shortest seek time first) • SCAN • … GMU – CS 571

40. Implementation of “Open” and “Read” • Figure (a) refers to opening a file. • Figure (b) refers to reading a file. GMU – CS 571

41. Allocation Methods • The allocation method refers to how disk blocks are allocated for files: • Contiguous allocation • Linked allocation • Indexed allocation GMU – CS 571

42. Contiguous Allocation • Each file occupies a set of contiguous blocks on the disk. • Simple – only starting location (block #) and length (number of blocks) are required. GMU – CS 571

43. Contiguous Allocation • Efficient access to multiple blocks of a file • Both sequential and direct access can be supported. • A major problem is determining how much space is needed for a new file. • How to let files grow? • Finding space for a new file: First-fit and best-fit … • These algorithms suffer from external fragmentation:free space is broken into multiple chunks. GMU – CS 571

44. Extent-Based Systems • Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme • Extent-based file systems allocate disk blocks in extents • An extent is a contiguous block of disks • Extents are allocated for file allocation • A file consists of one or more extents. GMU – CS 571

45. pointer block = Linked Allocation • Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. GMU – CS 571

46. Linked Allocation • Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk. • Each block contains a pointer to the next block. • Each directory entry has a pointer to the first and last disk blocks of the file. GMU – CS 571

47. Linked Allocation • External fragmentation is eliminated. • The size of a file does not need to be declared at the time of creation. • However, it can be used effectively only for sequential access files. Inefficient for direct-access files. • Another disadvantage is the space required for the pointers. • One solution is to collect blocks into multiples (clusters) and to allocate the clusters rather than blocks. • Another problem of linked allocation is reliability: what will happen if a pointer is lost or damaged? GMU – CS 571

48. File-Allocation Table (FAT) • A variation of the linked allocation method • A section of the disk at the beginning of each partition is used as the File Allocation Table. • The table entries give the block number of the next block in the file. • The scheme can result in a significant number of disk head seeks, unless the FAT is cached. GMU – CS 571

49. Indexed Allocation • Indexed allocation supports direct access, without suffering from external fragmentation or size-declaration problems. • However, wasted space may be a problem. • How large the index block should be? • To reduce the wasted space, we want to keep the index block small • If the index block is too small, it will not be able to hold pointers for a large file. • Linked scheme • Multilevel scheme • Combined scheme index table GMU – CS 571

50. Indexed Allocation – Mapping (Cont.)  outer-index file index table GMU – CS 571