File System: Concepts and Operations

1. CENG 334 – Operating Systems07- File Systems Asst. Prof. Yusuf Sahillioğlu Computer Eng. Dept, , Turkey

2. File Concept • We think and use files when we (process) want to store something • Can your code write something to the disk without using files? NO • fprintf • File is not a physical thing; it is an abstract entity • Actual data we’re writing to a file sits in the storage media (disk/CD) • OS provides that abstract entity via its File System component • How to place files on physical media • How to read/write • ..

3. File Concept • File concept illustration • Don’t deal with the disk directly • Instead deal with the files (logical storage: byte 0 to n)

4. File Concept • OS File System Component is a software that views the hardware as a sequence of blocks of some size • Those blocks are mapped to sectors of the disk by Disk Driver • Given a block number Disk Driver finds the corresponding sector num

5. File Concept • OS File System Component views the underneath storage as a sequence of blocks • Applications are viewing the storage as a set of FILES • Hardware has the storage as a set of sectors

6. File Concept • OS File System Component views the underneath storage as a sequence of blocks • Applications are viewing the storage as a set of FILES • There has to be a mapping from FILES to blocks

7. File Concept • A file is sitting in some blocks (contiguous or non-contiguous) • If a file content needs to occupy 4 blocks, OS File System Component decides which 4 blocks should contain the content

8. File Concept • We will first see File System Interface (which functions provided) • Then we will see how those functions are implemented • File  block mapping • So we will understand OS File System Component thoroughly

9. File Concept • File is just a contiguous logical address space (a storage) • Actual content may be non-contiguous blocks in the disk • OS makes the arrangement so that you view the file as a contiguous logical space/storage • What can we store in a file? • Data • numeric • character • binary • Program User’s (process) view of a file

10. File Structure • None (no structure at all): sequence of words, bytes • Unix, Windows • Simple record structure: sequence of records • Lines • Fixed length • Variable length • Complex Structures • Formatted document: understood by Word program, not OS • Executable file: understood by OS • Can simulate last two with first method by inserting appropriate control characters • Who decides: • Operating system • Program

11. File Attributes • Name – only information kept in human-readable form • Identifier – unique tag (number) identifies file within file system • Type – needed for systems that support different types • Location – pointer to file location on device • Size – current file size • Protection – controls who can do reading, writing, executing • Time, date, and user identification – data for protection, security, and usage monitoring • Information about files are kept in the directory structure, which is maintained on the disk, not in memory

12. File Attributes • There are 2 basic things stored on disk as part of the area controlled by the file system • Files: storage content • Directory info (can be a tree): info about files, attributes, locations • Organize files into a directory structure for efficient access • One entry per file: filename + pointer to attrbts

13. File Operations • File is an abstract data type: A class with attributes and operations • Create • Write • Read • Reposition within file • Delete • Truncate • These ops are implemented by the File Sys Component of the OS • These ops are used by the application programmer • Open(Fi) – search the directory structure on disk for entry Fi, and move the content of entry to memory • Close(Fi) – move the content of entry Fi in memory to directory structure on disk

14. File Operations • Efficient open-file manangement: open-file table • Say a read-operation comes for an open file • Search directory on the disk (slow disk access) to locate entry Fi • Go to that location indicated by the entry

15. File Operations • Efficient open-file manangement: open-file table • Say a read-operation comes for an open file • Repeat searching the directory for each read operation: slow  • OS maintains an open-file table for efficiency

16. File Operations • Efficient open-file manangement: open-file table • Only the open system call searches the directory, locates Fi, caches it into open-file table, and returns the index of open-file table entry to the process: File Descriptor

17. File Operations • Efficient open-file manangement: open-file table • Subsequent operations (read, write, ..) can then use File Descriptor handle without any further disk access

18. File Operations • Efficient open-file manangement • File pointer: pointer to last read/write location, per process that has the file open • File-open count: counter of number of times a file is open – to allow removal of data from open-file table when last processes closes it • Disk location of the file: cache of data access information • Access rights: per-process access mode information

19. File Operations • Each process has its own file-position pointer • System-wide open file table: location on disk & open-count same forall

20. File Types

21. File Operations • You can manually start the app that reads the file and load the file in it • Or OS automatically starts the app that reads the file (association)

22. File Access Methods • Sequential Access (fscanf) • read next • write next • reset • no read after last write • (rewrite) • Direct Access • read n • write n • position to n • read next • write next • rewrite n • n = relative block number

23. File Access Methods • Sequential Access

24. File Sharing • Sharing of files on multi-user systems is desirable • Sharing may be done through a protection scheme • On distributed systems, files may be shared across a network • Network File System (NFS) is a common distributed file-sharing method

25. File Sharing • User IDs identify users, allowing permissions and protections to be per-user • Group IDs allow users to be in groups, permitting group access rights • Attributes for protection

26. File Sharing • Protection is based on the use of UserIDs and GroupIDs • Each file has associated protection bits (permissions) for userID and groupID • userID: read, write, execute? • groupID: read, write, execute?

27. File Sharing Remotely • Idea: open, read, write, .. file as if it is a local file

28. File Sharing Remotely • Uses networking to allow file system access between systems • Manually via programs like FTP • Automatically, seamlessly using distributed file systems • Semi automatically via the world wide web • Client-server model allows clients to mount remote file systems from servers • Server can serve multiple clients • Client and user-on-client identification is insecure or complicated • NFS is standard UNIX client-server file sharing protocol • CIFS is standard Windows protocol • Standard operating system file calls are translated into remot calls • Distributed Information Systems (distributed naming services) such as LDAP, DNS, NIS, Active Directory implement unified access to information needed for remote computing

29. File Protection • File owner/creator should be able to control: • what can be done (read, write, execute, ..) • by whom (owner, others, group member, ..) • Types of access • Read • Write • Execute • Append • Delete • List

30. File Protection • How it is done in Unix • Mode of access: read, write, execute • Three classes of users • RWX • a) owner access 7  1 1 1 RWX • b) group access 6  1 1 0 • RWX • c) public access 1  0 0 1 • Ask manager to create a group (unique name), say G, and add some users to the group. • For a particular file (say game) or subdirectory, define an appropriate access.

31. File Protection • A sample Unix directory listing

32. File System Implementation • File system design involves • Defining File System Interface (Done!) • How file system looks to the user • What is a file and its attributes • What are the operations • Directory structure to organize files • How that File System can be implemented (Now!) • Design algorithms • Design data structures (in-memo and on-disk structures) • Map logical file system to physical storage device (disk, CD, ..) • Mapping depends on the storage device

33. File System Implementation • File control block: storage structure consisting of info about file (on disk) • Layered file system • file sys • file sys • file sys • device drivers

34. File System Implementation • Layers

35. File System Implementation • Layers

36. File System Implementation • Block to sector mapping by Disk Driver

37. File System Implementation • Now our problem is: how to map files on disk blocks? • Don’t care about details (blocksector) underneath (driver handles) • Block size is a multiple of sector size • Sector size = 512 bytes; block size can be 1024 or 4096 bytes • Unix: 4KB

38. File System Implementation • Now our problem is: how to map files on disk blocks? • 2 files occupying 2 (red) and 3 (blue) blocks

39. File System Implementation • Now our problem is: how to map files on disk blocks? • 2 files occupying 2 (red) and 3 (blue) blocks

40. File System Implementation • Now our problem is: how to map files on disk blocks? • 2 files occupying 2 (red) and 3 (blue) blocks • Eventually

41. File System Implementation • To implement a File System, file system software maintains major on-disk structures, conisiting of • Boot control block: contains info needed by system to boot OS from that volume (power on, 1st block of disk accessed (loaded into memory), a small program is run which knows where kernel is) • Volume control block: contains vol details (how many blocks, directory location, ..) • Directory structure: organizes files • Per-file file control block (FCB) contains many details about the file

42. File System Implementation • File system gets commands from the processes and issues to the driver • Driver than works on the disk

43. Directory Implementation • When you want to open/read/write/.. a file, you search the directory, for a given file name, to find the corresponding info of the file • Directory on disk; so fast search required • Linear list of file names with pointer to the data blocks • Simple to program • Time-consuming to execute • Hash table: linear list with hash data structure • Decreases directory search time • Collisions: situations where two file names hash to the same location • Only good if entries are fixed size, or use chained-overflow method

44. Allocation MEthods • How can we allocate disk space to files? Important issue! • A disk is seen (to the file sys) as a sequence of blocks • How disk blocks are allocated for files • Contiguous allocation • Linked allocation • Indexed allocation

45. Contiguous Allocation • Each file occupies a set of contiguous blocks on the disk • Simple: only starting location (block#) and length (num of blocks) are required to find out the disk data blocks of file • Random access is fast • Waste of space (dynamic storage-alloc problem; ext. fragmentation) • Files cannot grow (can grow only if block10 is empty)

46. Contiguous Allocation • Given an offset (byte X of file, or equivalenty, LA = X), what is the corresponding disk location? • Assume block size 1024 bytes • Wheich disk block contains the byte 0 of file X (LA=0)? What is the displacement on that block? • Answeer: Disk block=6 & displacement (disk block offset) = 0 • Which disk block contains the byte • at LA=2500? Where is LA=2500 mapped • on disk? • Answer: 2500/1024 = 2 • 2500%1024 = 452 • Disk block = startAddr + 2 = 6 + 2 = 8 & displacement = 452

47. Contiguous Allocation

48. Extent-Based Systems • Problem with contiguous allocation: file growing • A modified contiguous allocation scheme (newer systems) • Initially make a guess: size of my file will be X • When file tries to grow more, alloc another extent and point there • Extent-based file systems allocate disk blocks in extents • An extent is a contiguous block of disks • Extents are allocated for file allocation • A file consists of one or more extents

49. Linked Allocation • Each file is a linked list of disk blocks: blocks may be noncontiguous • Block structure:

50. Linked Allocation • File starts at block5 • Now you can write some data in block5 • If block5 filled up and more to write, find another empty block (3)