1 / 64

File Management

13. File Management. WriteFile(). CreateFile(). ReadFile(). CloseHandle(). SetFilePointer(). Fig 13-2: The External View of the File Manager. Application Program. mount(). write(). open(). close(). read(). lseek(). File Mgr. Device Mgr. Memory Mgr. File Mgr. Device Mgr.

ahmed-johnson
Download Presentation

File Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Operating Systems: A Modern Perspective, Chapter 13

  2. 13 FileManagement Operating Systems: A Modern Perspective, Chapter 13

  3. WriteFile() CreateFile() ReadFile() CloseHandle() SetFilePointer() Fig 13-2: The External View of the File Manager Application Program mount() write() open() close() read() lseek() File Mgr Device Mgr Memory Mgr File Mgr Device Mgr Memory Mgr Process Mgr Process Mgr UNIX Windows Hardware Operating Systems: A Modern Perspective, Chapter 13

  4. foo.html File Manager File Manager <head> … </head> <body> … </body> • Structured information • Can be read by any applic • Accessibility • Protocol • Persistent storage • Shared device Why Programmers Need Files <head> … </head> <body> … </body> HTML Editor Web Browser Operating Systems: A Modern Perspective, Chapter 13

  5. File Management • File is a named, ordered collection of information • The file manager administers the collection by: • Storing the information on a device • Mapping the block storage to a logical view • Allocating/deallocating storage • Providing file directories • What abstraction should be presented to programmer? Operating Systems: A Modern Perspective, Chapter 13

  6. Information Structure Applications Records Structured Record Files Record-Stream Translation Byte Stream Files Stream-Block Translation Storage device Operating Systems: A Modern Perspective, Chapter 13

  7. Byte Stream File Interface fileID = open(fileName) close(fileID) read(fileID, buffer, length) write(fileID, buffer, length) seek(fileID, filePosition) Operating Systems: A Modern Perspective, Chapter 13

  8. Low Level Files fid = open(“fileName”,…); … read(fid, buf, buflen); … close(fid); ... ... b0 b1 b2 bi int open(…) {…} int close(…) {…} int read(…) {…} int write(…) {…} int seek(…) {…} Stream-Block Translation Storage device response to commands Operating Systems: A Modern Perspective, Chapter 13

  9. Structured Files Records Record-Block Translation Operating Systems: A Modern Perspective, Chapter 13

  10. Record-Oriented Sequential Files Logical Record fileID = open(fileName) close(fileID) getRecord(fileID, record) putRecord(fileID, record) seek(fileID, position) Operating Systems: A Modern Perspective, Chapter 13

  11. Record-Oriented Sequential Files Logical Record H byte header k byte logical record ... Operating Systems: A Modern Perspective, Chapter 13

  12. Record-Oriented Sequential Files Logical Record H byte header k byte logical record ... ... Physical Storage Blocks Fragment Operating Systems: A Modern Perspective, Chapter 13

  13. Electronic Mail Example struct message { /* The mail message */ address to; address from; line subject; address cc; string body; }; struct message *getRecord(void) { struct message *msg; msg = allocate(sizeof(message)); msg->to = getAddress(...); msg->from = getAddress(...); msg->cc = getAddress(...); msg->subject = getLine(); msg->body = getString(); return(msg); } putRecord(struct message *msg) { putAddress(msg->to); putAddress(msg->from); putAddress(msg->cc); putLine(msg->subject); putString(msg->body); } Operating Systems: A Modern Perspective, Chapter 13

  14. Indexed Sequential File • Suppose we want to directly access records • Add an index to the file fileID = open(fileName) close(fileID) getRecord(fileID, index) index = putRecord(fileID, record) deleteRecord(fileID, index) Operating Systems: A Modern Perspective, Chapter 13

  15. Indexed Sequential File (cont) Application structure index = i Account # 012345 123456 294376 ... 529366 ... 965987 Index i k j index = k index = j Operating Systems: A Modern Perspective, Chapter 13

  16. More Abstract Files • Inverted files • System index for each datum in the file • Databases • More elaborate indexing mechanism • DDL & DML • Multimedia storage • Records contain radically different types • Access methods must be general Operating Systems: A Modern Perspective, Chapter 13

  17. Implementing Low Level Files • Secondary storage device contains: • Volume directory (sometimes a root directory for a file system) • External file descriptor for each file • The file contents • Manages blocks • Assigns blocks to files (descriptor keeps track) • Keeps track of available blocks • Maps to/from byte stream Operating Systems: A Modern Perspective, Chapter 13

  18. Disk Organization Boot Sector Volume Directory Blk0 Blk1 … Blkk-1 Track 0, Cylinder 0 … Blkk Blkk+1 Blk2k-1 Track 0, Cylinder 1 … … Blk Blk Blk Track 1, Cylinder 0 … … Blk Blk Blk Track N-1, Cylinder 0 … … Blk Blk Blk Track N-1, Cylinder M-1 Operating Systems: A Modern Perspective, Chapter 13

  19. Low-level File System Architecture Block 0 … … b0 b1 b2 b3 bn-1 . . . Randomly Accessed Device Sequential Device Operating Systems: A Modern Perspective, Chapter 13

  20. File Descriptors • External name • Current state • Sharable • Owner • User • Locks • Protection settings • Length • Time of creation • Time of last modification • Time of last access • Reference count • Storage device details Operating Systems: A Modern Perspective, Chapter 13

  21. An open() Operation • Locate the on-device (external) file descriptor • Extract info needed to read/write file • Authenticate that process can access the file • Create an internal file descriptor in primary memory • Create an entry in a “per process” open file status table • Allocate resources, e.g., buffers, to support file usage Operating Systems: A Modern Perspective, Chapter 13

  22. File Manager Data Structures Keep the state of the process-file session 2 Copy info from external to the open file descriptor 1 Open File Descriptor Process-File Session Return a reference to the data structure 3 External File Descriptor Operating Systems: A Modern Perspective, Chapter 13

  23. Opening a UNIX File fid = open(“fileA”, flags); … read(fid, buffer, len); On-Device File Descriptor 0 stdin 1 stdout 2 stderr 3 ... File structure inode Open File Table Internal File Descriptor Operating Systems: A Modern Perspective, Chapter 13

  24. Block Management • The job of selecting & assigning storage blocks to the file • For a fixed sized file of k blocks • File of length m requires N = m/k blocks • Byte bi is stored in block i/k • Three basic strategies: • Contiguous allocation • Linked lists • Indexed allocation Operating Systems: A Modern Perspective, Chapter 13

  25. Contiguous Allocation • Maps the N blocks into N contiguous blocks on the secondary storage device • Difficult to support dynamic file sizes File descriptor Head position 237 … First block 785 Number of blocks 25 Operating Systems: A Modern Perspective, Chapter 13

  26. Linked Lists • Each block contains a header with • Number of bytes in the block • Pointer to next block • Blocks need not be contiguous • Files can expand and contract • Seeks can be slow First block … Head: 417 ... Length Length Length Byte 0 Byte 0 Byte 0 ... ... ... Byte 4095 Byte 4095 Byte 4095 Block 0 Block 1 Block N-1 Operating Systems: A Modern Perspective, Chapter 13

  27. Length Length Length Indexed Allocation • Extract headers and put them in an index • Simplify seeks • May link indices together (for large files) Byte 0 ... Index block … Head: 417 ... Byte 4095 Block 0 Byte 0 ... Byte 4095 Block 1 Byte 0 ... Byte 4095 Block N-1 Operating Systems: A Modern Perspective, Chapter 13

  28. File Descriptor 43 254 43 … 107 Disk Block Disk Block Disk Block 107 254 File Access Table (FAT) DOS FAT Files File Descriptor 43 254 … 107 Disk Block Disk Block Disk Block Operating Systems: A Modern Perspective, Chapter 13

  29. Data Index Index Index Index Index Index Index Index Index Data Data Data Data Data UNIX Files inode mode owner … Direct block 0 Direct block 1 … Direct block 11 Single indirect Double indirect Triple indirect Data Data Data Operating Systems: A Modern Perspective, Chapter 13

  30. Unallocated Blocks • How should unallocated blocks be managed? • Need a data structure to keep track of them • Linked list • Very large • Hard to manage spatial locality • Block status map (“disk map”) • Bit per block • Easy to identify nearby free blocks • Useful for disk recovery Operating Systems: A Modern Perspective, Chapter 13

  31. Marshalling the Byte Stream • Must read at least one buffer ahead on input • Must write at least one buffer behind on output • Seek  flushing the current buffer and finding the correct one to load into memory • Inserting/deleting bytes in the interior of the stream Operating Systems: A Modern Perspective, Chapter 13

  32. Full Block Buffering • Storage devices use block I/O • Files place an explicit order on the bytes • Therefore, it is possible to predict what is likely to be read after bytei • When file is opened, manager reads as many blocks ahead as feasible • After a block is logically written, it is queued for writing behind, whenever the disk is available • Buffer pool – usually variably sized, depending on virtual memory needs • Interaction with the device manager and memory manager Operating Systems: A Modern Perspective, Chapter 13

  33. Directories • A set of logically associated files and sub directories • File manager provides set of controls: • enumerate • copy • rename • delete • traverse • etc. Operating Systems: A Modern Perspective, Chapter 13

  34. Directory Structures • How should files be organized within directory? • Flat name space • All files appear in a single directory • Hierarchical name space • Directory contains files and subdirectories • Each file/directory appears as an entry in exactly one other directory -- a tree • Popular variant: All directories form a tree, but a file can have multiple parents. Operating Systems: A Modern Perspective, Chapter 13

  35. Directory Implementation • Device Directory • A device can contain a collection of files • Easier to manage if there is a root for every file on the device -- the device root directory • File Directory • Typical implementations have directories implemented as a file with a special format • Entries in a file directory are handles for other files (which can be files or subdirectories) Operating Systems: A Modern Perspective, Chapter 13

  36. / bin usr etc foo / bill nutt FS abc cde xyz blah mount FS at foo UNIX mount Command / bin usr etc foo bill nutt / FS abc cde xyz blah Operating Systems: A Modern Perspective, Chapter 13

  37. VFS-based File Manager Exports OS-specific API File System Independent Part of File Manager Virtual File System Switch MS-DOS Part of File Manager ISO 9660 Part of File Manager … ext2 Part of File Manager Operating Systems: A Modern Perspective, Chapter 13

  38. Rensselaer Polytechnic Institute CSCI-4210 – Operating Systems David Goldschmidt, Ph.D. Operating Systems{week 13}

  39. Hierarchical storage architecture very fast very small volatile managed as part of the filesystem non-volatile very slow very large

  40. File management • A File Management System is a set of OS services that supports files and directories for user applications, OS programs, etc. • Data should be organized andavailable in a convenient andefficient manner • Files are the basicbuilding blocks

  41. Files • A file is an abstraction that represents user data, OS data, an executable, a device, etc. • A file is simply a sequence of bytes • Actual storage location (via network?) andformat are transparent to users • Storage scheme on disk is also transparent • Typically involves cylinders, tracks, sectors, etc.

  42. File attributes • File attributes include: • Human-readable symbolic name • Type (e.g. executable, directory, text, PDF, etc.) • Logical location (i.e. containing directory or path) • Physical location on disk • Size (in bytes) • Protection or security (i.e. permissions) • Timestamps (created, last modified, last accessed)

  43. Pathnames • The pathname (or just path) of a file specifies the sequence of directoriesone must traverse to locatethe file • An absolute path startsat the root node • A relative path startsanywhere

  44. Links • A link provides a shortcut to a file and may circumvent the given directory hierarchy • A hard link in Unix isindistinguishable fromthe original file • A symbolic link in Unixis merely a shortcut • A Windows shortcutis just a symbolic link

  45. Creating and accessing files • File creation requires space allocation • Opening a file returns a handle or file descriptor • Read and write operationsuse the handle and anoffset (or file pointer) • The close operation simplydeletes the handle anddeallocates any memory

  46. Deleting files • Deleting a file deallocates all disk spacemarked as in use by the file • But likely does not erase file contents • Deleted files are recoverable until the disk space is used for (and overwritten by) other file(s) • The delete operation also removes the corresponding entry in the containing directory

  47. Access methods • An access method describes the mannerand mechanisms by which a process accesses the data in a file • Two common access methods: • Sequential access (open, read, write, close) • Random access (open, read, write, seek, close)

  48. Contiguous disk space allocation (i) • In a contiguous disk space allocation scheme, files are allocated to contiguous blocks of disk space

  49. A A A B B C C C D A A A C C C D B B B B A A A C C C B B B B D D A A A C C C B B B B D D Contiguous disk space allocation (ii) • Four files allocated contiguously to disk: • File B outgrows its space and is reallocated: • File D outgrows its space and is reallocated: • Defragmentation combines free disk space:

  50. Clustered disk space allocation (i) • In a clustered disk space allocation scheme, files are allocated to clusters of disk space on an as needed basis

More Related