1 / 34

CSCI1412 Lecture 11

phones off (please). CSCI1412 Lecture 11. Operating Systems 3 File Systems Dr John Cowell. Overview. Low level file systems partitions, sectors, clusters formatting FAT’s what is a FAT and how a file is stored fragmentation and defragmentation how a file is located

willow
Download Presentation

CSCI1412 Lecture 11

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. phones off(please) CSCI1412Lecture 11 Operating Systems 3File Systems Dr John Cowell CSCI1412-OS-3

  2. Overview • Low level file systems • partitions, sectors, clusters • formatting • FAT’s • what is a FAT and how a file is stored • fragmentation and defragmentation • how a file is located • High level file systems • drives, directories, files • file attributes and permissions • sharing information CSCI1412-OS-3

  3. Low-level File Systems CSCI1412-OS-3

  4. Low-level File Management • Physically, a hard disk consists of platters, surfaces, cylinders, heads and sectors • Logically, a hard disk consists of partitions or drives (C:, D:, E:, etc.), directories and files • It is the operating system that carries out the translation between the underlying physical organisation and representation of data and the logical layer visible to the user • different partitions (drives) may even have different translations that are entirely transparent to the user CSCI1412-OS-3

  5. logical file system files, directories, etc. BIOS Basic Input Output System IDE controller integrated drive electronics (hard disk drive) controller hard disk File System Layers • In order to provide a convenient and efficient translation, an operating system generally implements a number of intermediate layers • to insulate logical structure from device dependencies application programs CSCI1412-OS-3

  6. Partitions • A partition is a collection of adjacent cylinders on the hard drive that are grouped together • each partition can have its own operating system and so its own logical file system • the partitions are defined (position, size) in a table • there are only four entries allowed in the partition table • MS-DOS/Win uses a logical structure referred to as FATthis is specific to MS-DOS/Win and is not generic • used for floppy drives and flash memory • other operating systems have their own file systems • XP, Vista: NTFS; Linux: ext2 • these aren’t compatible: partition may be unreadable CSCI1412-OS-3

  7. Formatting • An analogy • imagine a large area of tarmac • it would be chaos if drivers tried to park anywhere! • now imagine that ‘spaces’ are painted with white lines • the area of tarmac has now become a car park • Formatting is the process of laying down the file system structures on the partition • each partition needs to be formatted separately • two file allocation tables are written • one root directory entry is written • all sectors are tested to see if they are ‘bad’ & marked CSCI1412-OS-3

  8. Surface Organisation • The very first sector on a hard drive is the primary boot sector which also contains the partition table CSCI1412-OS-3

  9. Sectors and Clusters • Remember that the surface of the disk is divided into blocks of bytes called sectors • on current PC’s 1 sector = 512 bytes • Since hard disks have become so much larger there are so many sectors that is inefficient to deal with sectors individually • sectors are grouped into blocks, known as clusters • A file is stored in a whole number of clusters • any space left over in the final cluster is wasted • sometimes called ‘slack space’ CSCI1412-OS-3

  10. file name ext reserved time date start size 8 3 1 10 2 2 2 4 attribute flags Directory Entries • After the two FAT’s on a partition, the operating system puts the root directory (\) • a directory consists of a number of directory entries • Each directory entry consists of • the file name, extension, other information and the cluster number of the initial data stored in the file • The directory entry holds the cluster number of the first cluster in which the file is stored CSCI1412-OS-3

  11. File Allocation Tables • The file allocation table (FAT) is a database that holds the status of every cluster on the partition • because it is so important two copies of FAT are stored • if it is corrupted all data on the partition can be lost! • The directory entry holds the start cluster of a file • the FAT entry indexed by that cluster number contains the number of the next cluster storing the file’s data • there is a special value to signify end of file (FFF816) • Special values signify free blocks and bad blocks • free blocks are signified by 000016 • bad blocks are signified by FFF716 CSCI1412-OS-3

  12. FAT root directory name start command.com 2 autoexec.bat 3 config.sys 4 fred.doc 8 bill.doc 9 jack.doc 10 0 0xFFFD 16 17 0 1 0xFFF8 17 24 0 2 0xFFF8 0 18 0 22 3 0 0xFFF8 19 0 4 0xFFF8 0 20 0 0 1 2 3 5 0 21 0 0xFFF8 6 0 4 5 6 22 0 21 7 7 0xFFF7 23 0 8 9 10 11 8 0 12 24 25 0 12 13 14 15 9 16 0 25 0 0xFFF8 10 0 26 26 0 27 16 17 18 19 11 0 27 14 0 20 21 22 23 12 0 0xFFF8 28 0 13 0 18 24 25 26 29 0 27 14 13 0 30 0 28 29 30 31 15 0 31 0 FAT Illustration • Suppose the clusters are 4K in size and we want to store a file, “\FRED.DOC”, which is 6 Kb long CSCI1412-OS-3

  13. cluster contents 0 FAT-1 FAT-2 FAT root directory 1 root directory (\) name start command.com 2 autoexec.bat 3 config.sys 4 fred.doc 8 bill.doc 9 jack.doc 10 0 0xFFFD 16 0 17 2 unused 1 0xFFF8 17 0 24 2 0xFFF8 0 18 0 22 3 unused 3 0 0xFFF8 19 0 4 unused 4 0xFFF8 0 20 0 5 free 5 0 21 0xFFF8 0 6 0 22 0 21 6 free 7 0xFFF7 23 0 7 bad 8 0 12 24 0 25 8 FRED.DOC (0..4095) 9 16 0 25 0xFFF8 0 10 0 26 26 0 27 9 BILL.DOC (0..4095) 11 0 27 0 14 10 JACK.DOC (0..4095) 12 0xFFF8 0 28 0 11 free 13 0 18 29 0 14 0 13 30 0 12 FRED.DOC (4095..6143) 15 0 31 0 Cluster Contents • More details on the cluster contents CSCI1412-OS-3

  14. Windows 9X File System • Windows 9X uses a minor variation of the MS-DOS FAT file system structure, called VFAT (which developed into FAT32) • it is still a linked-list allocation using a FAT index • Directory entries have been expanded to allow file names of up to 256 characters • The FAT table can be selected as 16 bit or 32 bit • 16-bit FAT gives 65536 entries • the file system is compatible with MS-DOS • 64K entries  32K clusters  2Gb maximum partition size • 32-bit FAT gives 4  109 entries • extremely large maximum partition size! CSCI1412-OS-3

  15. NTFS • The (V)FAT structure has serious shortcomings • large disk sizes are only possible using large clusters • the FAT offers poor performance and is non-robust (key file data such as time, date, size are kept in the directory entry) • there is no file access protection mechanism • Windows XP and Vista use an entirely different system • full access protection security • large capacity allocation scheme • internationalisation using long Unicode file names • robust fault-tolerance with transaction logging CSCI1412-OS-3

  16. NTFS • NT File System was developed for Windows NT • Implemented as the file system used by Windows 2000 and XP • Uses 64-bit addresses • theoretically supporting disk partitions of up to 264 bytes capacity • Supports 255 character case sensitive file names in Unicode • but the case sensitivity is not supported by the Win32 API! • Paths are limited to 32767 bytes CSCI1412-OS-3

  17. File System Structure • NTFS partition is organised as linear sequence of blocks • block size can be 512 bytes to 64 Kb • actual block size is a compromise between large blocks (more efficient data transfer) and small blocks (less file fragmentation) • Most important part of the file structure is the Master File Table (MFT) • each row (record) is 1 Kb long • Each record contains description of a single file or directory, including • the file’s attributes • the addresses of the data blocks actually containing the file’s data • Data block address list may require more than one record in the MFT if the file is especially large • in that case, the base record contains the addresses of the overflow MFT records • Size of the MFT is also flexible • may grow to a maximum of 248 records CSCI1412-OS-3

  18. Master File Table • Any file system needs to store information about itself, such as blocks used, volume name, etc • This metadata is stored in a series of files • file names start with a ‘$’ • NTFS uses first 16 records for this • The address of the MFT itself is stored in the boot block of the disk partition when the partition is created • allows the MFT to be placed anywhere on the disk • to avoid bad blocks CSCI1412-OS-3

  19. MFT Records • Each MFT record consists of a series of paired values • Each pair of values comprises attribute header, value • several different types of these such as file name, attribute list, and data • Value(s) associated with the data attribute contain either addresses of data blocks, or for a small file, the data itself • an immediate file CSCI1412-OS-3

  20. How NTFS Works • The OS will attempt always to place data into contiguous blocks • but this is not always possible • The diagram details a file of nine blocks in size • This example shows the data blocks grouped into three runs of blocks • This is a short file, ie the data blocks will not fit into the MFT record, but all the information about the file will • The header shows that the offset of the first block is 0 • ie, how far from the start of the file is the first data block • The second header value gives the number of the first block that is not part of the file • effectively, the length of the file in blocks • After the header come the pairs of values pointing directly to the used blocks • first value gives the block number • second value gives the number of contiguous blocks CSCI1412-OS-3

  21. Integrity Checking • If the computer crashes or is switched off in the middle of updating the file system(s) it can be left in a state of confusion or ‘corruption’ • the two FAT’s may not contain the same information • the file clusters may not reflect the FAT list • the initial directory entry may not point to the file • Some operating systems (e.g. UNIX) automatically check the file system integrity on start-up • MS-DOS/Win requires you to run SCANDISK • NTFS uses journaling • logs file system updates that are about to happen • checks afterwards to see that they did happen • undoes any partially completed updates CSCI1412-OS-3

  22. FRED.DOC BILL.DOC JACK.DOC FREE • Suppose BILL.DOC is deleted and the clusters freed FRED.DOC FREE JACK.DOC FREE • Then a longer file, JOHN.DOC, is stored on disk FRED.DOC JOHN.DOC JACK.DOC JOHN.DOC Fragmentation • Initially all the free space is in one big block • clusters are allocated in sequence to files as needed • the file JOHN.DOC is said to be fragmented • after lots of deletions / additions a disk can be very fragmented • a badly fragmented disk will be much slower to read and write CSCI1412-OS-3

  23. 0 1 2 3 0 1 2 3 4 5 6 7 4 5 6 7 8 9 10 11 8 9 10 11 12 13 14 15 12 13 14 15 16 17 18 19 16 17 18 19 20 21 22 23 20 21 22 23 24 25 26 27 24 25 26 27 28 29 30 31 28 29 30 31 Defragmentation • Defragmentation is the name given to the process of rearranging the clusters so that all files are held in adjacent clusters and all free space is at the end • the whole disk is read and the clusters are moved • a defragmented disk will improve performance • this process can be very time consuming • a defragmenting tool is included in later versions of Windows CSCI1412-OS-3

  24. Finding Files • When the system request a file from disk • directory is read to see if file exists • directory entry has address of first cluster of file • disk drive heads are moved to correct track • OS waits for correct sector to arrive under disk head • sector is read & data transferred to disk cache buffer • when buffer full, contents transferred to RAM • next sector is read, until all sectors in cluster have been accessed • FAT re-accessed to find next cluster • process repeated until end-of-file is found in FAT CSCI1412-OS-3

  25. High-level File Systems CSCI1412-OS-3

  26. High Level File Systems • In addition to dealing with low-level issues such as partition tables, FAT’s, clusters and sectors the operating system deals with higher-level issues of file management • directory structures • file attributes • security • Most operating systems also have some concept of file types or special files • directories are identified by attribute in directory entry • other file types in Windows are identified by extension • e.g. EXE, COM, BAT, DOC, XLS, TTF, DLL CSCI1412-OS-3

  27. Directories and Folders • Directories, or folders as they are also known, are used to organise collections of files • in a typical modern computer, even single-user PC, there can be 100,000’s of files • if they were kept in a single, flat, file structure it would be extremely time consuming to find any particular file • using directories users can group together files • applications, system files, user files • documents, spreadsheet, presentations • A directory is a special file type in which the OS stores the names of other files or other directories • directories within directories are sub-directories CSCI1412-OS-3

  28. Typical Single User File System • Note: the separation of • user files • applications • operating system • Allows • backups of work • easy installation, reinstallation and removal of applications • operating system upgrades CSCI1412-OS-3

  29. Drives • In Windows systems different physical devices and file systems are assigned different drive labels • A: Floppy disk • C: First hard disk (partition) • D: Second hard disk (partition) • E: CD-ROM • this organisation is visible to the user and has big drawbacks if the physical organisation needs alteration • In Unix there is one logical file system structure which may consist of many physical devices • the devices may even be distributed across a network CSCI1412-OS-3

  30. File Attributes • The operating system keeps track of certain file properties or attributes • date / time of creation & last modification • in Windows • R: read-only, H: hidden, S: system • A: archive (has the file changed since last backup) • in Unix • file owner, group; file permissions • In MS-DOS/Win the file extension also has meaning to the operating system • EXE: executable, BAT: batch command script CSCI1412-OS-3

  31. File Permissions • In both Windows XP/Vista and UNIX the concept of MS-DOS/Win file attributes has been extended to a more comprehensive implementation of file permissions • the directory entry for each file stores the user name of its owner (the creator) and their default group • there are three sets of file permissions flags user (owner) read (r), write (w) and execute (x) permissionsgroup read (r), write (w) and execute (x) permissionsother read (r), write (w) and execute (x) permissions CSCI1412-OS-3

  32. Personal & Shared Information - 1 • Using these file permissions it is possible to restrict other users’ access to your files • the user flags apply to the owner of the file • e.g. if there is no user write permission, not even the owner can write to the file (i.e. the file is read-only) • the group flags apply to other users who belong to the same group as the group of the file • e.g. other members of group staff may be granted permission to read documents belonging to jcowell(staff) • jcowellcould write to a file owned by group cciwith group write permissions (as jcowellis also a member of group cci) CSCI1412-OS-3

  33. Personal & Shared Information - 2 • Using these file permissions it is possible to restrict other users’ access to your files, cont. • the other flags apply to all other users who are not even in the same group • if others have read & write access, then any other user can read or write to the files • In UNIX the (x) flags indicate either • an ordinary file is executable (like .EXE in MS-DOS) • a directory is searchable by the appropriate users • e.g. if jcowellmakes a directory rwx,r-x,--- then group staff members can read it and enter into it but other users cannot CSCI1412-OS-3

  34. Summary • Low level file systems • partitions, sectors, clusters • formatting • FAT’s • what is a FAT and how a file is stored • fragmentation and defragmentation • how a file is located • High level file systems • drives, directories, files • file attributes and permissions • sharing information CSCI1412-OS-3

More Related