370 likes | 512 Views
Files. CS 470 - Spring 2002. Overview. Example: FAT File System File Organization File System Organization File Directories and File Sharing Record Blocking Disk space allocation Example: Unix File Systems. Fat Disk Structure. Boot Sector. Reserved Sectors. Other Reserved.
E N D
Files CS 470 - Spring 2002
Overview • Example: FAT File System • File Organization • File System Organization • File Directories and File Sharing • Record Blocking • Disk space allocation • Example: Unix File Systems
Fat Disk Structure Boot Sector Reserved Sectors Other Reserved Allocation chains and Free List File Allocation Table (FAT) File Allocation Table Copy Redundant Copy Root Directory Fixed size Array of root directory entries File and Directory Area All other directories and files are located here
FAT Boot Sector • Location: First sector of the drive • Contains: • Jump to first level boot code (byte offset 0) • Disk Parameter Block (byte offset 11) which describes the drive • Other identifying information like the volume label (byte offset 43) • First level boot code (typically at offset 0x40)
Geometry BytesPerSector[2] SectorsPerTrack[2] Heads[2] Sectors[2] or LargeSectors[4] Media HiddenSectors[4] Structure Sizes ReservedSectors[2] NumberOfFats SectorsPerFat[2] NbrOfRootEntries[2] SectorsPerCluster Disk Parameter Block Field sizes in bytes are indicated in square brackets
Typedef struct _DIRENT { UCHAR FileName[11]; UCHAR Attributes; UCHAR NtByte; UCHAR CreationCSec; FAT_TIME_STAMP CreationTime; FAT_DATE LastAccessDate; USHORT ExtendedAttributes; FAT_TIME_STAMP LastWriteTime; USHORT FirstClusterOfFile; ULONG FileSize; } DIRENT; FAT Directory Entries
File Allocation Table FAT12 Directory Entry 0xfff 14 8dot3 Name 18 . . . 0 16 First Cluster (15) 0xff7 17 File Size (5a0) 1a 18 0 Dir Entry Bad Cluster 19 FOO BAR 0xfff Free Cluster 1a . . . 1b 14 32f
Fat_Offset to SectorOffset • How to translate FAT tables entries to sector offsets in the disk partition • RootDirSize = NbrRootEntries * sizeof(DIRENT) / SectorSize • FileAreaOffset = NbrReservedSectors + NbrFat*SectorsPerFat + RootDirSize • SectorOffset = FileAreaOffset + (FAT_Offset - 2) * SectorsPerCluster
Long Filename Directory Entry typedef struct _LFN_DIRENT { UCHAR Ordinal; UCHAR Name1[10]; // 5 Unicode chars UCHAR Attributes; // Should be 0x0f UCHAR Type; UCHAR CheckSum; // of 8dot3 name WCHAR Name2[6]; USHORT MustBeZero; WCHAR Name3[2]; } LFN_DIRENT;
DIRENT with LFN Ordinal 0x44 Up to 13 Unicode Chars 3 Characters 26 to 38 All have same Checksum Field 2 Characters 13 to 25 1 First 13 Unicode Chars 8dot3 Name Calculate rotated checksum from this
File Structure CS 470 - Spring 2002 Segment II
File Organization • Vocabulary • Field - basic element of data • Record - collection of related fields • File - collection of similar records • File Organization Choice Criteria • Rapid access • Ease of updates • Economy of storage • Reliability
Some File Types • Pile • Sequential • Indexed Sequential • Indexed • Direct or Hashed
Pile • Variable length records with variable fields • Fields must be self describing • No order to records - exhaustive search Record 1 F1 F2 F3 F5 Record 2 F1 F3 F4 Hdr F4 Data
Sequential File • Fixed length records of fixed format fixed length fields. Distinguished key field. • Records sorted in key field order • Search is typically sequential • Insertions are not easy - so usually accumulated in separate overflow file with periodic update of the master file Record 1 14 F2 F3 F4 F5 Key Record 2 17 F2 F3 F4 F5 24 F2 F3 F4 F5 Record 3 26 F2 F3 F4 F5 Record 4
Indexed Sequential Files • Sequential file with index file and overflow file for updates • Records still in order of key index • Overflow file also accessible through index - Records have Next_Record field which can point into overflow file. • Multiple level so indexing can speed lookups
Indexed Sequential File Key Index 5 Master File 5 9 18 11 35 18 42 35 42 13 Overflow File 38 15
Indexed Files • Index for each searchable field, but each record must have at least one searchable field • Records not sorted, no overflow needed • Allows for variable length fields and records with missing fields • Exhaustive index - one entry per record, versus Partial index - on entry per record containing the indexed field.
Indexed File Indexed File Partial Index Exhaustive Index
Hashed Files • Hashed access with overflow tables to handle collisions • Usually used with fixed length records • Very fast access • No way to sequentially access all records
Hashed File Hashed File 54 Overflow Table 37 8 24 37 87 87 91 23 23 231 43 91 19 14 24 83 14 Pointers calculated via hashing function
File Systems CS 470 - Spring 2002 Segment III
Directories • Contents • Name • File type and organization • Address: Size, location • Access control information • Usage Information - times, current usage • Structure - e.g. directory tree • Operations: Search, Create file, Delete file, and List Directory
File Sharing • Access Rights - Allow various users or groups to share files • Simultaneous Access control - Allow multiple threads to share the same file at the same time
Record Blocking • Three approaches • Fixed Blocking • Variable length records with records restricted to a single block - no spanning • Variable length records which can overlap (span) two blocks • First two may have internal fragmentation • Third is hard to update
Record Blocking Methods Block 1 Rec 1 Rec 2 Rec 3 Rec 4 Rec 5 Rec 6 Block 2 Fragment Block 1 Rec 1 Rec 2 Rec 3 Rec 4 Rec 5 Block 2 Spanning Records Block 1 Rec 1 Rec 2 Rec 3 Rec 4 Rec 4 Rec 5 Rec 6 Rec 6 R7 Block 2
Disk Space Allocation (1 of 2) • Recorded in File Allocation Table • Pre-allocation (at file creation) versus dynamic allocation of space • Allocation size - variable or fixed, large or small, contiguous or not • Allocation Methods • Contiguous - just need address and total size • Chained - pointer to next block • Indexed - index refers to file portions
Allocation Methods Contiguous 3 5 0 1 2 U U 0 1 2 Chained 2 0 1 Indexed 1 2 3 3 4 5
Disk Space Allocation (2 of 2) • Free Space Management (Disk Allocation Table) • Bit maps of Free Blocks • Chained Free Portions • Index of Free Portions • Reliability Problems • Caused by improper shutdown when File and Disk allocation tables kept in memory • Locking is often too expensive
Free List Management Free Allocated Bit Map 0 1 1 1 0 0 1 0 1 1 1 1 14 205 42 Chained Free 2 51 52 Indexed 1 32 3 432 433 434
Unix File System CS 470 - Spring 2002 Segment IV
Classic Unix File System Disk Drive Main Memory Superblock Superblock Inode Table Free Inode List Files and Directories Free Block List File Table Inode Table Buffer Cache
V6 Unix Inode Entries • i_mode - type (dir, char or block special, or ordinary) and file permissions • i_nlink - number of names (hard links) • i_uid, i_gid - user and group owners • i_size0, i_size1 - size of file in bytes • i_ addr[8] - location of file on disk • i_atime, i_mtime - access and modification times
Classic V7 Unix Inode File location part of inode Inode Rest of inode 10 Direct Block Nbrs Block number of Triple Indirect Block 256 More Block Nbrs Single Indirect Block Double Indirect Block 256 More Indirect Block Nbrs 256 Block Nbrs 255 More of these
Unix File Allocation Limits Level Nbr of Blks Nbr of Bytes Direct 10 10 KB Single Indirect 256 256 KB Double Indirect 256 x 256 64 MB = 64 K Triple Indirect 256 x 64 K 16 GB = 16 M
Classic Unix Free Lists Superblock Free Block List 255 Free Block Nbrs One per disk Partition 255 Free Block Nbrs One per disk Partition Superblock Free Disk Inode List
Summary • File Organization • File System Organization • File Directories • File Sharing • Record Blocking • Disk space allocation • Examples: Unix and FAT File Systems