250 likes | 491 Views
Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems. Emery Berger University of Massachusetts Amherst. Advanced File Systems. Motivation & Brief review Journaling Log-structured file system. New Challenges. Servers have special needs! Large hard-disk partitions
E N D
Operating SystemsCMPSCI 377Lecture 23: Advanced File Systems Emery Berger University of Massachusetts Amherst
Advanced File Systems • Motivation & Brief review • Journaling • Log-structured file system
New Challenges • Servers have special needs! • Large hard-disk partitions • Quick crash recovery • High-performance I/O • Storing thousands of files, TB of data • None supported well by ext2
Review: Blocks & Fragmentation • Logical block – smallest unit of storage that can be allocated by file system • Internal fragmentation – occurs when file does not fill block completely • Example: file = 10K, block = 8K: wastes 6K • External fragmentation – occurs when logical blocks that make up file are scattered over the disk • Causes poor performance
Review: Organization • Extent – contiguous blocks: • Triple: (file offset, starting block number, length) • file offset – offset of extent's first block from file start • starting block number – first block in extent • length –number of blocks in extent • Meta data – file system's internal data structures • Time stamps, ownership, size & locations on disk, etc.
Extents • Triple:(offset, start, length) • Contiguous blocks
Review: Inodes • inode – stores information about file • e.g., permissions, types, # of links to file • pointers to file data blocks (direct pointers) • pointers to direct pointers (indirect pointers) • each has unique inode number • directory = special file: contains pointers to other files
Metadata Updates • Lots of meta data being updated • Scattered on disk = slow, and non-atomic • Example: Creating a new file • Modifies inodes, free lists, directory entries • What happens if interrupted? • e.g., power outage, crash • Interrupted meta-data write) file system in inconsistent state
Repairing File System Inconsistency • UNIX solution: fsck =“file-system check” • Analogous utilities for Windows, etc. • Detect and repair structural integrity problems • Scan file system’s entire meta data • Looks for inconsistencies & fixes them • Problems? • S…l…o…w… • Might not succeed
Journaling File Systems • Solve problem by maintaining journal • Log all transactions to disk • Updates to the disk committed atomically • Example: Creating a new file • Log modifications to inodes • Log changes to free lists, directory entries • Now begin update • When finished, write “committed” into journal
Recovery in JFS • Power outage: • Some updates fully committed to file system • No problem • Not yet fully committed • File system reads journal, replays transaction • Advantages over fsck: • Much quicker than scan of disk • Guarantees file system always consistent • But: does not guarantee zero data loss
Advanced File Systems • Brief review • Journaling • Log-structured file system
LFS: Beyond JFS • Insight:While logging updates, why not logthe data too? • “Log-structured file system” • Preserves data integrity • Provides improved performance, too!
Sprite LFS[Rosenblum & Ousterhout] • Outperforms then-current Unix file systems by an order of magnitude for small-file writes • Matches or exceeds Unix performance for reads and large writes • Even with overhead • Can use 70% of disk bandwidth for writing • Unix file systems typically can use only 5-10%
Technology Trends • Disk technology improving rapidly • But: capacity rather than performance • Increasing RAM ) cache more effective • Disk traffic dominated by writes • Logs: write new information sequentially • Increase write performance, eliminating seeks
Existing File Systems • Exercise worst-case given trends! • Information spread around disk • Many small accesses • Synchronous writes • Application performance can be bottlenecked by disk performance
Log-Structured File Systems • Approach: improve write performance by combining into single sequential writes • Converts small random writes into large sequential transfers • Use file cache as write buffer • But where does meta data go?
Structure of LFS (1/3) • Some index structures used to retrieve information • Inodes not fixed: inode map used to maintain location of each inode • Fixed checkpoint region on each disk: identifies locations of inode map blocks
dir1 dir2 Log Disk Sprite LFS file1 file2 file1 file2 Disk Unix FFS dir1 dir2 Inode Directory Data Inode map Structure of LFS (2/3) • Disk layout (compared with Unix FFS)
Structure of LFS (3/3) • Segments • Important to maintain large free extents for writing new data • Divide disk into large fixed-size extents called segments • Segments always written sequentially • Before segment is rewritten,all live data must be copied out (“cleaned”)
Crash Recovery (1/2) • Checkpoints • Position in log at which all of file system structures are consistent and complete • Checkpoint region written on special fixed position: • Periodic intervals • When the file system unmounted or shut down
Crash Recovery (2/2) • Roll-forward • After crashes, LFS scans through log segments that were written after last checkpoint • Use information in segment summary blocks • When summary block indicates presence of new inode, LFS updates inode map
LFS Wrap-Up • Extends journaling to data • Sequential writes in segments • Fast! • Periodic clean-up • Periodic checkpoints
Summary • Traditional filesystems – integrity problems • Solved by journaling • Log-structured further improves speed by optimizing write performance