Operating Systems CMPSCI 377 Lecture 23: Advanced File Systems

1. Operating SystemsCMPSCI 377Lecture 23: Advanced File Systems Emery Berger University of Massachusetts Amherst

2. Advanced File Systems • Motivation & Brief review • Journaling • Log-structured file system

3. New Challenges • Servers have special needs! • Large hard-disk partitions • Quick crash recovery • High-performance I/O • Storing thousands of files, TB of data • None supported well by ext2

4. Review: Blocks & Fragmentation • Logical block – smallest unit of storage that can be allocated by file system • Internal fragmentation – occurs when file does not fill block completely • Example: file = 10K, block = 8K: wastes 6K • External fragmentation – occurs when logical blocks that make up file are scattered over the disk • Causes poor performance

5. Review: Organization • Extent – contiguous blocks: • Triple: (file offset, starting block number, length) • file offset – offset of extent's first block from file start • starting block number – first block in extent • length –number of blocks in extent • Meta data – file system's internal data structures • Time stamps, ownership, size & locations on disk, etc.

6. Extents • Triple:(offset, start, length) • Contiguous blocks

7. Review: Inodes • inode – stores information about file • e.g., permissions, types, # of links to file • pointers to file data blocks (direct pointers) • pointers to direct pointers (indirect pointers) • each has unique inode number • directory = special file: contains pointers to other files

9. Metadata Updates • Lots of meta data being updated • Scattered on disk = slow, and non-atomic • Example: Creating a new file • Modifies inodes, free lists, directory entries • What happens if interrupted? • e.g., power outage, crash • Interrupted meta-data write) file system in inconsistent state

10. Repairing File System Inconsistency • UNIX solution: fsck =“file-system check” • Analogous utilities for Windows, etc. • Detect and repair structural integrity problems • Scan file system’s entire meta data • Looks for inconsistencies & fixes them • Problems? • S…l…o…w… • Might not succeed

11. Journaling File Systems • Solve problem by maintaining journal • Log all transactions to disk • Updates to the disk committed atomically • Example: Creating a new file • Log modifications to inodes • Log changes to free lists, directory entries • Now begin update • When finished, write “committed” into journal

12. Recovery in JFS • Power outage: • Some updates fully committed to file system • No problem • Not yet fully committed • File system reads journal, replays transaction • Advantages over fsck: • Much quicker than scan of disk • Guarantees file system always consistent • But: does not guarantee zero data loss

13. Advanced File Systems • Brief review • Journaling • Log-structured file system

14. LFS: Beyond JFS • Insight:While logging updates, why not logthe data too? • “Log-structured file system” • Preserves data integrity • Provides improved performance, too!

15. Sprite LFS[Rosenblum & Ousterhout] • Outperforms then-current Unix file systems by an order of magnitude for small-file writes • Matches or exceeds Unix performance for reads and large writes • Even with overhead • Can use 70% of disk bandwidth for writing • Unix file systems typically can use only 5-10%

16. Technology Trends • Disk technology improving rapidly • But: capacity rather than performance • Increasing RAM ) cache more effective • Disk traffic dominated by writes • Logs: write new information sequentially • Increase write performance, eliminating seeks

17. Existing File Systems • Exercise worst-case given trends! • Information spread around disk • Many small accesses • Synchronous writes • Application performance can be bottlenecked by disk performance

18. Log-Structured File Systems • Approach: improve write performance by combining into single sequential writes • Converts small random writes into large sequential transfers • Use file cache as write buffer • But where does meta data go?

19. Structure of LFS (1/3) • Some index structures used to retrieve information • Inodes not fixed: inode map used to maintain location of each inode • Fixed checkpoint region on each disk: identifies locations of inode map blocks

20. dir1 dir2 Log Disk Sprite LFS file1 file2 file1 file2 Disk Unix FFS dir1 dir2 Inode Directory Data Inode map Structure of LFS (2/3) • Disk layout (compared with Unix FFS)

21. Structure of LFS (3/3) • Segments • Important to maintain large free extents for writing new data • Divide disk into large fixed-size extents called segments • Segments always written sequentially • Before segment is rewritten,all live data must be copied out (“cleaned”)

22. Crash Recovery (1/2) • Checkpoints • Position in log at which all of file system structures are consistent and complete • Checkpoint region written on special fixed position: • Periodic intervals • When the file system unmounted or shut down

23. Crash Recovery (2/2) • Roll-forward • After crashes, LFS scans through log segments that were written after last checkpoint • Use information in segment summary blocks • When summary block indicates presence of new inode, LFS updates inode map

24. LFS Wrap-Up • Extends journaling to data • Sequential writes in segments • Fast! • Periodic clean-up • Periodic checkpoints

25. Summary • Traditional filesystems – integrity problems • Solved by journaling • Log-structured further improves speed by optimizing write performance