CS140 Review Session Project 4 – File Systems

1. CS140 Review SessionProject 4 – File Systems Samir Selman Due Thursday March,11 02/27/09 cs140 Review Session

2. General Points • May build project 4 on top of Project 2 or Project 3 • Up to 5% Extra Credit if built on top of Project 3 • Good News: “Easier than Project 3” • Not so good news: Probably the biggest assignment in terms of code lines. • Read the Design Document before starting. It will help you design your project better. • Open ended design • Like project 3 you have to figure out the design to get the required functionality. 02/27/09 cs140 Review Session 2

3. Requirements • Indexed and Extensible Files • Subdirectories • Buffer Cache • Synchronization at a finer level 02/27/09 cs140 Review Session 3

4. Indexed and Extensible Files • Current file system is an extent based file system. • The size of file is specified at file creation time. • Continuous sectors allocated on disk for the file. • It suffers from external fragmentation. struct inode_disk { disk_sector_t start; /* First data sector. */ off_t length; /* File size in bytes. */ unsigned magic; /* Magic number. */ uint32_t unused[125]; /* Not used. */ }; 02/27/09 cs140 Review Session 4

5. Indexed and Extensible Files (cont..) IndirectBlock IndirectBlock Double Indirect IndirectBlock • Modify the on-disk inode structure, to remove external fragmentation. • Generally some indexed structure of direct, indirect and double indirect blocks is used.struct inode_disk { disk_sector_t start; /* First data sector. */ off_t length; /* File size in bytes. */ unsigned magic; /* Magic number. */ uint32_t unused[125]; /* Not used. */ }; Data Blocks Inode 02/27/09 cs140 Review Session 5

6. Indexed and Extensible Files (cont..) • Size of the on disk inode structure exactly equal to DISK_SECTOR_SIZE. • Size of each block is 512B. • Each block can store 512B/4B = 128 block addresses. • Assume that disk will not be larger than 8MB(minus metadata). • Must support files are large as the disk. • Don’t keep any upper limit on the number of files which can reside in the FS. 02/27/09 cs140 Review Session 6

7. Indexed and Extensible Files(cont..) • Implement File Growth. • Writing past the EOF should be possible and should extend the file to that position. • Might need to get additional data blocks and update the inode data structure. • A read from a position past the EOF should return zero bytes. • Handle race between a reader reading past EOF and the writer writing past EOF 02/27/09 cs140 Review Session 7

8. Subdirectories • In current FS, all files live in a single directory. • Directories are files, but have a different layout. • Consist of an array of directory entries. • Ensure directories can expand just like any other file. • Extend the directory structure in directory.c so that directory entries can be both files and other directories. “/a/b/” “/” “/a/” File1.txt hello.c a b y.c 02/27/09 cs140 Review Session 8

9. Subdirectories (cont…) • Update existing system calls, to allow relative or absolute path wherever a file name is required. • Each process should have a separate cwd. • May use strtok_r() to parse path names • Implement new system calls • bool chdir (const char *dir) • bool mkdir (const char *dir) • bool readdir (int fd, char *name) • … 02/27/09 cs140 Review Session 9

10. Buffer Cache File Inode Buffer Cache Disk • Integrate buffer cache early into your design. • Similar to VM concept • Keep a cache of file blocks in main memory. • Read/Write call must first check the cache and then go to disk if not present in cache. • Cache is limited to 64 sectors in size. • Implement a cache replacement policy as good as “clock” algorithm ( may give preference to metadata). • Allowed to keep a copy of the free map in memory (doesn’t count against cache usage) 02/27/09 cs140 Review Session 10

11. Buffer Cache (cont…) • Write-behind policy: • Don’t write the dirty block immediately to disk. • Flush the dirty blocks when they are evicted. • Flush the entire cache at a periodical interval and also in filesys_done(). • Can use timer_sleep from Project 1 for periodic flushing. • Read ahead policy: • When one block is read, automatically read the next block from disk. • Should be done asynchronously (can use a helper thread). 02/27/09 cs140 Review Session 11

12. Synchronization • Possibly the trickiest part of the assignment. • Remove the single file system lock you are currently using. • Multiple readers can read the same file simultaneously. • Multiple writers can write to same file simultaneously. • Their data may be interleaved. 02/27/09 cs140 Review Session 12

13. Synchronization (Cont …) • Extending a file should be atomic. If A and B trying to write past the EOF, only one should be allowed. • Operations on difference directories should be concurrent. • Operations on same directory can be serialized. 02/27/09 cs140 Review Session 13

14. Synchronization (Cont …) • Two Writers pointing to EOF and writing 10 bytes concurrently. How to Synchronize? • A Writer extending file past EOF and a reader wanting to read past EOF. How to Synchronize? • Its ok to grab a coarse lock to update couple of variables. But you should not grab a coarse lock and do I/O 02/27/09 cs140 Review Session 14

15. Suggested Order of Implementation • Add buffer cache to existing file system • All tests from projects 2 and 3 should still pass • Implement extensible files • After this, you should pass file growth tests • Implement subdirectories • After this, you should pass subdirectory tests • Everything else * Remember to think about synchronization during all 4 steps 02/27/09 cs140 Review Session 15