CS140 Project 4 Due Thursday March 10th

1. CS140 Project 4Due Thursday March 10th Kiyoshi Shikuma Slides adapted from Samir Selman’s

2. Overview • Things to know before you start • Project Requirements: • Buffer Cache • Indexed and Extensible Files • Subdirectories • Synchronization

3. Things to know before you start • May build on top of Project 2 OR 3 • Up to 5% extra credit if built on Project 3 • Open Ended Design • Many ways to achieve functionality but you should think through the design first • Good News: Easier than Project 3 • Bad News: Longest project in terms of lines of code • Read Design Doc and FAQ before you start, will help you guide your design.

4. Project Requirements • (In suggested order of implementation): • Buffer Cache – Caches reads from disk much like paging and virtual memory • Indexed and Extensible Files – Indexed files avoid external fragmentation, Extensible allows files to grow • Subdirectories – Hierarchical namespace along with associated syscalls • Synchronization** – Remove global filesys lock • **Should think about this throughout project • Miscellaneous other things

5. Buffer Cache File Inode Buffer Cache Disk • Suggested that you implement this early • Similar to VM • Keep cache of file blocks in memory • Read/Write checks cache first before going to disk • Cache limited to 64 disk sectors • You can keep a copy of free map in memory, doesn’t count against this limit • Implement an eviction policy that approximates LRU and is at least as good as clock algorithm (may give preference to metadata)

6. Buffer Cache cont’d • Write Behind • Don’t write dirty blocks out to disk immediately • Write to disk when block is evicted • Write all dirty blocks to disk periodically and in filesys_done • This is a good application for timer sleep • Read Ahead • Whenever a block is read in, automatically read in the next block • Do this asynchronously (helper thread)

7. Project 2 (or Project 3 if you are extending) tests should still pass!

8. Indexed and Extensible Files • Current file system is an extent based file system. • The size of file is specified at file creation time. • Continuous sectors allocated on disk for the file. • It suffers from external fragmentation. struct inode_disk { disk_sector_t start; /* First data sector. */ off_t length; /* File size in bytes. */ unsigned magic; /* Magic number. */ uint32_t unused[125]; /* Not used. */ };

9. Indexed and Extensible Files (cont..) IndirectBlock IndirectBlock Double Indirect IndirectBlock • Modify the on-disk inode structure, to remove external fragmentation. • Generally some indexed structure of direct, indirect and double indirect blocks is used.struct inode_disk { • disk_sector_t start; /* First data sector. */ • off_t length; /* File size in bytes. */ • unsigned magic; /* Magic number. */ • uint32_t unused[125]; /* Not used. */ • }; Data Blocks Inode 02/27/09 cs140 Review Session 9

10. Indexed and Extensible Files (cont..) • Size of the on disk inode structure exactly equal to DISK_SECTOR_SIZE. • Size of each block is 512B. • Each block can store 512B/4B = 128 block addresses. • Assume that disk will not be larger than 8MB(minus metadata). • Must support files that are large as the disk. • Don’t keep any upper limit on the number of files which can reside in the FS. 02/27/09 cs140 Review Session 10

11. Indexed and Extensible Files(cont..) • Implement File Growth. • Writing past the EOF should be possible and should extend the file to that position. • Might need to get additional data blocks and update the inode data structure. • A read from a position past the EOF should return zero bytes. • Handle race between a reader reading past EOF and the writer writing past EOF 02/27/09 cs140 Review Session 11

12. File growth tests should pass!

13. Subdirectories • In current FS, all files live in a single directory. • Directories are files, but have a different layout. • Consist of an array of directory entries. • Ensure directories can expand just like any other file. • Extend the directory structure in directory.c so that directory entries can be both files and other directories. “/a/b/” “/” “/a/” File1.txt hello.c a b y.c 02/27/09 cs140 Review Session 13

14. Subdirectories (cont…) • Update existing system calls, to allow relative or absolute path wherever a file name is required. • Each process should have a separate cwd. • May use strtok_r() to parse path names • Implement new system calls • bool chdir (const char *dir) • bool mkdir (const char *dir) • bool readdir (int fd, char *name) • bool isdir (int fd) • int inumber (int fd) 02/27/09 cs140 Review Session 14

15. Subdirectory tests should pass!

16. Synchronization • NEED TO THINK ABOUT THIS DURING ALL PARTS • Possibly the trickiest part of the assignment. • Remove the single file system lock you are currently using. • Multiple readers can read the same file simultaneously. • Multiple writers can write to same file simultaneously. • Their data may be interleaved. 02/27/09 cs140 Review Session 16

17. Synchronization (Cont …) • Extending a file should be atomic. If A and B trying to write past the EOF, only one should be allowed. • Operations on different directories should be concurrent. • Operations on same directory can be serialized. 02/27/09 cs140 Review Session 17

18. Synchronization (Cont …) • Two Writers pointing to EOF and writing 10 bytes concurrently. How to Synchronize? • A Writer extending file past EOF and a reader wanting to read past EOF. How to Synchronize? • Its ok to grab a coarse lock to update couple of variables. But you should not grab a coarse lock and do I/O 02/27/09 cs140 Review Session 18

19. Questions?

20. Good Luck!