Trends In Distributed File Systems

1. Section 5.3 Trends In Distributed File Systems Professor: Dr. Zhang CSc 8320 Advanced Operating Systems Presented by: Tu Tran

2. Outline • New Hardware • Scalability • Wide Area Networking • Mobile Users • Fault Tolerance • Multimedia • Interesting Research Papers/Projects

3. New Hardware • Costs continue to drop at an amazing rate • Rapidly dropping memory costs make it possible to have every larger databases in main memory  Servers could have entire file system in main memory • Advantage: • Gain in performance • Simplify file system structure

4. New Hardware (cont’d) • Simpler to store each file contiguously in memory (UNIX – tree, MS-DOS – link listed) • Disadvantage: • Power fails, all files stored are lost. However, current technology allows to back up continuously files on tapes or other storage media. • RAM disk and file system

5. New Hardware (cont’d) • Optical Disk » Write-Once Read Many (WORM) » CD-ROM/DVD burners » Excellent backup and archiving method » Increasingly cheap, but slow. • Huge capacity networks » Server with a main memory & a fast fiber optic network. May not need client’s cache and server’s disk. Files are backed up by optical disk.

6. New Hardware (cont’d) » Problem: memory cache in multiprocessor is similar to the case in which two clients are caching the same file. » Solution: design a simple network interface which has a bit map, one bit per cached file. • Specialized hardware for sophisticated systems » Real-time support » Distributed synchronization and control

7. Scalability • Distributed system size strongly affects algorithm choice » Working well for 100 machines means nothing for 10,000. • Centralized algorithms do not scale well » Opening a file requires to contact the centralized server to record the fact that the file is open. As the system grows, the algorithms do not work well.

8. Scalability (cont’d) » Solution: partitioning the system into smaller units often helps • Broadcasts are a problem » Consider CPU broadcasting one message per second » N of these generate N interrupts at N machines » Not a problem for N=10 » VERY problematic for N=10,000

9. Scalability (cont’d) • Data structures become important with scaling » Linear search easiest and fastest for 10 » Self abuse for even 100 • Strict semantics (Unix) are harder to implement as systems scale » Design Principle: use weakest semantics that make sense

10. Scalability (cont’d) » Trade off ease of programming with scalability • Name space » A single Unix-like file tree. How long can/should path names get? » Partition the tree into smaller trees.

11. Wide Area Networking • Virtually all distributed system research has been done in the context of LANs » Considerable changes with WAN context » Latency » Loss » Cost » Interaction

12. Wide Area Networking (cont’d) • WAN access of major economic importance » WWW commerce » Video on demand » Distributed Virtual Environments • Economies of scale » IP phones

13. Mobile Users • Network addressing is a big challenge - Mobile IP » May be transparent to distributed computing level • Often seen as highly variable communication bandwidth » Isolated » Wireless » Wired

14. Mobile Users (cont’d) • Interesting effects on caching » CODA file system claims to support mobility and intermittent connection • Rapidly Deployable Radio Network – RDRN[2] » Wireless end-user and network nodes » Steerable communication beams » Self-organizing network structure

15. Fault Tolerance • Most systems are not fault tolerant » But the general population expects things to work » Phone system  IP phones? • Requires considerable redundancy » Hardware » Communication infrastructure » Software » Data – File replication

16. Fault Tolerance (cont’d) • Downtimes and periodic crashes will become less and less acceptable as computers spread to non-specialists » ATM machines » Microwaves » Phone system (IP mode)

17. Multimedia • Current data files are rarely more than a few megabytes » Multimedia files can exceed gigabytes » Compression clearly popular because of this and has a fundamental affect on network requirements and economics

18. Multimedia (cont’d) • Video-on-demand » Significant affect on network traffic » Perhaps also on file systems » Real-time support is interesting as well

19. Interesting Research Papers/Projects • xFS: Serverless Network File Service [3] • Distributed file systems for grid computing [4] • Mobile and Distributed file systems [5,6,7] • A Scalable Distributed File System [8]

20. References [1] Andrew S. Tanenbaum. Distributed Operating Systems. Prentice-Hall, Englewood Cliffs, NJ 07632, USA, 1995 [2] Rapidly Deployable Radio Network, http://www.ittc.ku.edu/RDRN/ [3] xFS: Serverless Network File Service, http://now.cs.berkeley.edu/Xfs/xfs.html [4] http://www.gridbus.org/papers/gridtech.pdf

21. References (cont’d) [5] MFS: an Adaptive Distributed File System for Mobile Hosts http://www.cs.cornell.edu/Info/Projects/Spinglass/public_pdfs/mfs.pdf [6] Azzedine Boukerche, Raed Al-Shaikh, Bo Marleau, "Disconnection-Resilient File System for Mobile Clients," lcn, pp. 608-614, The IEEE Conference on Local Computer Networks 30th Anniversary (LCN'05)l, 2005. http://ieeexplore.ieee.org/iel5/10397/33047/01550932.pdf?tp=&arnumber=1550932&isnumber=33047 [7] Haddock-FS, http://www.gsd.inesc-id.pt/~jpbarreto/Haddock-FS.html [8] DiFFS: a Scalable Distributed File System http://www.hpl.hp.com/techreports/2001/HPL-2001-19.pdf