1 / 30

O’Reilly Open Source Convention 2000 Linux Filesystems for Real-World Needs

O’Reilly Open Source Convention 2000 Linux Filesystems for Real-World Needs. Æleen Frisch aefrisch@lorentzian.com www.aeleen.com e x ponential Consulting, LLC North Haven, Connecticut, USA. Itinerary. The Kernel’s VFS Local Filesystems Network Shared Filesystems Advanced Filesystems

zena-washington
Download Presentation

O’Reilly Open Source Convention 2000 Linux Filesystems for Real-World Needs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. O’Reilly Open Source Convention 2000Linux Filesystems for Real-World Needs Æleen Frischaefrisch@lorentzian.comwww.aeleen.comexponential Consulting, LLCNorth Haven, Connecticut, USA

  2. Itinerary • The Kernel’s VFS • Local Filesystems • Network Shared Filesystems • Advanced Filesystems • Futures LinuxFilesystemsforReal-WorldNeeds

  3. Virtual File System (VFS) • Kernel subsystem • Provides a consistent interface forlow-level file I/O • Filesystem needs only use that API andprovide specified functionality, andit is automatically supported. LinuxFilesystemsforReal-WorldNeeds

  4. Filesystems for Local Disks • Lots of supported types: • ext2 • CD-ROM: iso9660, Microsoft Joliet extensions • ufs, fat, vfat, umsdos, ntfs, sysv, affs, adfs, hfs, hpfs, qnx4, … • minix • procfs • /dev/pts LinuxFilesystemsforReal-WorldNeeds

  5. Journaled Filesystems • ReiserFS in SuSE 6.3 • ext3 • dtfs: • www.complang.tuwien.ac.at/czezatke/lfs.html • SGI’s xfs being ported LinuxFilesystemsforReal-WorldNeeds

  6. Network Filesystems • NFS • AFS • smbfs • ncpfs LinuxFilesystemsforReal-WorldNeeds

  7. Linux NFS • Longtime weak spot • Unreliable • Poor performance • nfsstones benchmark • Unable to communicate with non-Linux clients • knfsd is what you want to use • Need to upgrade all current distributions as delivered to latest available version • Kernel patches • New daemon • New utilities LinuxFilesystemsforReal-WorldNeeds

  8. Production Filesystem Features • Bigger than one disk partition (flexibility) • Faster I/O • Fault tolerant • Distributed • Load balancing • Enhanced Security • Access Control Lists • Encryption LinuxFilesystemsforReal-WorldNeeds

  9. Logical Volumes • Dynamically-resizable filesystem consisting of multiple, independent disk partitions (physical volumes), upon which a virtual structure is imposed: • Volume groups--virtual disks-- divisible into … • Logical volume--virtual partitions--which hold … • Filesystems • Common in modern OS’s: AIX, HP, True64 • Logical Volume Manager (lvm): alpha code • Veritas Volume Manager: $$$$ LinuxFilesystemsforReal-WorldNeeds

  10. lvm • linux.msede.com/lvm/ • Basics seem to work, but still Alpha code (version 0.7) • Maximum (theoretical) filesystem size: > 1000 TB LinuxFilesystemsforReal-WorldNeeds

  11. lvm Commands • pv: create, change, display • vg: create, change, display, ck, cfgbackup/restore, export, extend, reduce, remove, split • lv: create, change, display, extend, reduce, remove • e2fsadm (needs PartitionMagic’s resize2fs) LinuxFilesystemsforReal-WorldNeeds

  12. lvm Example • Run fdiskto set partition type to 0xFE • pvcreate /dev/sdb1 /dev/sdc1 • vgcreate new_vg /dev/sd{b,c}1 • /dev/new_vg/group • /etc/lvmconf/new_vg.conf • lvcreate –L 2g –n new_lv –r 4 –C y • mke2fs /dev/new_vg/new_lv • mount /dev/new_vg/new_lv /somewhere • e2fsadm –L +500m /dev/new_vg/new_lv • vs. lvextend LinuxFilesystemsforReal-WorldNeeds

  13. RAID • Redundant Arrays of Inexpensive Disks • Choices • Software • Hardware • Controller • RAID device LinuxFilesystemsforReal-WorldNeeds

  14. RAID Levels • 0: Disk striping—performance only • Best large transfer I/O bandwidth • No loss of storage capacity • 1: Disk mirroring • Best data redundancy • Good performance on small transfers • 3: Disk striping with parity disk • Optimizes I/O bandwidth for large transfers • Low impact of single disk failure • Obsolete! • 5: Parity block rotates among component disks • Optimizes I/O operations/sec • Not vulnerable to single disk failure LinuxFilesystemsforReal-WorldNeeds

  15. Linux RAID • 0, 1, 4(!), 5 • Linear • Kernel support must be enabled • Patches needed (still): • ftp.*.kernel.org/pub/linux/daemons/raid/alpha • RedHat installs for you • Special files: /dev/mdx • Configuration file: /etc/raidtab • mkraid [device] • Persistent superblock • Automatic detection • raidstart and raidstop to control manually LinuxFilesystemsforReal-WorldNeeds

  16. Kernel Support LinuxFilesystemsforReal-WorldNeeds

  17. Raid 0: Striping raiddev /dev/md0raid-level 0nr-raid-disks 2chunk-size 64 persistent-superblock 1 device /dev/sdc1raid-disk 0 device /dev/sdd1raid-disk 1 Raid 1: Mirroring raiddev /dev/md0raid-level 1nr-raid-disks 2 persistent-superblock 1 device /dev/sdc1raid-disk 0 device /dev/sdd1raid-disk 1 Sample /etc/raidtab Entries LinuxFilesystemsforReal-WorldNeeds

  18. raiddev /dev/md0raid-level 0nr-raid-disks 2chunk-size 64k … device /dev/sdc1raid-disk 0 device /dev/sdd1raid-disk 1 raiddev /dev/md1raid-level 0nr-raid-disks 2chunk-size 64k device /dev/sde1raid-disk 0 device /dev/sdf1raid-disk 1 raiddev /dev/md2raid-level 1nr-raid-disks 2 device /dev/md0raid-disk 0 device /dev/md1raid-disk 1 But why? Mirroring a Striped Disk: “RAID 10” LinuxFilesystemsforReal-WorldNeeds

  19. RAID 5 raiddev /dev/md0raid-level 5nr-raid-disks 3nr-spare-disks 1persistent-superblock 1 device /dev/sdc1raid-disk 0 device /dev/sdd1raid-disk 1 device /dev/sde1raid-disk 2 device /dev/sdf1spare-disk 0 LinuxFilesystemsforReal-WorldNeeds

  20. General RAID Considerations • Stripe size matters! • Best value depends on typical I/O transfer size • Testing: trial and error • Underlying filesystem block size = 4K • mke2fs –b 4 … • Don’t overload controllers LinuxFilesystemsforReal-WorldNeeds

  21. Coda • Follow-on to the Andrew File System (AFS) • www.coda.cs.cmu.edu • Distributed filesystem: • /coda on client can have components (“volumes”) from all over the network (transparent to user) • Lots of metadata, stored in a database, using logging and other transaction-based procedure to ensure integrity • Complex configuration task LinuxFilesystemsforReal-WorldNeeds

  22. Coda Features • Read-write data replication • Automatic modification propogation and synchronization/repair • Caching (performance, network failures, mobile computing) • Client-requested files • Automatic update requests • Client-modifications when server is unavailable; attempts automatic synchronization of conflicts • Kerberos-like authentication tokens • Access control lists LinuxFilesystemsforReal-WorldNeeds

  23. Intermezzo • www.inter-mezzo.com • Distributed file system • Designed for high availability • Palimpsest on Coda LinuxFilesystemsforReal-WorldNeeds

  24. Global File System (GFS) • www.globalfilesystem.org • Network-shared storage LinuxFilesystemsforReal-WorldNeeds

  25. Access Control Lists • POSIX ACLs • Alpha code • Recent call for testing … • Specifies file access on per-user/group basis • ACLs inherited from parent directory • Deny>Allow (null=>deny) LinuxFilesystemsforReal-WorldNeeds

  26. Enabling ACLs • Patch and build kernel • Prompt for development option • Get/build utilities • Patch and build ext2fs utilities • mount … -o acl • setfacl and getfact • Do frequent backups (files and ACLs) LinuxFilesystemsforReal-WorldNeeds

  27. Encryption • Encryption • Steganography • Filesystem hidden in the low bits of each byte of an audio file • EFS • ftp.csua.berkeley.edu/pub/cypherpunks/filesystems/linux • PPDD • linux01.gwdg.de/~alatham/ppdd.html LinuxFilesystemsforReal-WorldNeeds

  28. Setup • Patch/configure/build kernel • Get/build utilities • Patch/rebuild standard tools • dd to create file • losetup to set algorithm and pass phrase • mke2fs LinuxFilesystemsforReal-WorldNeeds

  29. Using an Encrypted filesystem • fstab: devicemount-pt ext2 defaults,noauto,loop,encryption=type 0 0 • Prompted for pass phrase when filesystem is mounted LinuxFilesystemsforReal-WorldNeeds

  30. Filesystem Futures LinuxFilesystemsforReal-WorldNeeds

More Related