1 / 46

File Layer and Virtual File System

File Layer and Virtual File System. Chapter One. Topics. File System Abstractions File System Layers The File Layer The Virtual File System Selected File Related Calls. UNIX File Abstraction. The File Stream of bytes - sequence of bytes therefore buffer cache is required

bsloan
Download Presentation

File Layer and Virtual File System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. File Layer and Virtual File System Chapter One

  2. Topics • File System Abstractions • File System Layers • The File Layer • The Virtual File System • Selected File Related Calls

  3. UNIX File Abstraction • The File • Stream of bytes - sequence of bytes • therefore buffer cache is required • any record structure is imposed by application • Sequential or Random Access • The Directory Structure • Tree-like directory hierachry • File sharing • hard links - multiple names for same disk file • link count for hard links (same disk only) • soft (symbolic) links - stored path shortcut • Access control associated with the file

  4. File Related System Calls • open(), close() • creat(), unlink() • read(), write() • seek() • getattr(), setattr() • mmap() • ioctl() • fsync() • dup(), dup2()

  5. File Descriptor • Process applications name for an open file • small integer returned by open() • The first three file descriptors are 0 -- standard input 1 -- standard output 2 -- standard error and are usually associated with a terminal • Each has an associated offset or file position pointer

  6. Types of Files • Regular - sequence of bytes • Directory • Block Special (Device) File • Character Special (Device) File • FIFO (Named Pipe) • Symbolic Link • Socket (In AF_UNIX Domain)

  7. UNIX Disk Abstraction • Partitions • Subsets of the disk that may be treated as logical disk drives. • Partitioning a Large Disk • Overcomes 32-bit UNIX limit problems • Isolates directories • Decreases fsck time • disklabel utility writes/edits disk label. • Partition identified by a special file • block /dev/rz[LUN_letter]/dev/rz[bus_number*8+target][partition_letter] • character /dev/rrz[LUN_letter]/dev/rrz[bus_number*8+target][partition_letter] • Where LUN_letter is optional

  8. UNIX File System Abstraction • Two Senses • A mountable directory hierarchy administered in /etc/fstab. • A specific implementation of the UNIX file abstraction (UFS, NFS, AdvFS, CDFS, etc). • One file system is the root file system • Other file systems are graphed in to the root by mounting.

  9. File System System Calls • mount(), unmount() • sync()

  10. DOS UNIX rz0a A: B: C: rz3c rz0g The Virtual File System:Transparent Access

  11. Application Process Common system calls: open(), close(), read(), write, seek() VFS To specific filesystem type implementation of the call The Virtual File System: Uniform Access

  12. File System Management Layers System Call read(), write() etc. Manage file access state for a given process Manage file descriptors Manage file structures File Layer Represent filesystem and files generically Virtual File System Specific file system implementation, UFS, Advfs NFS, MFS, etc. File System(s) In memory block storage for a file system. Could be traditional buffer cache, unified buffer cache or home grown. Cache Device Local Block Device, Network Interface or a Logical Volume/RAID.

  13. Digital UNIX File Systems • “True”Data File Systems • UNIX File System (UFS) • Network File System (NFS) • Advanced File System (AdvFS) • Memory File System (MFS) • CD File System, ISO 9660:1988 (CDFS) • Pseudo-File Systems or Layers • Proc File System (procfs) • File Descriptor File System (FDFS) • File-on-File File System

  14. File Layer and VFS Structures UFS VFS UNIX Domain vnode inode socket vnode inode vnode inode f_data file mount ttyvp proc and session ofile ufs_mount cdir rdir proc utask uthread utask

  15. The File Layer (1) • Relates a process to an open file • Per process descriptor table • within process’s utask structure for first 64 open files • entry for every file open by this process • includes pointer to a file structure • entries are allocated when • a file is opened or a pipe or socket is created • inherited in a fork() • a descriptor is copied via dup() • entries are deallocated when • a file, pipe or socket is closed • a process terminates

  16. The File Layer (2) • File structure • records state of access to a file • a signal process or parent child as the result of a fork() • for regular files • includes an ops vector for manipulating regular files • includes a pointer to a vnode • for sockets • includes an ops vector for manipulating sockets • includes a pointer to a socket

  17. struct utask file descriptor table (1) struct utask { ... struct ufile_state uu_file_state; ... } struct ufile_state { int utask_need_to_lock; /* set if utask locks needed */ udecl_simple_lock_data(,uf_ofile_lock) struct file *uf_ofile[NOFILE_IN_U]; /* file structs of open files */ ....

  18. struct utask file descriptor table (2) .... char uf_pofile[NOFILE_IN_U]; /* per-process flags of open files */ int uf_lastfile; /* high-water mark of uf_ofile */ u_int uf_of_count; struct file **uf_ofile_of; /* Pointer to KALLOC'ed buffer */ char *uf_pofile_of; /* Pointer to KALLOC'ed buffer */ }

  19. struct file (1) struct file { udecl_simple_lock_data(,f_incore_lock) int f_flag; /* see below */ uint_t f_count; /* reference count */ int f_type; /* descriptor type */ int f_msgcount; /* references from message queue */ struct ucred *f_cred; /* descriptor's credentials */ struct fileops *f_ops;/* operations on f_data */ caddr_t f_data; /* vnode, socket, etc. */pointer to vnode or socket ....

  20. struct file (2) .... union { /* offset or next free file struct */ off_t fu_offset; struct file *fu_freef; } f_u; uint_t f_io_lock; /* i/o lock (low half of thread ptr) */ int f_io_waiters; /* number of waiters on i/o lock */ };

  21. struct fileops struct fileops { int (*fo_read)(); int (*fo_write)(); int (*fo_ioctl)(); int (*fo_select)(); int (*fo_close)(); }

  22. struct fileops Implementations • Regular Files: vfs/vfs_vnops.c struct fileops vnops = { vn_read, vn_write, vn_ioctl, vn_select, vn_close }; • Sockets: bsd/sys_socket.c struct fileops socketops = { soo_read, soo_write, soo_ioctl, soo_select, soo_close };

  23. Virtual File System • Originally designed for UNIX by Sun Microsystems, Inc., to support the Network File System (NFS). • Allows flexible support of multiple file system types • Object-oriented - magic word • classes of objects with member functions • struct vnode - for every active file system • a generic representation of a file for all types of file system implementations. • struct mount - for every active file system • a generic representation of a whole mountable file system for all file system implementations. • a file system implements its own set of; • member functions for vnodes and mount structures • data structures to combine with generic vnode and mount structures

  24. <locks> v_flag v_usecount v_holdcnt lock counts v_lastr v_id v_type v_tag mount structure v_mount vnodeops structure v_op vnode structure v_freef vnode structure v_freeb vnode structure v_mountf vnode structure v_mountb buf structure v_cleanblkhd buf structure v_dirtyblkhd ..... struct vnode (1) multiprocessor exclusion vnode flags reference count of users page & buffer references user-level lock counts last read (read-ahead) capability identifier vnode type type of underlying data ptr to vfs we are in vnode operations vnode freelist forward vnode freelist back vnode mountlist forward vnode mountlist back clean blocklist head dirty blocklist head

  25. v_ncache_time v_free_time v_numoutput v_outflag v_cache_lookup_refs v_rdcnt v_wrcnt mount structure v_mountedhere v_un vm_object structure v_object vnsecops structure v_secops v_data[ ] struct vnode (2) .... last cache activity time time on vnode free_list num of writes in progress output flags count of readers count of writers ptr to mounted vfs ptr to sock, dev specinfo, pipe VM object for vnode vnode security ops placeholder, private data

  26. Types of Vnodes Type Description VNON Allocated, but as-yet untyped vnode VREG Vnode representing a regular file VDIR Directory vnode VBLK Block device vnode VCHR Character device vnode VLNK Symbolic link vnode VSOCK Vnode representing a UNIX domain socket VFIFO FIFO special file vnode

  27. struct vnodeops (1) Operation Function vn_lookup Looks up a file vn_create Creates a regular file vn_mknod Creates a fifo or device special file vn_open Opens a file vn_close Closes a file vn_access Checks the access for a file vn_getattr Gets file attributes vn_setattr Sets file attributes vn_read Reads a file vn_write Writes to a file vn_ioctl Controls a device vn_select Performs synchronous I/O multiplexing vn_mmap Map memory of a character device

  28. struct vnodeops (2) Operation Function vn_fsync Synchronizes file data and statistics vn_seek Sets position on a file vn_remove Removes a file vn_link Creates a hard link to a file vn_rename Renames a file vn_mkdir Creates a directory vn_rmdir Removes a directory vn_symlink Creates a symbolic link to a file vn_readdir Reads a directory vn_readlink Reads contents of a symbolic link vn_abortop Aborts operation vn_inactive Sets inactive vn_reclaim Reclaims a vnode

  29. struct vnodeops (3) Operation Function vn_bmap Maps to file system block vn_strategy Calls device strategy routine vn_print Prints the contents of an inode vn_pgrd Reads a page vn_pgwr Writes a page vn_swap Swaps handler vn_bread Reads buffer vn_brelse Releases buffer vn_lockctl Provides file locking vn_syncdata Synchronizes a byte range of an open file

  30. struct vnodeops (4) Operation Function vn_lock Locks an inode vn_unlock Unlocks an inode vn_getproplist Gets extended attributes vn_setproplist Sets extended attributes vn_delproplist Deletes extended attributes vn_pathconf Checks path

  31. m_lock m_flag m_funnel mount structure m_next mount structure m_prev vfsops structure m_op vnode structure m_vnodecovered vnode structure m_mounth m_vlist_lock m_exroot m_uid m_stat m_data m_nfs_errmsginfo m_unmount_lock struct mount Lock for synchronization (SMP) Flags Flag for SMP Next in mount list Previous in mount list Operations on file system Vnode we are mounted on List of all vnodes for this mount Lock for vnode list Exported mapping for UID 0 UID of mounter File system statistics Private data NFS error information Lock for synchronization

  32. struct vfsops Operation Function vfs_mount Mounts the file system vfs_start Starts the file system vfs_unmount Unmounts the file system vfs_root Returns the vnode for the root of the file system. vfs_quotactl Performs operations associated with quotas vfs_statfs Updates file system statistics vfs_sync Synchronizes the file system vfs_fhtovp Returns the vnode pointer, given a file handle vfs_vptofh Returns a file handle, given a vnode pointer vfs_init Initializes the file system vfs_mountroot Mount root file system vfs_swapvp Is not used

  33. VFS Switch Table • Identifies file system types that have been implemented. • Contains an entry point for file system operations for each supported file system type. struct vfsops *vfssw[MOUNT_MAXTYPE];

  34. mount vfssw vfsops NULLPTR ufs_mount *m_op &ufs_vfsops ufs_start &nfs_vfsops ufs_unmount ufs_root ufs_quotactl Setting Up File System Operations

  35. rootfs m_next m_next m_next Mount Table m_prev m_prev m_data v_mount v_mount v_mount Vnodes v_data v_data v_data file system specific file information file system specific file information file system specific file information file system specific file information Mounted File System Structures

  36. Recording Mount PointsHow are they mounted? (1) A A B B C C

  37. rootfs next next Mount Structs mounted here Vnode Structs covered VROOT VDIR VDIR Recording Mount PointsHow are they mounted? (2)

  38. File System Operations namei() Interprets a pathname mount() Mounts a file system open() Opens a file read()/write() Reads or writes a file

  39. Namei (1) • VFS routine that maps pathnames to vnodes • performs access checks on each component of that pathname. • Uses VOP_LOOKUP to move down the path • Special Cases • Symbolic Links - pre-fastened • Mount Points • Process-Specific root (chroot()) • Special Care - unmounts

  40. Namei (2) • A LRU Hash Table < parent-vnode, component-name> to < target-vnode, capabilities> • Capabilities are "tags" assigned to vnodes • prevent cache entries from referring to out-of-date associations • Related data structures include: • namecache - namei cache • nchash - hash list for cache • nchsize - size of cache • nchsz - size of hash list

  41. Start Copy name into local buffer Copy next component to buffer Yes Find parent vnode ".." ? No Call file system specific lookup routine VOP_LOOKUP() Yes Copy name to buffer Symbolic link? No Yes Find root vnode of mounted file system: VFS_ROOT() Mounted on? No No Yes More components? Done namei()flow

  42. namei() mount() vmountset() Mount Table UFS ufs_mount() VFS_MOUNT mount()flow

  43. File Table falloc() open() namei() VOP_LOOKUP() vn_open () VOP_CREATE() open() flow

  44. Process Descriptor Table File Table Vnode Table File read() write() getf() rwuio() FOP_READ() FOP_WRITE() vn_read() VOP_READ() VOP_WRITE() ufs_read() ufs_write() read()and write()flow

  45. Source Reference (1 of 2) • kernel/sys/users.h • defines a process’s open file table ufile_state in struct utask • kernel/sys/file.h • defines a struct file and fileops • kernel/vfs/vfs_vnops.c • implementation of fileops for file structs referencing vnode • kernel/bsd/sys_socket.c • implementation of fileops for file structs referencing sockets • kernel/sys/vnode.h • definition of vnode and vnodeops

  46. Source Reference (2 of 2) • kernel/sys/mount.h • definition of struct mount and vfsops • kernel/vfs/vfs_syscalls.c • vfs_switch[] • kernel/vfs/vfs_lookup.c • implementation of namei() • kernel/vfs/vfs_syscalls.c • implementation of mount() and open() calls • kernel/bsd/sys_generic.c • implementation of read() and write() calls

More Related