1 / 26

Chapter 16: Remote Files

Chapter 16: Remote Files. Prof. Steven A. Demurjian, Sr. † Computer Science & Engineering Department The University of Connecticut 191 Auditorium Road, Box U-155 Storrs, CT 06269-3155. steve@engr.uconn.edu http://www.engr.uconn.edu/~steve (860) 486 - 4818.

vance-pace
Download Presentation

Chapter 16: Remote Files

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 16: Remote Files Prof. Steven A. Demurjian, Sr. † Computer Science & Engineering Department The University of Connecticut 191 Auditorium Road, Box U-155 Storrs, CT 06269-3155 steve@engr.uconn.edu http://www.engr.uconn.edu/~steve (860) 486 - 4818 † These slides have been modified from a set of originals by Dr. Gary Nutt.

  2. Purpose of this Chapter • Examine Ways that OS can Leverage Network to Reference Remote Secondary Storage Devices • Objectives - to Examine … • Computational Model for Network-Based Computing • OS Strategies for Distributing File Systems Across Network • File Manager Use Storage Devices on Remote WSs • File Manager Functionality Distributed Across WSs • OS Transparently Cop Files/Maintain Consistency • “Same” Abstraction for Local vs. Remote Files • Examine “Classic” and “Current” Approaches

  3. Impact of Network on OS • Can a Process on One Machine use Primary & Secondary Memory on Another Machine? • How must OS be Redesigned to Support this? • How can Existing Abstractions be Leveraged? • Process Already Uses Local Primary/Secondary Memory Interfaces • Allow Process to Utilize these Abstractions for Remote Access • Remote File Technology Evolved Dramatically since FTP (File Transfer Protocol) and UUCP (Unix to Unix CoPy) • Example: Mounting Unix Account/Files on Learning Center NTs

  4. Traditional Memory Interfaces • In Component Diagram Below • Can Process Access to Remote Information be Transparent? • How Can Existing Abstractions be Exploited? Process Primary Memory Interface Secondary Memory Interface Virtual Memory File Management Device Interface Physical Memory Storage Devices

  5. Explicit File Copying • Need a Way for a Process on One Machine to Pass Information to a Process on Another Machine • Technique (FTP and UUCP) • Sender Writes a File • User Manually Copies File to a Remote Machine • Receiver Opens the File and Reads It • Very Coarse Grained • Very High Latency • While Shell Scripts/Batch Files Can Automate the Process -- it Remains • Too High Level • Not Suited to Application Programs

  6. Implicit File Copying • Rephrase the Question: • Need a Way for a Process on One Machine to Transparently Access Information on Another Machine • One Solution: Some OS Utilize Primary Memory Interface to Also Access Secondary Storage • Virtual Memory Implementation Handles Retrieving/Storing to Secondary Storage • Also, Maintains Referenced Blocks in Memory • Multics Uses Such an Approach • Segment/Offset Combination Allows Segment Names to be Mapped to File Names • Memory Manager Also Controls Files

  7. Multics Segmented Memory Process Primary Memory Interface Secondary Memory Interface Segmented Virtual Memory File Management Device Interface Physical Memory Storage Devices Normal data flow Alternative data flow

  8. Local File Manager Able to Access Both Local and Remote Files Remote File Access is Transparent to Process Advantage: Reuse Existing Interface Interface to Remote Server can be Remote Disk Remote File System We’ll Explore Each Approach Complementary ApproachRemote Secondary Memory Process Secondary Memory Interface File Management Client Interface to Remote Server Device Interface Storage Devices Comm Network

  9. Remote Access Module Implements Remote File Ops, Network Interactions, Transparency, etc. Remote Storage Module Accepts Commands and Performs Operations Refined View: Client/Server ArchitectureRemote File Facility as Server Client Machine Process Secondary Memory Interface File Management Server Machine Remote Access Remote Storage Storage Devices Communication Network Storage Devices

  10. What are Critical Questions? • Questions Below Represent Fundamental Design Issues for Remote File Systems • How are Normal File System Functions Divided Between Client (Remote Access) and Server (Remote Storage)? • What Network Protocols are Best Suited for Distributed File Manager? • Can Distributed File Manager Provide Satisfactory Performance? • Can Network and Server Reliability be Attained? • What Basic Strategies have Emerged for Partitioning File System Functionality?

  11. Strategies for File System Distribution • Remote Disk Systems • Client Interface Similar to Local File System • Disk Server Manages Blocks - Doesn’t Know About File Abstraction (Dumb Server) • Remote File Systems • File System Functionality Spread Between Client and Server • Remote File Server Manages Files, Descriptors, etc. • File Caching • Must Track Multiple Copies of Same File in Different Locations • Manage Content and Insure Consistency

  12. Remote Disk Systems • History • Motivated by Cost and Performance (Heat and Noise) of Drives • Network Computers (No Local Disk) • Partitioned into • Virtual Disk Drive (VDD) • Replaces Conventional Local Drive • Exploits “Same” Abstraction as Local Drive • Remote Disk Application (RDA) • Abstract Disk can be Read/Written Like Local One • However, Client Restricted in Formatting, Reading Boot Records, etc.

  13. Architecture of Remote Disk Server Client Machine General File Management Server Machine Virtual Disk Driver Remote Disk Application Storage Devices Communication Network Storage Devices

  14. Remote Disk Server • VDD Encapsulates Disk Commands Into Network Packet and Transmits to • RDA which Unpacks Packet and Generates a Request to Local Disk file mgr: diskRequest(details); VDD: Pack parameters; VDD: Send request; (wait for response) (waiting for a request) RDA: Unpack parameters; RDA: Generate local disk request; (waiting) RDA: Generate reply; RDA: Send reply (waiting for a request) VDD: Receive reply; VDD: Unpack parameters; VDD: Return to file mgr

  15. Performance & Reliability • Became Commercially Feasible in About 1986 • Biggest Concern Was Reliability • Reliability Related to Two Issues • Ensure Disk Command Eventually Executed by Server • Synchronizing Operations of Client and Server Should One or Other Crash During Execution • Focus of Reliability • Ensuring that System Works Properly Even if Packet is Lost • Lost can Occur Two Ways • Command Packet is Lost Before Receipt by Server • Result was Lost After Server Completes Read Op.

  16. Reliable Command Execution • Time-outs • VDD Starts Count Down Timer After Send • If Timer Gets to Zero & No Result --> Resend • What Happens when Timer Expires? • If Result of Read Operation Lost, Server can Re-execute and Produce Same Result • This is Referred to as Idempotent Operation • Similar Situation for Write Operations • Crash Recovery: Employ Stateless Servers • Disk Server Doesn’t Know About File Abstraction nor Maintain Request Queue • Clients Responsible for Repetitive Commands via Timer Expiration Paradigm

  17. Remote File Systems • Provide Same Interface to Application Program Regardless of Remote or Local File • Differ from • Local File Systems by Naming Scheme for Files (Mounting) • Remote Disk by Partitioning of File Manager Functionality Between Client and Server • General Architecture Employs • Client that Implements Operations Needed to Interact with Server • Server that Provides Much of Functionality of File Manager (File Descriptors, Access Abstraction, etc.)

  18. Architecture of Remote File Server General File Management Client Machine Server Machine Client Part of Remote File System Server Part of Remote File System Storage Devices Communication Network Storage Devices

  19. Remote File Server Client Machine Local File Management Server Machine • Block Management • Buffering • Device Management • . . . • Read/Write Management • Pack/Unpack Byte Stream • Buffering • . . . Communication Network Storage Devices

  20. Block Caching: Making RFS Smarter in Providing Remote Files for Clients • Widely Used in All File Systems • Remote Access Involves Network and Extra Components - Hence Want to Reduce Traffic • In RFS, Can Buffer at: • Server: Eliminates I/O Latency • Doesn’t Avoid Network Latency • Still Need to Send Packet When Needed • Client • Consistency if Multiple Access to Same File • Sometimes Use Sequential Write Consistency (No Sharing If There Are Multiple Writers) • Overall, Various Techniques Attempt to Exploit LRU Concepts to Optimize Access

  21. Crash Recovery • Client Has a File Open and Server Crashes • Distributed State Makes Recovery Difficult • Can Counteract With a Stateless Server • But It Requires That State Be Transmitted With Every Service Request • Originally, Crash Recovery Greatest Challenge, Since Tied to Long-Term Acceptance of RFS • Approaches Include • Recovery-Oriented File Service, e.g., Sun NFS • Operate Reliably at Cost of Performance • Performance-Oriented File Service • Complex Algorithms that Require Overhead for Each Remote File Operation

  22. Overview: Sun NFS • Sun Network File System (NFS) Groundbreaker in Remote File Systems • Goal: Support Heterogeneous File System Across Network (Suns, PCs, Digital Workstations, etc.) • Sun’s NFS • Replaces Unix File System with Virtual File System in Client and Server Kernels • Different Parts of NFS File Hierarchy Can be Implemented by Different OS File Managers • Abstract File Descriptor, vnode, and Peer-to-Peer Protocol Between Client and Server • Sun’s NFS Popularity Resulted in Standard NFS Protocol

  23. Architecture of Sun NFS Client Machine Server Machine VFS Interface VFS Interface Unix (& other File Mangers) Client Part of NFS Server Part of NFS Unix File Manager Storage Devices Communication Network Storage Devices

  24. File-Level Caching for Remote Disk or Remove File Server • How and When are Files Copied from Serer to Client and Back Again? • Immutable Files Cannot be Changed by Client • Maintain Multiple Versions of Each File • Server Manages All Versions • When Two Clients Write Same File and Return to Server, How is Inconsistency Handled? • Server Assigns Different Version Numbers • Allows Multiple Clients to Write Own Version • Mutable Files can be Altered by Clients and Thus, Must be Managed by Server • File Management and Configuration Management are Very Related

  25. Directories • Hierarchical Structure Employed Across Entire Remote File System • Names Differentiated by • Superpath Names • Machine Name Plus File: csent9:/usr/steve/cs258/test • Remote Mount: Based on Where Mounting Occurs • /usr/steve/cs258/test and /home/steve/cs258/test on Two Different Machines are “Same” File • Hence, Processes Can’t Exchange Mount Addresses to Share Files • Global Name Space Needed

  26. Concluding Remarks/Looking Ahead • Network File Services Extend Local File Services • Remote Disk Servers Implement Disk Driver on Server - File Manager Replicated on Clients • Remote File System Partitions Functionality of File Manager Across Client and Server • Key Issues are: Performance and Recoverability • Interesting Exercise 4 in Section 16.7 • Suppose a RFS has File State Distributed Between Client and Server • Why Must Client/Server Both have Copies of File Locks for Open Files? • Looking Ahead to … • Distributed Computing (Chapter 17), Linux Presentations, and Final Exam

More Related