1 / 15

Chapter 6.5 Distributed File Systems S ummary

Chapter 6.5 Distributed File Systems S ummary. Junfei Wen Fall 2013. Outline. 6.1 Characteristics of DFS 6.2 DFS Design and Implementation 6.3 Transaction and Concurrency Control 6.4 Data and File Replication Current Work Future Work. 6.1Characteristics of DFS. Dispersion

truly
Download Presentation

Chapter 6.5 Distributed File Systems S ummary

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 6.5 Distributed File Systems Summary JunfeiWen Fall 2013

  2. Outline • 6.1 Characteristics of DFS • 6.2 DFS Design and Implementation • 6.3 Transaction and Concurrency Control • 6.4 Data and File Replication • Current Work • Future Work

  3. 6.1Characteristics of DFS Dispersion • Dispersed Files Location Transparent Location Independent • Dispersed Clients login transparency access transparency Multiplicity • Multiple Files Replication Transparency • Multiple Clients Concurrency Transparency Others (general) • Fault Tolerance – crash of server or client, loss of message • Scalability – Incremental file system growth • Efficient

  4. 6.2DFS Design and Implementation • Hierarchy files structure • File mounting protocol • Explicit mounting, manual • Boot mounting, boot up time mounting • Auto mounting, mounting at use • Distribute state information between server and clients. Stateless or stateful server

  5. File access • Space multiplexing, multiple copy of file • Remote access • Cache access • Download/upload access • Time multiplexing, concurrent control. Same file different time • Simple RW • Transaction • session • File sharing semantics • Unix, update propagated immediately • Session, delayed update • Transaction, delayed update

  6. COMPONENTS IN A FILE SYSTEM

  7. 6.3Transaction and Concurrency Control • Distribdute Transaction Processing System: • Transaction Manager: correct execution of local and remote transactions. • Scheduler: schedules operations to avoid conflicts using locks, timestamps and validation managers. • Object Manager: coherency of replicas/caches; interface to the file system.

  8. Serializability: A schedule is Serializable if the result of execution is equivalent to that of a serial schedule. • Concurrency Control Protocol required to maintain ACID Semantics for Concurrent transactions. • Two phase locking • Timestamp ordering • Optimistic

  9. 6.4Data and File Replication • Architecture • Client chooses one / more FSA to access data object. • FSA acts as front end to replica managers RMs to provide replication transparency. • FSA contacts one or more RMs for actual updating and reading of data objects.

  10. One-copy Serializability: • The execution of transaction on replicated objects is equivalent to the execution of the same transactions on non-replicated objects • Read Operations : Read-one-primary, Read-one ,Read-quorum • Write Operations:Write-one-primary,Write-all,Write-all-available,Write-quorum,Write-gossip

  11. Quorum Voting : Uses Read Quorum, Write Quorum • Write-write conflict: 2 * Write quorum > all object copies • Read-write conflict: Write quorum + read quorum > all object copies. • Gossip Update Propagation: • Read: if TSfsa<=TSrm, RM has recent data, return it, otherwise wait for gossip, or try other RM • Update :if Tsfsa>TSrm, update. Update TSrm send gossip. Otherwise, process based on application, perform update or reject • Gossip : update RM if gossip carries new updates.

  12. Current work • A Distributed Cache for Hadoop Distributed File System in Real-Time Cloud Service • Intensive Workload Consolidation for the Hadoop Distributed File Systems • An integrated high-performance distributed file system implementation on existing local network • A Cost-Effective File Lookup Service in a Distributed Metadata File System • The Mobile Agent-based Distributed Network File system

  13. Future work • Innovations in the area of security for Distributed/Cloud Computing • Improve efficiency of Parallel/Distributed system Concurrency control protocol • Improve Efficiency and Effectiveness of file replication scheme • Integrate File Replication and Consistency Maintenance

  14. Reference [1]Distributed Operating Systems and Algorithm Analysis, Andy Chow & Theodore Johnson,1997 [2] “Optimizing File Replication over Limited-Bandwidth Networks using Remote Differential Compression” IEEE Infocom Conference, 2006. [3] Transaction Management and Concurrency control by Connolly & Begg. Chapter 19. Third edition [4]"Distributed File System Replication: Frequently Asked Questions";http://technet2.microsoft.com/WindowsServer/en/library/f9b98a0f-c1ae-4a9f-9724-80c679596e6b1033.mspx?mfr=true [5]http://blogs.cs.st-andrews.ac.uk/angus/2009/09/ [6]http://www.quora.com/Distributed-Systems/What-is-the-future-of-file-systems -Future of File Systems

  15. Thank you！

More Related