6.4 Data and File Replication

6.4 Data and File Replication Gang Shen

Why replicate • Performance • Reliability • Resource sharing • Network resource saving

Challenge • Transparency • Replication • Concurrent control • Failure recovery • Serialization

Atomicity • In database systems, atomicity is one of the ACID transaction properties. An atomic transaction is a series of database operations which either all occur, or all do not occur[1]. • All or nothing

Atomicity • In DFS (Distributed File System), replicated objects (data or file) should follow atomicity rules, i.e., all copies should be updated (synchronously or asynchronously) or none.

Goal • One-copy serializability: The effect of transactions performed by clients on replicated objects should be the same as if they had been performed one at a time on a single set of objects.[2]

Architecture • FSA , File service agent, client interface • RM, replica manager, provide replication functions [3]

Architecture[3]

Read operations [3] • Read-one-primary, FSA only read from a primary RM, consistency • Read-one, FSA may read from any RM, concurrency • Read-quorum, FSA must read from a quorum of RMs to decide the currency of data

Write Operations[3] • Write-one-primary, only write to primary RM, primary RM update all other RMs • Write-all, update to all RMs • Write-all- available, write to all functioning RMs. Faulty RM need to be synched before bring online.

Write Operations • Write-quorum, update to a predefined quorum of RMs • Write-gossip, update to any RM and lazily propagated to other RMs

Read one primary, write one primary • Other RMs are backups of primary RM • No concurrency • Easy serialized • Simple to implement • Achieve one-copy serializability • Primary RM is performance bottleneck

Read one, Write all • Provides concurrency • Concurrency control protocol needed to ensure consistency (serialization) • Achieve one-copy serializability • Difficult to implement (there will be failed TM to block any updates)

Read one, Write all available • Variation of Read one, Write all • May not guarantee one-copy serializability • Issue of loss conflict in transactions

Loss of Conflicts • Assume to RMs, (a,b), object X,Y replicated to both. • Two transactions • T1:R(X),W(Y),commit • T2:R(Y),W(X),commit

Loss of Conflict • If Xa,Yb failed, transaction as follows • T1:R(Xa),(Yb failed),W(Ya),commit • T2:R(Yb),(Xa failed),W(Xb),commit • There is no conflict since no object is shared. Thus loss conflict. This can introduce error.

Read quorum,Write quorum • Version number attached to replicated object • Highest version numbered object is the latest object in read. • Write operation advances version by 1 • Write quorum > half of all object copies • Write quorum+read quorum > all object copies

Gossip Update • Applicable for frequent read, less update situations • Increased performance • Typical read one, write gossip • Use timestamp

Basic Gossip Update • Used for overwrite • Three operations, read, update, gossip arrive • Read, if TSfsa<=TSrm, RM has recent data, return it, otherwise wait for gossip, or try other RM • Update, if Tsfsa>TSrm, update. Update TSrm send gossip. Otherwise, process based on application, perform update or reject • Gossip arrive, update RM if gossip carries new updates.

Causal Order Gossip Protocol[3] • Used for read-modify • In a fixed RM configuration • Using vector timestamps • Using buffer to keep the order

Windows Server 2003[4] • Support DFS • “State based, multi master” scheduled replication • Use namespace for transparent file sharing • Use Remote Differential Compression to propagate change only to save bandwidth

Continued[5] If replication detects a conflict, last update wins. No merge files, but copies are kept for reference.

Reference [1] Wikipedia; http://en.wikipedia.org/wiki/Atomicity [2] M. T. Harandi;J. Hou (modified: I. Gupta);"Transactions with Replication";http://www.crhc.uiuc.edu/~nhv/428/slides/repl-trans.ppt [3] Randy Chow,Theodore Johnson, “Distributed Operating Systems & Algorithms”, 1998 [4] "Overview of the Distributed File System Solution in Microsoft Windows Server 2003 R2";http://technet2.microsoft.com/WindowsServer/en/library/d3afe6ee-3083-4950-a093-8ab748651b761033.mspx?mfr=true [5] "Distributed File System Replication: Frequently Asked Questions";http://technet2.microsoft.com/WindowsServer/en/library/f9b98a0f-c1ae-4a9f-9724-80c679596e6b1033.mspx?mfr=true

6.4 Data and File Replication