200 likes | 396 Views
Tam Chantem, Philip Little and Faruck Morcos. iDIBS : Reliable and Efficient Distributed Backup. Improving Peer-to-Peer Backup Systems. Objective : Improve peer-to-peer backup techniques. Approach : Built on DIBS , an existing peer-to-peer backup system. Results : Improved 3 aspects of DIBS:
E N D
Tam Chantem, Philip Little and Faruck Morcos iDIBS: Reliable and Efficient Distributed Backup
Improving Peer-to-Peer Backup Systems • Objective: Improve peer-to-peer backup techniques. • Approach: Built on DIBS, an existing peer-to-peer backup system. • Results: Improved 3 aspects of DIBS: • Reliability, Network Load, and Computation Time.
Erasure Codes iDIBS Peer List iDIBS Peer File Encoding Peer Peer Database
Erasure Codes iDIBS Peer List iDIBS Peer File Encoding Peer Peer Database
a b c d e f g h RS Encoder a d g j i j b e h k k l c f i l Backup and Recovery Encoding and Distributing: Peer File Peer Peer
a b c d e f g h RS Decoder a d j i j b k k l c i l Backup and Recovery Recovering and Decoding: Peer File Dead Peer Peer
? Data Peer List ? a b c ? ? d e f Peer-List Backups Unmodified DIBS after a crash: Do you have my data? Client
? Data Peer List ? a Peer List b c ? ? d Peer List e f Peer-List Backups iDIBS after a crash: Do you have my peer list? Client
Data Peer List a b c d e f Peer-List Backups iDIBS after a crash: Client
Thresholds • K – Required number of pieces to recover • N – Total number of pieces transmitted • Redundancy level: • DIBS: N = 2K • iDIBS: Recovery when up to 1 peer is dead
Example Assume: Peers = 4 K = 5 N = 6 Peer 1 Peer 2 Peer 3 Peer 4 Can’t recover!!! So, N = 8
Luby Transform Codes • Reed-Solomon (RS) codes not scalable • Luby Transform (LT) codes • Digital Fountain concept • Probabilistic nature • 15% redundancy needed for successful decoding • Flexible symbol size T, to speed up decoding
Implementation: LT Codes • Encoder and Decoder modules • Encoder: • Calculates redundancy needed • Encodes and splits file depending on T, number of users, and probabilistic parameters • Decoder: • Decodes pieces • Reconstructs file
Luby Transform Codes Encoder/Decoder Performance LT codes vs. RS codes Encoder T=256 Decoder T=256 Processing Time (sec) File Size (Kb)
iDIBS vs. DIBS network utilization behavior • DIBS network utilization is flat. • iDIBS has decreased network utilization. (OH when users<10) • Tradeoff in the theoretical minium is better in iDIBS. • Results shown for T=32, for larger numbers iDIBS plots are better!
iDIBS Advantages Disadvantages LT codes are: 1. Faster than RS 2. Flexible performance given T. Decreased overall Network Utilization. Peer Lists: 1. Allow faster recovery. 2. Increase reliability. LT codes are probabilistic. As T > 256 decoding is less stable LT need a minimum of 15% of extra overhead + ESI to decode. Peer lists induce a small amount of overhead The iDIBS tradeoff
Contribution We contributed to the improvement of DIBS in the following ways: • Increase of performancethrough the introduction of a new encoding technology. (LT codes) • Introduction of this encoding scheme to the application of peer-to-peer backup systems. • More Reliability through redesign of the system recovery scheme. (Peer-Lists) • Reduces Network utilization by changing the philosophy of number of transmitted pieces N and LT codes.
Implementation: Timeouts & Thresholds If: Peers > K then N = K + 1;else: PiecesPerPeer = 1;while PiecesPerPeer * (Peers - 1) < K do: PiecesPerPeer = PiecesPerPeer + 1; N = N * K;end if If a peer is offline, recovery is still possible