Network Coding for Distributed Storage Systems

1. Network Coding for Distributed Storage Systems IEEE TRANSACTIONS ON INFORMATION THEORY, SEPTEMBER 2010 Alexandros G. Dimakis Brighten Godfrey Yunnan Wu Martin J. Wainwright KannanRamchandran

2. Outline • Introduction • Background • Analysis • Evaluation • Conclusion

3. Introduction • Distributed storage systems provide reliable access to data through redundancy spread over individually unreliable nodes. • Storing data in distributed storage systems • the encoded data are spread across nodes. • require less redundancy than replication. • replace stored data periodically.

4. Introduction • Key issue in distributed storage systems. • repair bandwidth • storage space • How to generate encoded data in a distributed way as little data as possible ?

5. MDS Codes • A common practice to repair from a single node failure for an erasure coded system. • a new node to reconstruct the whole encoded data object. • then, generate just one encoded block. • Maximum Distance Separable (MDS) code. • (n, k)-MDS property • recover original file by any k set of encoded data.

6. MDS Codes M/k M/k MDS encode M/k store at n nodes File divide encode M/k

7. Introduction • Redundancy must be continually refreshed as nodes fail in distributed storage systems. • large data transfers across the network.

8. Introduction • The erasure codes can be repaired without communicating the whole data object. • (4, 2)-MSR example when node is fail. • generate smaller parity packets of their data. • forward them to the newcomer. • the newcomer mix packets to generate two new packets. 0.5 0.5 0.5 0.5 0.5 0.5 0.5

9. Introduction • This paper identifies that there is a optimal tradeoff curve between storage and repair bandwidth. • smaller storage space => less redundancy => more repair bandwidth • This paper calls codes that lie on this optimal tradeoff curve regenerating codes.

10. Introduction • Minimum-Storage Regenerating (MSR) codes. • can be efficiently repaired. • Minimum-Bandwidth Regenerating (MBR) codes. • storage node stores slightly more than M/k . • the repair bandwidth can be reduced.

11. Outline • Introduction • Background • Analysis • Evaluation • Conclusion

12. Erasure Codes • Classical coding theory focuses on the tradeoff between redundancy and error tolerance. • In terms of the redundancy-reliability tradeoff, the Maximum Distance Separable (MDS) codes are optimal. • the most well-known is Reed-Solomon codes.

13. Network Coding • Network coding allows • the intermediate nodes to generate output data by encoding previously received input data. • information to be “mixed” at intermediate nodes. • This paper investigates the application of network coding for the repair problem in distributed storage. • tradeoff between storage and repair network bandwidth

14. Distributed Storage Systems • Erasure codes could reduce bandwidth use by an order of magnitude compared with replication. • Hybrid strategy: • one special storage node maintains one full replica. • multiple erasure encoded data. • transfer only M / kbytes for a new encoded data by replica node. • there is the problem when replica data lost.

15. Outline • Introduction • Background • Analysis • Evaluation • Conclusion

16. Information Flow Graph

17. Storage-Bandwidth Tradeoff • The normal redundancy we want to maintain requires active storage nodes • each storing αbits • β bits each from any d surviving nodes • total repair bandwidth is γ = dβ • For each set of parameters (n, k, d, α,γ), there is a family of information flow graphs, each of which corresponds to a particular evolution of node failures / repairs.

18. Storage-Bandwidth Tradeoff • Denote this family of directed acyclic graphs by • (4, 2, 3, 1 Mb, 1.5 Mb) is feasible.

19. Storage-Bandwidth Tradeoff • Theorem 1 : For any α≥ α*(n, k, d, γ), the points are feasible.

20. Theorem Proof (1/4)

21. Theorem Proof (2/4) • . • . • . • .

22. Theorem Proof (3/4) • . • .

23. Theorem Proof (4/4) • . • .

24. Storage-Bandwidth Tradeoff • Code repair can be achieved if and only if the underlying information flow graph has sufficiently large min-cuts.

25. Storage-Bandwidth Tradeoff • Optimal tradeoff curve between storage α and repair bandwidth γ • (γ = 1, α= 0.2) (γ = 1, α= 0.1)

26. Special Cases (1/2) • Minimum-Storage Regenerating (MSR) Codes • . • .

27. Special Cases (2/2) • Minimum-Bandwidth Regenerating (MBR) Codes • . • .

28. Outline • Introduction • Background • Analysis • Evaluation • Node Dynamics and Objectives • Model • Quantitative Results • Conclusion

29. Node Dynamics and Objectives (1/2) • A permanent failure • the permanent departure of a node from the system • a disk failure resulting in loss of the data stored on the node • A transient failure • node reboot • temporary network disconnection

30. Node Dynamics and Objectives (2/2) • A file is available • it can be reconstructed from the data stored on currently available nodes. • A file is durability • after permanent node failures, it may be available at some point in the future.

31. Model (1/5) • The model has two key parameters, fand a. • a fraction f of the nodes storing file data fail permanently per unit time. • at any given time, the node storing data is available with some probability a. • The expected availability and maintenance bandwidth of various redundancy schemes can be computed to maintain a file of M bytes.

32. Model (2/5) • Replication • redundancyR replicas • store total R M bytes • replace f RM bytes per unit time • the file is unavailable if no replica is available • probability • Ideal Erasure Codes • n = kR, redundancy Rn / k • transfer just M / k bytes each packet • replace fRM bytes per unit time • unavailability probability

33. Model (3/5) • Hybrid • n = k(R− 1) • store total RM bytes • transfer fRM bytes per unit time • The file is unavailable if the replica is unavailable and fewer than k erasure-coded packets are available • probability

34. Model (4/5) • Minimum-Storage Regenerating Codes • store total RM bytes • redundancy Rn / k • replace fRMbytes per unit time • extra amount of information • unavailability

35. Model (5/5) • Minimum-Bandwidth Regenerating Codes • store total M n bytes • redundancy Rn / k • replace f M n bytes per unit time • extra amount of information • unavailability

36. Estimating f and a

37. Quantitative Results (1/2)

38. Quantitative Results (2/2)

39. Quantitative Comparison • Comparison With Hybrid • Disadvantage : asymmetric design • MBR codes • Disadvantage : • reconstruct the entire file, requires communication with n1 nodes • if the reading frequency of a file is sufficiently high and kis sufficiently small, this inefficiency could become unacceptable.

40. Outline • Introduction • Background • Analysis • Evaluation • Conclusion

41. Conclusion • This paper presented a general theoretic framework that can determine the information. • communicate to repair failures in encoded systems. • identify a tradeoff between storage and repair bandwidth. • One potential application area for the proposed regenerating codes is distributed archival storage or backup. • regenerating codes potentially can offer desirable tradeoffs in terms of redundancy, reliability, and repair bandwidth.