Samsara: Honor Among Thieves in Peer-to-Peer Storage

1. Samsara: Honor Among Thieves in Peer-to-Peer Storage Landon P. Cox and Brian D. Noble University of Michigan

2. Samsara • From Wikipedia, the free encyclopedia • Saṅsāra or Saṃsāra (Sanskrit: संसार) • Literally means "continuous flow" • Is the cycle of birth, life, death, rebirth or reincarnation within many Eastern religions

3. Paper overview • Proposes an incentive mechanism motivating participants in a P2P distributed file system to contribute as much space as they consume • Addresses the tragedy of the commons • Requires each peer that requests storage from another peer to hold a claim for same amount of storage • Claims can be exchanged

4. The tragedy of the commons • Assume a group of herders that a common pasture, on which they are entitled to let their cows graze • To maximize his/her personal benefit, each herder will put as many cows as it can on the common pasture • As a result, the common pasture becomes overgrazed and useless • Happened to the Boston Common

5. Boston common

6. Introduction • P2P file systems have many advantages • Require users to consume storage according to their contribution • Otherwise system will collapse • Solution is a mechanism enforcing"storage fairness" • Incentive mechanism

7. Extant solutions • A trusted third-party enforcing quotas • Requires a centralized administration • Letting people buy and sell storage space • Requires a trusted clearance infrastructure • Using certified identities and trusted keys • Requires a trusted certification authority • Enforcing total symmetry within pairs of peers • Unpractical

8. Samsara key idea (I) • Manufacture symmetric relations • through claim forwarding • All exchanges of data for claims form symmetric contracts • Each node periodically checks the other for compliance • Done in a probabilistic fashion • When a node breaches the contract, other node is free to drop the data of its partner

9. Samsara key idea (II) • Nodes can forward claims rather than honoring them • Still remain responsible for the claims they have forwarded • Mechanism penalizes unresponsive nodes in a probabilistic fashion • A node suffering a short outage may lose some replicas of its data

10. Background • Samsara is an add-on to Pastiche a P2P cooperative backup system • To be discussed later • Built itself on top of Pastry network

11. Overall design • Objective is equal exchange • If A stores data for B then B must store an equal-size claim for B • If B discards A’s claim then A can discard B’s data • Equal exchange is enforced by periodic queries • Not answering a query is a sufficient reason to have you data dropped

12. The problem • This simple claim model punishes nodes too severely for transient failures • New approach • Is probabilistic • Takes into account transient failures • When a node fails to answer a query, each of is replica sites drops data with some probability

13. Claim construction (I) • Claims are “incompressible placeholders” • Computing a claim requires • a secret passphrase P • asecret symmetric key K • and a location in storage space

14. Claim construction (II) • Assuming we have 512-byte claims • The first claim C0 would contain • Twenty-five 20-bit hashes hi = SHA1(P, i) where P is the secret pass phrase and i the hash index • First 12 bits of next hash in sequence all encrypted with the symmetric key K C0 = {h0, h1, …, first 12 bits of h25}K

15. Claim construction (III) • Successive claims are built using repeating the process C1 = {h26, h27, …, first 12 bits of h51}K Ci = {hj, hj+1, …, first 12 bits of hj+25}K where j = 26i

16. Answering claim queries • Can be done with a single SHA1 hash • Querying party provides • Unique value h0 • List of objects to verify • Responding party • Append h0 to first object O0 in list and compute h1 = SHA1(O0, h0) • Recursively computes hi+1 = SHA1(Oi, hi) • Returns last hj

17. Example (I)

18. Example B has claim β1 on A and B has claim γ1 on B

19. Example Node B does not have enough space to hold claim γ1

20. Example Node B forwards its claim for space on node A to node C

21. Claim forwarding • If a node X • has a claim ξ on another node Y and • owns a claim ζ to a third node Z • It can forward its claim ζ to node Y • Everything works fine until a node fail

22. Failures in dependency chains

23. Failures in dependency chains • Before failure, • B stores data for A, • C stores data for B • … • E stores data for D and hold a claim ε1 on A • When C fails and stop answering queries from B, • B uses it storage rights on A and replaces claim ε1 by its own claim β1

24. Failures in dependency chains • After that we have a cascade of damaging actions • A fails to answer queries from E • E holds D responsible for loss of claim ε1 and discards the data it had stored for D • D loses its backup data on E even though it had always operated in a correct fashion • Forwarding claims increases the risk of data losses

25. Failures in dependency cycles

26. Failures in dependency cycles • The effect of a failure is much less dramatic when we have a dependency cycle, where • B stores data for A, • C stores data for B • … • E stores data for D • A stores data for E

27. Failures in dependency cycles • When C fails and stop answering queries from B, • B uses it storage rights on A and requests it to store its claim β1 • Since A stores data for E, it can forward claim β1 to E • Since E stores data for D, it can forward claim β1 to E • E keeps claim β1 because it has data on E

28. Evaluation • Samsara is faster than scp • Most chain are short as long as there is free space • Great news! • Nodes should forward claims in a very conservative fashion to minimize data losses