Fault-tolerant Routing in Peer-to-Peer Systems

1. Fault-tolerant Routing in Peer-to-Peer Systems James Aspnes Zoë Diamadi Gauri Shah Yale University PODC 2002

2. Peers Resources P2P network Key • Bunch of peers. • Store resources identified by keys. • Peers subject to crash failures. • Goal: locate resources ‘’efficiently’’.

3. Properties of ideal network • Data availability • Decentralization • Fault-tolerance • Scalability • Load balancing • Maintaining network • Dynamic node addition/deletion • Self-stabilization • Efficient searching • Incorporating geography • Incorporating locality

4. Gnutella ? Inefficient flooding Early P2P systems Napster x ? x x Central server bottleneck

5. 427 768 368 123 327 365 135 360 Tapestry [JKZ’01] Uses Plaxton’s Algorithm: Node xyz links to *XX, x*X and xy* [* = all digits, X = any digit] Correct one digit at a time to reach target. Pastry [DR’01] is also similar.

6. CAN [RFHKS’01] Partition d-dimensional co-ordinate space into zones. (0,1) (1,1) zone 3 5 d=2 7 2 8 (1,0) (0,0) Nodes own zones and keys hashed to them. Greedy routing: forward to neighbor closest to target.

7. 0 3 6 Chord [SMKKB‘01] Nodes and resources mapped to identifier circle. Routing table: successor nodes at distances . 0 7 1 successors identifier circle (n=8) 0 0 3 6 2 6 6 0 3 5 4 Greedy routing: forward to node in routing table closest to target

8. Common underlying structure • Underlying metric space. • Nodes embedded in metric space. • Location determined by key. • Hashing to balance load. • Greedy routing. • O(log n) space at each node. • O(log n) routing time.

9. Unifying approach Virtual Route v4 Nodes Keys v2 v1 HASH Physical Link v3 Virtual Link v1 v2 v3 v4 Actual Route PHYSICAL NETWORK VIRTUAL OVERLAY NETWORK

10. Links chosen as per Link Distribution Each node independently selects k long-hop links as per some distribution . x-d1 x Nodes x-d2

11. Abstract model Simple metric space: 1D line. Hash(key) = Metric space location. Short-hop links: immediate neighbors. Long-hops links:inverse-distance distribution. Pr[edge(u,v)] = 1/d(u,v) / Greedy Routing: forward message to neighbor closest to target in metric space. 1/d(u,v’)

12. What do we care about? • Do we get similar upper bounds on routing • time with failures? • Is it possible to design a link distribution • that beats the O(log2n) bound for routing • given by 1/d distribution? • Can we dynamically construct such a network?

13. Greedy routing with failures Analyze message delivery in phases [Kleinberg ‘99]. Phase 0 Phase 1 Target t Phase 2 Message at node n in phase i: 2i d(n, t) < 2i+1 At most (log n + 1) such phases.

14. [1..log n] long-hop links Suppose each node has k long-hop links. Average time spent in each phase: ((log n)/k). With O(log n) such phases: Total time: O((log2n)/k). With failures: Suppose each node/link fails with prob (1-p). Average time spent in each phase: ((log n)/pk). Total time: O((log2n)/pk)

15. Simulation results n=131072 nodes log n=17 links What happens with > log n links?

16. What do we care about? • Do we get similar upper bounds on routing • time with failures? • Is it possible to design a link distribution • that beats the O(log2n) bound for routing • given by 1/d distribution? • Lower bound on routing time as a function • of number of links per node. • Can we dynamically construct such a network?

17. Intuition for lower bound [KUW’88] Time needed for a non-increasing real-valued Markov chain X0, X1, X2…. to drop to 1 bounded by: where = E[Xt –Xt+1: Xt = z] is a non-decreasing function of z.

18. Upper bound on time SFO NYC x z Starting from x, average speed at z = . gives lower bound on average crossing speed. ( is non-decreasing so ) gives upper bound on time.

19. Lower bound on time SFO NYC x z gives upper bound on average crossing speed. mz= sup gives lower bound on time. This may give too large an estimate, so condition against high bursts of speed.

20. Pr[Xt – Xt+1 U : Xt = x] E[Xt – Xt+1 : Xt=x, Xt – Xt+1 < U] Upper bound on speed [no long jumps] mz = sup { : x S, x [z, z+U) } Time from x [no long jumps] E[Time to reach 0] T(X0)/[ T(X0) + (1- )] Tool for lower bound Non-increasing Markov chain: X0, X1, X2 ….., state space S. Few long jumps

21. So use an aggregate chain St of nodes for collective behavior of nodes in some range. Links d3 St+1 St Nodes d2 d1 0 0 Track ln(|St|) for recurrence relation. Applying tool to routing Cannot bound progress of single node with an arbitrary distribution!

22. link ignored 1-sided routing: s link ignored 2-sided routing: * s Lower bounds Random graph G. Node x has k independent links on average. x links to (x-1) and (x+1). Expected time to reach 0 from a Point chosen uniformly from 1..n: * Probability of choosing links symmetric about 0 and unimodal. (ln2n) worse that O(ln n) for a tree: cost of assuming symmetry between nodes.

23. What do we care about? • Do we get similar upper bounds on routing • time with failures? • Is it possible to design a link distribution • that beats the O(log2n) bound for routing • given by 1/d distribution? • Can we dynamically construct such a network?

24. Heuristic for construction New node chooses neighbors using inverse distance distribution. Links to live nodes closest to chosen ones. Selects older nodes to point to it. new link adjusted link initial link ideal link x y older node new node absent node

25. [Aspnes, Shah ’02] submitted to SODA • Design a self-stabilization mechanism. ? Open problems • Does lower bound generalize to multidimensional • metric spaces? • Does backtracking give provably good routing bound? • Analyze security properties such as anonymity • and byzantine failures.