Symphony : Distributed Hashing in a Small World

1. Symphony: Distributed Hashing in a Small World Gurmeet Singh Manku MayankBawa PrabhakarRaghavan Presented by Satpreet Singh

2. Motivation • GOAL: To maintain a large DHT over a WAN • DESIRED CHARACTERISTICS: • Scalability (work for a range of network sizes) • Stability (handle churn) • Performance (provide low-latency lookups & low maintenance costs with churn) • Flexibility (provide design knobs, preferably run-time) • Simplicity (easy to understand, code, deploy…) • SOLUTION: Symphony (fuses ideas from Chord & Klienberg’s Small World greedy routing algorithm/result)

3. Features/Advantages of Symphony • Low state maintenance (conseq. of low degree) • Fewer pings/keep-alives, less (ambient) control traffic • Distributed locking and coordination overhead over smaller sets of nodes • Smaller bootstrapping time when a node joins • Smaller recovery time when a node leaves • 2. Smooth out-degree vs latency tradeoff • Only protocol that offers this tuning knob even at run time! • Out-degree not fixed at runtime, or as function of network size. • 3. Flexibility and support for heterogeneity • Different nodes can have different number of links • 4. Fault tolerance • Only short links are bolstered. No backups for long links

4. node long link short link A typical Symphony network Architecture: Overview • Establish keyspace as [0, 1) (wrap around a ring like in Chord) • Every node ‘manages’ subrange from own-id. to next-clockwise-node’s-id. (~ equi-sized) • Objects hash to m-bit hash-key K, managed by node that manages real number containing K/2m • 2 Short-links: one with each immediate neighbor

5. node long link short link A typical Symphony network Architecture: Overview • k (≥ 1) Long Links (uni-/bi-direct.) • draws a rand. number (x) from a Probability Distribution Function • contacts manager of (x) using a Routing Protocol • Establishes a link (if incoming links at manager ≤ 2k, if not resample PDF) • PDF is a type of harmonic distribution (so, Symphony) • Pn(x) = 1/(x ln [n]) in x Є [1/n, 1] • = 0 otherwise • PDF estimates [n] using an Estimation Protocol

6. Network Size Estimation Protocol • Goal: to estimate ‘n’ - the current total number of nodes in the DHT • So if, • - S is any set of s distinct nodes, • Xs is the sum of segment-lengths managed by them, • Estimated n = s/Xs • All s nodes update their estimate, • Experimentally s = 3 found good enough, • So simply use node and it’s two immediate neighbors • Fact: Impact of increasing s on avg. latency is insignificant x = Length of arc 1/x = Estimate of n (Idea from Viceroy)

7. Routing Protocol(s) Klienberg’s Small World result: A message can be routed to any node by greedy routing in O(log2n) hops, in a construction where each node has one link to each of it’s 4 directional neighbors and a single long-distance link to a node chosen from a suitable PDF. To lookup hash key xЄ [0,1), contact the manager of x : Unidirectional Routing Protocol: Node forwards a lookup for x along (short or long) link that minimizes the clockwise distance to x Bidirectional Routing Protocol:Node forwards a lookup for x along (short or long) link that minimizes the absolute distance to x In both cases, expected path length in an n-node network with k = O(1) linksis O(1/k log2n) hops. Bidirectional & 1-Lookahead reduce latency by 40% and 30% each

8. Join/Leave Protocols • JOIN: • The new node chooses its id x from [0, 1) uniformly at random • Using the routing protocol it identifies the current manager of x • It then runs the estimation protocol using s = 3 • X then uses Pnx to establish its long distance links • Cost = k links * O(1/k log2n) msgs. = O(log2n) messages • LEAVE: • All out- and in- links to x’s long distance neighbors are snapped • Other nodes whose outgoing links to x are just broken, reinstate those links with other nodes • The immediate neighbors of x establish short-links between themselves • Successor of x initiates estimation protocol over s = 3 neighbors • Cost = O(log2n)messages

9. Re-linking Protocols etc… • RE-LINKING: • nx= x’s current estimate of n • nxlink= x’s estimate when long distance links were last established • When nxand nxlink differ → stale estimate • Re-link only when nx / nxlinkis not in the range [0.5, 2] • Re-linking gains are marginal, cost high: O(log2n)messages • LOOKAHEAD: • Node can maintain list of neighbor’s neighbors • Improves choice of neighbor for routing queries • No extra messages – piggyback on keep-alives of TCP link • Cost = O(k2)space. Number of long-links remains unchanged • FAULT TOLERANCE: • Deletion of short links more detrimental as leads to node isolation • Make f copies of node’s content in f next clockwise nodes

10. Experimental Data: • SETUP: • Large DHT: 25 to 215 nodes simulated in network • Four kinds of test networks: • Static, Expanding, Expanding-Relink & Dynamic Estimate Protocol Performance: estimate improves for log(n) neighbors, but impact on avg. latency is minimal (later)

11. Experimental Data: Routing Protocol Performance: Increasing links beyond 2 has marginal benefits. Bidirectional routing is good (30% reduction in latency)

12. Experimental Data: • Lookahead Performance: • 1-Lookahead reduces avg. latency by 40% for small value of k. • Also, it does not entail an increase in the no. of long-links per node. • Neighbor-lists are exchanged periodically piggy-backed on normal routing traffic or keep-alives

13. Experimental Data: • Fault-tolerance motivation: • [Left] On deleting a random set of links (short + long), successful lookups drops quite quickly [deletion of short links causes node isolation quickly] • [Right] Impact of removing only long links not as severe (and only avg. latency goes up) • Thus, only fortify short links. Make f copies of content in clockwise direction.

14. Comparisons & Conclusions: • Conclusions: • For large DHTs, (25 to 215 nodes) Symphony outperforms others • Avg. TCP links = 10; Latency is about 8 hops; • Lower costs of Join/Leave compared to Chord etc. • Number of neighbors not fixed at outset; No backup links

15. Wrapping up… • Symphony... • is a simple protocol for managing large DHTs • supports a dynamic network of hosts with relatively short lifetimes • scales well • has low lookup latency • has low maintenance cost • requires few neighbors per node • supports heterogenity in nodes (run-time knobs) • provides flexibility in design Questions… ?