Chord - A Distributed Hash Table

1. Chord - A Distributed Hash Table Yimei Liao

2. Outline • Lookup problem in Peer-to-Peer systems and Solutions • Chord Algorithm • Consistent Hashing • Scalable Key Location • Node joins • Stabilization • Summary

3. Peer-to-Peer Systems • Peer-to-Peer System: self-organizing system of equal, autonomous entities (peers) • Decentralized resource usage • Decentralized self-organization • Where to store? where to get? • Solutions • Centralized servers • Flooding search • Distributed hash tables

4. Solutions to lookup problem • Centralized servers • Maintain the current location of data items in a central server • Central server becomes crucial • Best for simple and small applications • Flooding search • Broadcast a request for an item among the nodes • No additional routing information • High bandwidth consumption

5. Solutions to lookup problem • Distributed hash tables • A global view of data distributed among many nodes • Mapping nodes and data items into a common address space • Each DHT node manages a small number of references to other nodes • Queries are routed via a small number of nodes to the target node • Load for retrieving items should be balanced equally among all nodes • Robust against random failure and attacks • Provides a definitive answer to a query

6. Chord Algorithm – Consistent Hashing • Supports just one operation: given a key of an item, it maps the key onto a node where the item is located • Consistent Hashing • Assign each node and key an m-bit identifier using a base hash function such as SHA-1 • Identifiers are ordered in an identifier circle modulo 2m (Chord ring) • Key k is assigned to the first node whose identifier is equal to or follows k: succ(k) = the node with the smallest id  k

7. 6 0 1 7 6 2 5 3 4 2 Chord Algorithm – Consistent Hashing identifier space : m=3 node key 0 1 3 1 1 2 6 identifier circle

8. 0 1 7 6 2 5 3 4 Chord Algorithm – Simple Key Lookup Node 0 sends a query for key 6 • Simple Key Lookup successor(0) = 1 • Queries are passed around the circle via successor pointers • Requires traversing All nodes to find the appropriate mapping 1 successor(1) = 3 6 successor(6) = 0 successor(3) = 6 2

9. Chord Algorithm – Scalable Key Location • Finger Table • Each node n maintains a routing table with up to mentries • The ith entry in the table at node n contains theidentifier of the firstnode s that succeeds n by at least 2i-1 on the identifier circle.(s = succ(n+2i-1)) • s is called the ith finger of node n • Definition of variables for node n

10. Chord Algorithm – Scalable Key Location 0 1 7 6 2 5 3 4 Finger table m = 3, each node n maintains at most 3 entries finger table keys 0+20 0+21 0+22 1 2 4 [1,2) 3 finger table keys [2,4) 3 [4,0) 6 5 6+20 6+21 6+22 7 0 2 [7,0) 0 [0,2) 0 [2,6) 3 finger table keys 1 3+20 3+21 3+22 4 5 7 [4,5) 6 2 [5,7) 6 [7,3) 0

11. Chord Algorithm – Scalable Key Location • Query • Upon receiving a query for key id, a node • Check whether it stores the item locally • If not, forwards the query to the largest node in its successor table that does not exceed id

12. 0 1 7 6 2 5 3 4 Chord Algorithm – Scalable Key Location finger table keys id=5 n=7 finger table keys 6 Successor 3 Predecessor 7 3 Successor 0 Predecessor 4 finger table keys 1 succ(5) = 7 finger table keys 4 2 4 7 7 4 Successor Predecessor 0 7 Successor Predecessor 3 O(logN) pred(k)?

13. Chord Algorithm - Node joins • Invariants to preserve • Each node’s successor is correctly maintained • For every key k, node succ(k) is responsible for k • It is desirable for the finger tables to be correct • Tasks to be performed by Chord • Initialize the predecessor and fingers of node n • Update the fingers and predecessor of existing nodes to reflect the addition of n • Notify the higher layer software so that it can transfer state associated with keys that node n is now responsible for

14. Chord Algorithm - Node joins 0 1 7 6 2 5 3 4 Initializing fingers and predecessor finger table keys find_succ(6); finger table keys 6 4 Successor 3 Predecessor 7 Successor 0 Predecessor 3 5 finger table keys 1 2 finger table keys Successor 7 Predecessor 0 7 7 3 7 Successor Predecessor 3

15. Chord Algorithm - Node joins 0 1 7 6 2 5 3 4 Updating fingers of existing nodes finger table keys finger table keys 3 3 6 5 4 Successor 3 Predecessor 7 3 Successor 0 Predecessor 3 5 finger table keys 1 5 2 finger table keys 5 5 Successor 7 Predecessor 0 7 7 3 P = find_pred(n-2i-1) i = 1, P = find_pred(4) i = 2, P = find_pred(3) i = 3, P = find_pred(1) O(log2N) 7 Successor Predecessor 3

16. Chord Algorithm - Node joins 0 1 7 6 2 5 3 4 Transferring Keys finger table keys finger table keys 6 5 4 Successor 3 Predecessor 7 Successor 0 Predecessor 3 5 finger table keys 1 5 2 finger table keys 5 5 Successor 6 Predecessor 0 7 7 3 7 Successor Predecessor 3

17. Chord Algorithm - Node joins 0 1 7 6 2 5 3 4 Query without all finger tables been updated finger table keys succ(4) = 7 finger table keys 6 7 Successor 3 Predecessor 7 Successor 0 Predecessor 3 5 finger table keys 1 7 2 finger table keys 7 7 4 Successor 6 Predecessor 0 7 7 3 7 Successor Predecessor 3

18. Chord Algorithm - Stabilization • Stabilization • Correctness and performance • Keep node‘s successor pointers up to date • Use successor pointers to verify correct finger table entries • Periodically node n 1) asks its successor, n’, about its predecessor n’’; 2) If n’’ is between n’ and n, then let n->successor = n’’ and repeat step 1) until we reach a point where predecessor of succ(n) = n

19. Chord Algorithm – Node Failure • Node Failure • Successor-list • If successor fails, replace it with the first live entry in the list • Later run stabilization to correct finger table and successor-list

20. Summary • Characteristics of Chord • Load balance distributed hash table • Decentralization fully distributed • Scalability cost of lookup grows logarithmic • Availability automatically adjusts internal tables • Flexible naming no constraints on the structure of the keys

21. References • I. Stoica, R. Morris, D. Karger, F. Kaashoek, and H. Balakrishnan. Chord: A scalable Peer-To-Peer lookup service for internet applications. In Proceedings of the 2001 ACM SIGCOMM Conference, pages 149–160, 2001. • R. Steinmetz, K. Wehrle (Edt.): "Peer-to-Peer Systems and Applications", LNCS 3485, Springer, Chapter 7-8, 2005. • http://www.wikipedia.org • http://www.google.com

22. Thank You Questions?