1 / 15

Distributed Hash Tables (DHT) Jukka K. Nurminen

Distributed Hash Tables (DHT) Jukka K. Nurminen. *Adapted from slides provided by Stefan G ötz and Klaus Wehrle (University of T übingen ). Distributed Management and Retrieval of Data. I want item „D“. Where can I find „D“?. I have item „D“. Where to place „D“?. Data item „D“. D. ?. ?.

olaf
Download Presentation

Distributed Hash Tables (DHT) Jukka K. Nurminen

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Hash Tables (DHT)Jukka K. Nurminen *Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen)

  2. Distributed Management and Retrieval of Data I want item „D“.Where can I find „D“? I have item „D“.Where to place „D“? Data item „D“ D ? ? ? distributed system 12.5.7.31 peer - to - peer.info planet - lab.org berkeley.edu 89.11.20.15 95.7.6.10 86.8.10.18 7.31.10.25 • Essential challenge in (most) Peer-to-Peer systems? • Location of a data item among systems distributed • Where shall the item be stored by the provider? • How does a requester find the actual location of an item? • Scalability: keep the complexity for communication and storage scalable • Robustness and resilience in case of faults and frequent changes 2 2008-11-05 /Jukka K. Nurminen

  3. The Architectures of 1st and 2nd Gen. P2P 1st Gen. 2nd Gen. 3 2008-11-05 /Jukka K. Nurminen

  4. Addressing in Distributed Hash Tables 3485 -610 611 -709 1008 -1621 1622 -2010 2011 -2206 2207-2905 2906 -3484 (3485 -610) 2m-1 0 H(Node Y)=3485 Often, the address space is viewed as a circle. Y X Data item “D”: H(“D”)=3107 H(Node X)=2906 • Step 1: Mapping of content/nodes into linear space • Usually: 0, …, 2m-1 >> number of objects to be stored • Mapping of data and nodes into an address space (with hash function) • E.g., Hash(String) mod 2m: H(„my data“)  2313 • Association of parts of address space to DHT nodes 4 2008-11-05 /Jukka K. Nurminen

  5. Step 2: Routing to a Data Item • put (key, value) • value = get (key) • Routing to a K/V-pair • Start lookup at arbitrary node of DHT • Routing to requested data item (key) H(„my data“) = 3107 1622 Node 3485 manages keys 2907-3485, 1008 2011 2207 709 ? 611 2906 3485 Key = H(“my data”) Initial node(arbitrary) (3107, (ip, port)) Value = pointer to location of data 5 2008-11-05 /Jukka K. Nurminen

  6. Step 2: Routing to a Data Item • Getting the content • K/V-pair is delivered to requester • Requester analyzes K/V-tuple(and downloads data from actual location – in case of indirect storage) In case of indirect storage: After knowing the actual Location, data is requested H(„my data“) = 3107 Get_Data(ip, port) 1622 1008 2011 2207 709 ? 611 2906 3485 Node 3485 sends (3107, (ip/port)) to requester 6 2008-11-05 /Jukka K. Nurminen

  7. Chord 7 2008-11-05 /Jukka K. Nurminen

  8. Chord: Topology Identifier Node 6 X Key 0 1 7 6 2 5 3 4 2 • Keys and IDs on ring, i.e., all arithmetic modulo 2^160 • (key, value) pairs managed by clockwise next node: successor 1 successor(1) = 1 Chord Ring successor(6) = 0 6 2 successor(2) = 3 8 2008-11-05 /Jukka K. Nurminen

  9. Chord: Primitive Routing 0 1 7 Key 6? 6 2 5 3 4 • Primitive routing: • Forward query for key x until successor(x) is found • Return result to source of query • Pros: • Simple • Little node state • Cons: • Poor lookup efficiency:O(1/2 * N) hops on average(with N nodes) • Node failure breaks circle 6 Node 0 1 2 9 2008-11-05 /Jukka K. Nurminen

  10. Chord: Routing finger table keys i start succ. 6 0 0 1 2 1 2 4 1 3 0 1 finger table keys 7 i start succ. 1 0 1 2 2 3 5 3 3 0 6 2 5 3 finger table keys 4 i start succ. 2 0 1 2 4 5 7 0 0 0 • Chord’s routing table: finger table • Stores log(N) links per node • Covers exponentially increasing distances: • Node n: entry i points to successor(n + 2^i) (i-th finger) 10 2008-11-05 /Jukka K. Nurminen

  11. Chord: Routing lookup (44) lookup (44) = 45 • Chord’s routing algorithm: • Each node n forwards query for key k clockwise • To farthest finger preceding k • Until n = predecessor(k) and successor(n) = successor(k) • Return successor(n) to source of query 63 60 4 56 7 54 52 13 14 49 49 16 19 45 45 44 42 42 23 39 26 37 30 33 11 2008-11-05 /Jukka K. Nurminen

  12. Comparison of Lookup Concepts 12 2008-11-05 /Jukka K. Nurminen

  13. Extra slides

  14. Summary of DHT • Use of routing information for efficient search for content • Self-organizing system • Advantages • Theoretical models and proofs about complexity (Lookup and memory O(log N)) • Simple & flexible • Supporting a wide spectrum of applications • <Key, value> pairs can represent anything • Disadvantages • No notion of node proximity and proximity-based routing optimizations • Chord rings may become disjoint in realistic settings • No wildcard or range searches • Performance under high churn. Especially handling of node departures • Key deletion vs. refresh • Many improvements published • e.g. proximity, bi-directional links, load balancing, etc. 14 2008-11-05 /Jukka K. Nurminen

  15. Different kinds of DHTs • Specific examples of Distributed Hash Tables • Chord, UC Berkeley, MIT • Pastry, Microsoft Research, Rice University • Tapestry, UC Berkeley • CAN, UC Berkeley, ICSI • P-Grid, EPFL Lausanne • Kademlia, Symphony, Viceroy, … • A number of uses • Distributed tracker • P2P SIP • ePOST 15 2008-11-05 /Jukka K. Nurminen

More Related