1 / 40

Finding Nearby Objects in Peer-to-Peer Networks

Finding Nearby Objects in Peer-to-Peer Networks. Kirsten Hildrum. Object Location. The Big Picture. Behind the Cloud. ?. One directory One participant stores a lot, but average is low Stretch is high Everyone has a directory Low stretch! Everyone has lots of data

aya
Download Presentation

Finding Nearby Objects in Peer-to-Peer Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum

  2. Object Location

  3. The Big Picture

  4. Behind the Cloud ?

  5. One directory One participant stores a lot, but average is low Stretch is high Everyone has a directory Low stretch! Everyone has lots of data Changes are very expensive Two solutions

  6. Distribute directory Each peer is responsible for part of directory Q: Who is responsible for my object? A: Hash peers into same namespace as objects. Q: How do I get there? A: Peer-to-peer networks route to names.  Must efficiently self-organize into network

  7. Efficiency is… With respect to network • Few neighbors • Short routes (few hops & low stretch) • Easy routing (No routing tables!) • Dynamic With respect to directory • Not too many entries hard (4,4) easy

  8. Outline • Low stretch dynamic peer-to-peer network • Tolerate failures in network • Adapting to network variation • Related results • Future work

  9. Distributed Hash Tables • These systems give • Guaranteed location • Join and leave algorithms • Load-balanced storage • No stretch guarantees

  10. Low Stretch Approaches • Not dynamic Result #1: Make a dynamic low-stretch scheme

  11. PRR/Tapestry Country State City

  12. PRR/Tapestry Level 3 Level 2 Level 1 Two object types: red and blue, so two trees

  13. Neighbor Table For “5471” (Octal) NodeID 5416 NodeID 5455 5470 540x 50xx 0xxx 5471 541x 51xx 1xxx 5472 542x 52xx 2xxx 3 3 5473 543x 53xx 3xxx Ø 544x 54xx 4xxx 3 5475 545x 55xx 5xxx NodeID 5432 NodeID 5471 2 Ø 546x 56xx 6xxx 5477 547x 57xx 7xxx 4 4 3 2 1 2 Routing Levels 2 NodeID 5470 NodeID 5061 Balancing Load 1 NodeID 5123

  14. Building It • Every node is a root! • Can extend to more than two colors • Pefix routing in base b  promoting at rate 1/b • b=16  match one hex digit or four bits at a time • Space: O(b logb n) • Every node at every level stores b parents We want the links to be of geometrically increasing length

  15. Node Joins • Correctness O(log n) insert & delete • Concurrent inserts in a lock-free fashion • Keeping low stretch • Required a nearest-neighbor search algorithm as subroutine • Keep O(1) stretch in special metrics • Algorithms implemented in version of Tapestry used in OceanStore

  16. Concurrent Insert

  17. Solving the Problem Theorem 1 When peer A is finished inserting, it knows about all relevant peers that have finished insertion. Hildrum, Kubiatowicz, Rao & Zhao, “Distributed Object Location in a Dynamic Network” SPAA2002 Sequential Consistency (without locks) A B

  18. Node Join in Practice 150 128 203 ~210 Constant churn largegroup join killnodes RouteToID operations on 40 node cluster Churn: nodes join/leave every 10 seconds, average lifetime = 2mins

  19. Dynamic challenge: find parent • Need to find closest at level • Idea Use routing structure backwards • Other approaches • Clarkson ‘97 • Karger & Ruhl ‘02 • Krauthgamer Lee ‘03

  20. 3r r Growth Restriction • Growth Restriction: • Ball of radius 3r contains only a factor of c more nodes than ball of radius r. • Stronger than “polynomial growth” • Base b at least c • Promote less than 1/c

  21. new Algorithm Idea, in pictures

  22. 3r The Trimming Lemma • Circle of radius r around new with at least one level i node. • To find level (i-1) must find its parent. • Level (i-1) node in little circle must point to a level i node in 3r of new <2r r

  23. How Many? • Keep fixed k= O(log n) • If k red nodes in little circle, then about ck in big circle • Expect only about ck/b blue nodes • If c/b < 1, expect less than k 3r r log n per level & log n levels, so total is O(log2 n)

  24. Summary In growth-restricted networks (with no additional space!): Theorem 2 [HKRZ02] Can find nearest neighbor with high probability with O(log2 n) messages Theorem 3[HKMR04]Can find nearest neighbor, and messages is O(log n) with high probability + more dynamic, low-stretch network

  25. Outline • Low stretch dynamic peer-to-peer network • Tolerate failures in network • Adapting to network variation • Related results • Future work

  26. Behind the Cloud Again

  27. Dealing with faults • Multiple paths • Castro et. al • One failure along path, path breaks • Wide path • Paths faulty at the same place to break • Exponential difference in width effect • “retrofit” Tapestry to do latter in slightly malicious networks Failed! Still good…

  28. Effective even for small overhead Theorem 4In growth restricted spaces, can make probability of failed route less than 1/nc for width O(clog n) Hildrum & Kubiatowicz, DISC02

  29. Wide path vs. multiple paths

  30. Outline • Low stretch dynamic peer-to-peer network • Tolerate failures in network • Adapting to Network Variation • Related results • Future work

  31. Network not homogeneous Previous schemes assumed global growth rate. But what if there isn’t one? Nebraska San Francisco Paris

  32. 3r r Using the local c • Idea: Determine local c and use it • Challenge: Paths still meet up, stretch still low, and space reasonable • Technique: “Fix” or match bits • In variable size chunks • PRR97, LAND, Tapestry, Pastry matched bits in fixed size chunks

  33. Summary • Previous approaches assumed a global c • Had to know cin advance of construction Hildrum, Krauthgamer & Kubiatowicz [SPAA04]: Dynamic, locally optimal low-stretch network

  34. Outline • Low stretch dynamic peer-to-peer network • Tolerate failures in network • Adapting to network variation • Related results • Future work

  35. Congestion • Latency might not be right measure • Reduce bandwidth usage • When demands are known, can use linear program to find a set of paths • But what if demands are not known…

  36. Summary • Räcke02 showed scheme exists O(log3 n) • Azar, Cohen, Fiat, Kaplan, & Räcke in STOC03 find optimal routing, but not hierarchical Theorem 6 Hierarchical scheme can be constructed in polynomial time. (Within O(log2 n log log n) of optimal) • Harrelson, Hildrum, & Rao, “A Polynomial-time Tree Decomposition to Minimize Congestion”SPAA03

  37. Online Auctions for Digital Goods Online Answer bidder i before seeing the next bid • Bob offers you 50¢, yes or no? • Don’t get to see Alice’s bid before answering Digital Each additional copy is free • Want to sell to everyone at their bid • Can’t sell to everyone at their bid, or else all bids will be low Bar-Yossef, Hildrum, Wu, “Online Auctions for Digital Goods” SODA 2002: Theorem 7 Constant competitive ratio when opt makes O(maxBid log log maxBid) Theorem 8 When best fixed price sells few, can get:

  38. Outline • Low stretch dynamic peer-to-peer network • Tolerate failures in network • Adapting to network variation • Related results • Future work

  39. Future Work • Object location questions • Tradeoff between degree, stretch, load balance as function of growth rate • Smooth transition between general-metric systems and growth-restricted systems • Distributed low-congestion network • Peer-to-peer networks with selfish players • Sensor networks

  40. Thank You!

More Related