1 / 18

Small-Scale Peer-to-Peer Publish/Subscribe

This study explores the benefits and performance of small-scale publish/subscribe systems using distributed hash tables. It examines load balancing, network efficiency, and the limitations of current assumptions in DHT applications.

caldwell
Download Presentation

Small-Scale Peer-to-Peer Publish/Subscribe

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Small-Scale Peer-to-Peer Publish/Subscribe Vinod Muthusamy, Hans-Arno Jacobsen University of Toronto July 17, 2005 P2PKM 2005San Diego, CA

  2. Motivation • Many distributed hash table (DHT)-based peer-to-peer applications • Storage (OceanStore, Past) • Database (PIER) • Naming service (Chord-DNS) • Publish/subscribe (Scribe, P2P-ToPSS, Meghdoot) • Typically designed for and evaluated on a large-scale • Small-scale performance is important • Even large systems start small • Benefits even on small scale • Small-scale case study of a DHT-based pub/sub protocol Small-Scale P2P Publish/Subscribe

  3. Agenda • Background • Publish/subscribe model • Distributed hash tables • Small-scale benefits • Publish/subscribe on DHT • Evaluation • Conclusions Small-Scale P2P Publish/Subscribe

  4. Notification Notification Publish/subscribe model TSX Stock markets NASDAQ • Simple interface • Decoupled interaction • Network efficiency NYSE Publisher Publisher AMGN=58 Publications IBM=84 ORCL=12 JNJ=58 HON=24 INTC=19 MSFT=27 BrokerNetwork Subscriptions Subscriber Subscriber Subscriptions: IBM > 85 ORCL < 10 JNJ > 60 Small-Scale P2P Publish/Subscribe

  5. Distributed hash tables • Example: Pastry • Distributed version of a hash table data structure • Store (key, value) pairs • Each node receives at most K/N keys • Each node has about O(logN) routing state • Lookups resolved with O(logN) hops Small-Scale P2P Publish/Subscribe

  6. Small-scale benefits Small-Scale P2P Publish/Subscribe

  7. Service oriented architecture (SOA) • An illustration of small-scale benefits • SOA: a distributed architecture • Loosely coupled reusable components • Enterprise service bus (ESB) comm. fabric • Can be realized as a pub/sub broker network • Pub/sub support in IBM Websphere, Sonic ESBs • Example • Three-tier Web server • Communication between components through pubs and subs • Decoupled interaction • Declarative bindings • Simple to add a Monitor component • Incremental service deployment • Late binding Application Server Enterprise Service Bus User Products Monitor Database Database Small-Scale P2P Publish/Subscribe

  8. Pub/sub case study • P2P-ToPSS • A pub/sub system built over DHT • Large-scale performance previously studied [under submission] • Evaluate small-scale benefits (of the same algorithm) now • Subscription load balance Small-Scale P2P Publish/Subscribe

  9. Subscription Publication price < 50 weight > 50 price = 10 weight = 70 Distributed multidimensional matching (DMM) 1) Pub/sub domain: 3) Network domain: - 0 1 2) Spatial domain: 100 00 01 10 11 01 11 0 1 weight 00 10 DHT network used to map from tree nodes to peers 0 0 price 100 Small-Scale P2P Publish/Subscribe

  10. peer-id AF3483BQ1 hash Region weight price name/code z-code p_w_1101 1101 Distributing the tree • Let region r have z-code z, and tree node n(r) • Store n(r) at peer p(r), where p(r) is where DHT would store key z • Each peer performs mapping locally • Hash function ensures even distribution Network Subscription price = … weight = … p/w Small-Scale P2P Publish/Subscribe

  11. Load balancing • Peers can be overloaded due to • Excessive subscriptions storage • Excessive publication traffic • Overloaded peer delegates subscriptions to children in the DMM tree • A local decision Network 0 00 01 p/w Small-Scale P2P Publish/Subscribe

  12. Evaluation • SimPastry simulator (Microsoft Research) • Pastry DHT • Simulate network latencies, message costs • Nodes connected in a LAN • 1ms latency between every pair of nodes • Subscriptions • Attribute name + range • E.g.: “20 < weight < 40” • Metric • Storage load of subscriptions Small-Scale P2P Publish/Subscribe

  13. DHT performance • 1000 random keys • DHT successfully balances load Small-Scale P2P Publish/Subscribe

  14. Pub/sub performance weight price • 1000 fine-grained subscriptions • Load balancing distributes storage load Small-Scale P2P Publish/Subscribe

  15. Pub/sub performance weight price • 1000 coarse-grained subscriptions • Load balancing distributes but also increases storage load Small-Scale P2P Publish/Subscribe

  16. Observation • Load balancing technique assumes available resources somewhere in the network • Leads to runaway subscription delegation • Common assumption in DHT applications • CAN DHT’s active caching and replication • Meghdoot’s hotspot circumvention • In DHT, no knowledge of aggregate available resources • Fundamental assumption of DHT applications not valid in small-scale networks Small-Scale P2P Publish/Subscribe

  17. Conclusions • Pub/sub and DHTs • Proven benefits in large-scale network • Untapped benefits in small-scale networks • Common assumption on DHT applications • Sufficient available resources in the network • Small-scale networks break this assumption • Future work • Implement algorithms to estimate global resource availability • Curb runaway subscription delegation with this information • Evaluate other DHT applications in small-scale networks Small-Scale P2P Publish/Subscribe

  18. Q&A Small-Scale Peer-to-Peer Publish/Subscribe www.msrg.toronto.edu Small-Scale P2P Publish/Subscribe

More Related