1 / 12

Subscription Partitioning and Routing in Content-based Publish/Subscribe Networks

Subscription Partitioning and Routing in Content-based Publish/Subscribe Networks. Yi-Min Wang, Lili Qiu, Dimitris Achlioptas, Gautam Das, Paul Larson, and Helen J. Wang Microsoft Research DISC 2002 Toulouse, France. Motivation. Network Architecture. Subscription Partitioning.

juan
Download Presentation

Subscription Partitioning and Routing in Content-based Publish/Subscribe Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Subscription Partitioning and Routing in Content-based Publish/Subscribe Networks Yi-Min Wang, Lili Qiu, Dimitris Achlioptas, Gautam Das, Paul Larson, and Helen J. Wang Microsoft Research DISC 2002 Toulouse, France

  2. Motivation

  3. Network Architecture

  4. Subscription Partitioning

  5. Event Space Partitioning for Equality Predicates

  6. Simulation Study with Stock-quote subscription data

  7. Equality Predicates • Hash predicates to get uniform distribution • Treat the hashed domain as the event space • Use Event Space Partitioning • Subscription is a point; does not intersect multiple sub-spaces • Use over-partitioning for better load balancing • Use offline greedy algorithm to assign buckets to servers for load balancing • Use indirection table to dynamically map buckets to servers for load re-balancing • Use Bloom filters to further reduce traffic • Fast detection of true negatives at the expense of (very low) false-positive rate

  8. Summary of simulation results • Actual MSN Money log: 1.48M subscription with 0.29M unique filters over 21,741 stock symbols • Zipf-like distribution: MSFT 58,518, CSCO 43,073, $COMPX 32,485, $INDU 31,111, INTC 25,,903, $INX 22,519, AOL 21,836, LU 20,056, ORCL 18,038, DELL 16,211, etc.

  9. Simulate 100M new subscriptions from 43,734 symbols • Scaled-up Zipf-like distribution • Perturbation and permutation • Uniform distribution • 50 servers with over-partitioning ratio = 10 • Without load re-balancing • Load imbalance (max/min) ranged from 1.41 to 6.66 (Uniform case) • With imbalance threshold of 2.0 • Re-balancing was triggered only 5 times, each time involving re-assignment of up to 3 buckets and migration of up to 0.7% subscriptions.

  10. Filter Set Partitioning for Range Predicates

  11. Related Work

  12. Summary

More Related