Opportunistic Multipath Forwarding in Publish/Subscribe Systems

1. Opportunistic Multipath Forwarding in Publish/Subscribe Systems Reza SherafatKazemzadehANDHans-Arno Jacobsen Middleware Systems Research Group University of Toronto

2. Agenda • The content-based publish/subscribe model • Characteristics and challenges • Our approach • Overlay neighborhoods • Adaptive multipath forwarding • Experimental evaluation results Opportunistic Multipath Forwarding

3. Content-Based Pub/Sub Model P P P P Publish P P Pub/Sub S S S S S P S S S S Subscribers Publishers Opportunistic Multipath Forwarding

4. Content-Based Pub/Sub Model • Many to many communication between a large number of publishers/subscribers • Selective delivery based on subscription matching • Unicast (one) • Multicast (some) • Broadcast (all) • Variations in traffic patterns makes it difficult to design an optimal overlay network P P Publish P Pub/Sub S S Subscribers Publishers Opportunistic Multipath Forwarding

5. Current Pub/Sub Overlays Forwarding paths in the overlay are constructed in fixed end-to-endmanner (no/little path diversity) This results in a high number of “pure forwarding” brokers P P ✗ P ✓ ✗ ✗ S A B ✓ C D E #msgs delivered #msgs sent System Yield = = 1/5 = 20% Opportunistic Multipath Forwarding

6. Summary of Our Goals • Construct a highly connected overlay mesh that provides high path diversity between publishers and subscribers • Avoid pure forwarders by allowing brokers to make fine-grained forwarding decisions based on individual publications and their matching sets • Improve system yield, efficiency, scalability and delivery delay • Support dynamic adaptive routing based on live traffic patters while avoiding high costs of full overlay reconfiguration Opportunistic Multipath Forwarding

7. Forwarding Strategies • Fixed end-to-end (baseline) Total msgs: 6 • Forwarding strategy 1 Total msgs: 5 • Forwarding strategy 2 Total msgs: 3 p * * * * * * * * * * * * * * * A A A B B B C C C D D D * * * p p Opportunistic Multipath Forwarding

8. Our Approach in a Nutshell Δ=3 Δ=2 Δ=1 A S Opportunistic Multipath Forwarding

9. Path Computation forForwarding Strategies Strategy 1 Strategy 2 * * * * * * * * * * * * * * A A A A B B C C C C D D * * * * p B B D D p Opportunistic Multipath Forwarding

10. Experimental Evaluations • We have implemented the algorithms and performed large-scale experimental evaluations with up to 500 brokers • Datasets • Synthesized based on Zipf distribution • Social networking traces from Facebook • We measured performance of the system in terms of: • Overlay mesh connectivity • Delivery delay • Maximum system throughput • System yield • Publication propagation path length • Memory and CPU overhead Opportunistic Multipath Forwarding

11. Path Diversity in Overlay Mesh Number of Available paths Network size:250Delta=3 Path diversity: 10% of brokers w/ 1000 paths Network size:120Delta=3 Path diversity: 20% of brokers w/ 100 paths Graph is based on a snapshot of the state of links in a running system Opportunistic Multipath Forwarding

12. 115% throughput enhancement Strategy 2 Fixed-end-to-end Publication Hop Count Experiment setup: 120 Brokers Publish rate is 1,800 msgs/sec and number of deliveries: 73,000 (in 5 min) Opportunistic Multipath Forwarding

13. Conclusions • Brokers build a highly connected overlay mesh and make fine-grained forwarding decisions for each publication in order to avoid pure forwarding neighbors • We used the notion of overlay neighborhoods to enable local traffic profiling and avoid high costs of overlay reconfiguration • Our approach enhances system’s efficiency and yield, and ultimately improves its scalability and performance Opportunistic Multipath Forwarding

14. Thanks Questions! Opportunistic Multipath Forwarding

15. Pub/Sub Characteristics • Many to many communication between a large number of publishers/subscribers • Selective delivery based on subscription matching • Unicast (one) • multicast (some) • broadcast (all) • Traffic patterns depends on workload application and may change over time Client Broker Opportunistic Multipath Forwarding

16. Related Work: Overlay Reconfiguration Broker overlay is “reconfigured” by addition and removal of links between brokers Advantages • Forwarding path may be improved bringing some publishers and subscribers closer together Disadvantages • Some forwarding paths between publishers and subscribers may indeed suffer • Resulting overlay still relies on fixed end-to-end paths • Reconfiguration is costly and requires full or partial re-propagation of subscriptions Re-configure     [1] Virgillito, A., Beraldi, R., Baldoni, R.: On event routing in content-based publish/subscribe through dynamic networks, FTDCS ‘03[2] Virgillito, A., Beraldi, R., Baldoni, R.: On event routing in content-based publish/subscribe through dynamic networks. In: FTDCS. (2003) Opportunistic Multipath Forwarding

17. Links Management Overlay Network • Links types • Primary links • Soft links – shortcut links • Candidate links – expeditionary links • Traffic profiling • In intervals of T sec, brokers count thenumber of pubs sent over each links • Gain function gain(A,B,T) = pub_traffic_during_T * dist(A,B) • Soft link selection • Profiling input • Link state measured • Broker load information exchanged Soft link Primary link Candidate link C B A B D Link quality Neighbor load High rank Links ranking high → low A B C D E F G E F … Opportunistic Multipath Forwarding

18. Master v. Working Routing Data Structures Master Routing Tables • Overlay map captured by brokers’ Δ-neighborhoods are relatively static Master overlay Map (MOM) • Brokers link connectivity change dynamically, brokers need an efficient way to compute forwarding paths over the changing set of links  Working Overlay Map (WOM) • MOM is a concise representation of the primary overlay that only contains brokers with a direct link construct Working Routing Tables Opportunistic Multipath Forwarding

19. Master v. Working Subscription Tables Working Overlay Map Master Subscription Table • Similar to WOM, brokers adapt their subscription tables based on the current set of available links: Working Subscription Table (WST) Working Subscription Table Between Brokers Sets Beyond Brokers Sets Behind Brokers Sets Opportunistic Multipath Forwarding

20. System Yield (measure of efficiency) Opportunistic Multipath Forwarding

21. Publication Hop Count Experiment setup: 120 Brokers Sparse and dense workloads Publish rate of 1,800 msgs/sec  Deliveries: 73,000 in 5 min Opportunistic Multipath Forwarding

22. Publication Hop Count Experiment setup: 120 Brokers Sparse publication/subscription workload Publish rate of 1,800 msgs/sec  Deliveries: 73,000 in 5 min Opportunistic Multipath Forwarding

23. Publication Hop Count Sparse Matching Workload Dense Matching Workload Multi-path forwarding is more effective in sparse workloads Opportunistic Multipath Forwarding

24. Impact of Broker Fanout on Subscription Covering Experiment setup: 500 brokers Fanout of 5-25 Opportunistic Multipath Forwarding