1 / 15

Cognitive Publish/Subscribe for Heterogeneous Clouds

Cognitive Publish/Subscribe for Heterogeneous Clouds. Šarūnas Girdzijauskas , Swedish Institute of Computer Science (SICS) sarunas@sics.se Joint work with: Fatemeh Rahimian (SICS). Future Clouds?. Based on decentralized architecture

abril
Download Presentation

Cognitive Publish/Subscribe for Heterogeneous Clouds

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Cognitive Publish/Subscribe for Heterogeneous Clouds ŠarūnasGirdzijauskas, Swedish Institute of Computer Science (SICS) sarunas@sics.se Joint work with: FatemehRahimian (SICS)

  2. Future Clouds? • Based on decentralized architecture • Abundance of networked collection of connected devices forming micro-clouds • Decentralized Publish/Subscribe service • Content distribution • IP TV Streaming • Online gaming • Collaborative editing • Etc.. • Adapting to the topology and network dynamics of microclouds • Adapting to different usage patterns ŠarūnasGirdzijauskas, Cloud Futures, Redmond, April 2010

  3. Pub/Sub Systems: Our Focus Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010 • Scalable pub/sub service • Very large number of nodes • Very large number of “topics” • Heterogeneous environments • Arbitrary geographical distribution • Arbitrary subscription and dissemination patterns • Central solutions will not scale

  4. Pub/Sub Systems: Our Focus (2) Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010 • Tradeoffs: • Node degree • Number of uninterested (relay) nodes involved • Dissemination delay • Dissemination cost • Cognitive pub/sub: • Fixed node degree • Account for the underlying topology (bandwidth & cost) • Minimize the number of relay nodes by exploiting user subscription correlation & event publication rates

  5. Conceptual Architecture Pub/sub Dissemination structures Cognitive Overlay Physical Network Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  6. Gossip based pub/sub • Gossip (epidemic) overlays • A lot of research (e.g., Cyclon, T-man) • Lightweight, scalable and robust mechanism • Cyclic/Periodic, pair-wise interaction between peers (bounded amount of information) Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  7. Towards Cognitive Structure • Gossiping enables us to find and cluster peers with similar interests connected by cheap and fast links • A node starts with a local fixed size view in a random network • Performs a bidirectional exchange of the view with a random node  2 views • Keeps the only the preferred (ranking function) nodes in the view  1 view • Repeat ŠarūnasGirdzijauskas, Cloud Futures, Redmond, April 2010

  8. Building Cognitive Structure Gossiping $ Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010 • Making clusters by utilizing ranking function which prefers neighbors with similar interests • Peer interest similarity metric • Node subscriptions s1, s2 ⊆ T • sim(s1, s2) =|s1∩ s2|/|s1∪ s2| • Weighted by link cost (bandwidth and $) • Weighted by Topic publication rates • Number of neighbors is limited! • Decided locally on each peer

  9. Problem: How to publish? • Clustering peers of similar interests into bandwidth and cost effective clusters • Clusters might (will) be disjoint • Event publishing requires connected components for each topic Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  10. 14 12 24 7 32 13 10 34 Navigable Small-World Network 9 6 20 5 29 16 18 2 31 21 8 15 22 28 3 4 1 23 27 19 35 17 26 30 11 Inter-Cluster Connectivity • Structure is added: Navigable Small-World topology • Purely by using gossiping Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  11. Building Navigable Structure 2 5 8 1 4 3 7 6 9 10 Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010 • Every peer decides on random ID • Updating ranking function for choice of neighbors: • Ring Link(s) • Long-Range link (Small-World style) for polylogarithmic routing performance

  12. 14 12 24 7 32 13 10 34 Navigable Small-World Network 9 6 20 5 29 16 18 2 31 21 8 15 22 28 3 4 1 23 27 19 35 17 26 30 11 Inter-Cluster Connectivity • Structure is added (Navigable Small-World made by gossiping) • Ring Links • Long-Range (finger) link(s) • Clustering (friend) links • Clusters are connected by greedy routes • Rendezvous node for each topic • All links are used! • All topics become connected • For publishing “flood the topic”, or • Choose a rendezvous node to publish Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  13. Ongoing work • Synthetic data sets for user subscription correlation • Twitter data set • Skype churn data • Our experiments show: • Up to 10 fold reduction of relay traffic as compared to existing approaches (e.g., Scribe, Bayeux) Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  14. Gossip based pub/sub (recap) • Large scale pub/sub for heterogeneous environments • Dissemination structures are self-organizing • Forming clusters of similar nodes • Converging into least expensive dissemination paths on the underlying physical network • Continuously adapting to the environment conditions • Fast convergence, robustness to churn and failures. Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

  15. Thank you!Questions? Šarūnas Girdzijauskas, Cloud Futures, Redmond, April 2010

More Related