1 / 74

The Publish/Subscribe Communication Paradigm and its Application to Mobile Systems

The Publish/Subscribe Communication Paradigm and its Application to Mobile Systems . Roberto Baldoni Dipartimento di Informatica e Sistemistica Università degli Studi di Roma “La Sapienza”. Outline. Introduction Basics of pub/sub communication systems Subscription models

Anita
Download Presentation

The Publish/Subscribe Communication Paradigm and its Application to Mobile Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Publish/Subscribe Communication Paradigm and its Application to Mobile Systems Roberto Baldoni Dipartimento di Informatica e Sistemistica Università degli Studi di Roma “La Sapienza” MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  2. Outline • Introduction • Basics of pub/sub communication systems • Subscription models • Architectural models • Event Routing • Impact of mobility MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  3. Introduction: The Publish/Subscribe Communication Style Publish/Subscribe Middleware Publisher Subscriber Publishers publish notifications withinteresting information Subscribers register subscriptionscharacterizing information interests Middleware delivers notifications tomatching subscribers Middleware determines whichsubscriptions match which notifications MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  4. Inroduction The Publish/Subscribe Paradigm • Publish/Subscribe (pub/sub): a powerful abstraction for building distributed applications • Message-based, anonymous communication • Participants are decoupled • in space: no need to be connected or even know each other • in flow: no need to be synchronized • in time: no need to be up at the same time • Good solution for highly dynamic, decentralized systems (e.g., wired environments with huge numbers of publishers and subscribers, MANETs, P2P etc) • Many research issues, involving several research areas (e.g., systems, software eng., databases etc) MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  5. Introduction • Widely popular both in the industry and in the research • Industry solutions for pub/sub are almost centralized • Research has mainy to face the scalability aspect • Applications • Stock information delivery • Auction system • Air traffic control • .etc… • Many names for pub/sub systems: notification service, data distribution service MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  6. notification service IBM: -3,75 IBM: -3,75 IBM: 12,51 S1: IBM S2: ACME S3: ACME ACME: +0,15 ACME: +0,15 ACME: +0,15 IBM: +2,51 Basic Interaction Model subscribers publishers “IBM” S1 “ACME” S2 “ACME” S3 subscription event notification MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  7. broker IBM: -3,75 IBM: -3,75 IBM: 12,51 ACME: +0,15 ACME: +0,15 ACME: +0,15 IBM: +2,51 Distributed Notification Service notification service “IBM” S1 S1: IBM “ACME” S2 S2: ACME “ACME” S3: ACME S3 • Matching • Determining the recipients for an event • Routing • Delivering a notification to all the recipients MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  8. Research Topics • System architecture • Efficient routing algorithm • Reliable messaging sending (the QoS issue) • Security • Special application issues • Millions of users • Mobile environment • P2P • Sensors networks……. MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  9. Pub/Sub Systems • IBM – Gryphon • Stanford – SIFT and more… • CU-Boulder – Siena • France – Le Subscribe • Technische University Darmstadt – REBECA • Microsoft – Herald • MIT • Others – XMLBlaster, Elvin4, TIB, Keryx, REBECA MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  10. IBM: -3,75 ACME: +0,15 IBM: +2,51 Subscription Models notification service IBM: -3,75 “IBM” & v < 0 S1 IBM: +2,51 S1: IBM S2: ACME S3: ACME S1: IBM,v<0 S2: ACME S3: ACME “ACME” ACME: +0,15 S2 “ACME” ACME: +0,15 S3 • Topic-Based [Oki et al. 93]: • events are divided in topics • subscribers subscribe for a single topic • Content-Based [Carzaniga et al 2001]: • subscriptions are generic queries SQL-like on the event schema • Type-Based [Eugster 2001]: • notifications are objects • type is the discriminating attribute MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  11. Pub/Sub Variants: Topic-based • Event space is divided in topics, corresponding to logical channels • Participants subscribe for a topic and publish on a topic • Receivers for an event are known a priori • Channel = Group • Therefore often exploit network-level multicast • Group communication MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  12. Content Based pub-sub: event schema • Subscriptions and events defined over an n-dimensional event space (E.g. StockName = “ACME” and change < -3) • Subscription: conjunction of constraints a2 subscription event Content-based subscriptions can include range constraints a1 MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  13. Pub/Sub Variants: Content-based • Receivers are calculated for each event being published • Exploiting multicast is not straighforward • How are subscribers assigned to groups? “ACME” ACME, change = -5 ACME, change = -2 “ACME”, change < -3 MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  14. Topic vs. Content • Topic-based pub/sub • Recipients are known a-priori • Many efficient implementations exist: e.g. tib • Limited expressiveness • Content-based pub/sub • Cannot determine recipients before publication • More flexible • More general • Much more difficult to implement efficiently MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  15. Architectural Model of a pub/sub • Network Multicasting • Broker layer overlay • Structured Overlay MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  16. Architectural Model of a pub/sub • Network Multicasting • Use of multicast networking facilities (also at data link level) • Broker layer overlay • Based on transport level connections between nodes • Hierarchical (Decision tree from publisher to subscribers) • Undirected Acyclic graph spanning all brokers • Structured Overlay • DHT (abstracting from physical nodes) MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  17. EVENT vs SUBSCRIPTION ROUTING • Extreme solutions • Sol 1 (event flooding) • flooding of events in the notification event box • each subscription stored only in one place within the notification event box • Matching operations equal to the number of brokers • Sol 2 (subscription flooding) • each subscription stored at any place within the notification event box • each event matched directly at the broker where the event enters the notification event box MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  18. Identification of intermediate routing solutions Connectivity publishers subscribers brokers Handling subscriptions MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  19. Subscription Partitioning (s) Connectivity publishers subscribers brokers Identification of intermediate routing solutions Handling subscriptions: “subscription partitioning” i.e., “where to store s” s MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  20. Subscription Partitioning (s) Connectivity publishers subscribers brokers Identification of intermediate routing solutions Handling subscriptions: “subscription routing” i.e., “how to bring s there” Subscription Routing s MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  21. Subscription Partitioning (s) Subscription Routing Connectivity publishers subscribers brokers Identification of intermediate routing solutions Handling events e MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  22. Subscription Partitioning (s) Subscription Routing Connectivity Event Partitioning (e) publishers subscribers brokers Identification of intermediate routing solutions Handling events: “Event Partitioning” i.e., “where to match e” e MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  23. Subscription Partitioning (s) Subscription Routing Connectivity Event Routing Event Partitioning (e) publishers subscribers brokers Identification of intermediate routing solutions Handling events: “Event Routing” i.e., “How to bring e there” e MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  24. Subscription Partitioning (s) Subscription Routing Matching (e) Connectivity e Event Routing Event Routing Event Partitioning (e) publishers subscribers brokers Identification of intermediate routing solutions Handling events: “Event Matching” i.e., “define the set of recipients of e” MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  25. Matching (e) Notification routing Event Routing Event Partitioning (e) Identification of intermediate routing solutions Handling events: “Notification Routing” i.e., “bring e to the destinations” Subscription Partitioning (s) Subscription Routing Connectivity e Event Routing publishers subscribers brokers MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  26. Identification of intermediate routing solutions Subscription Partitioning Subscription Routing Matching Notification routing Event Routing Event Partitioning (e) MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  27. Basic Techniques for intermediate routing solutions • Filtering-based Routing [Carzaniga et al. 2001] • Undirected Acyclic graph spanning all the brokers • Rendez-Vous Routing • [Wang et al. 2002] based on dynamic partitioning of the event space among a set of brokers • [Castro et al 2002] based on DHT MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  28. Filtering based routing • Identifying as soon as possible events that are not interesting for any subscriber and arrest their diffusion • Creating “diffusion paths” that lead to subscribers for the event • Construction of a diffusion path requires routing info to be maintained at brokers • Routing information consists of a set of filters (aggregate of subscriptions) that are reachable through that broker MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  29. Filtering based routing: Three main phases • Subscription forwarding: subscriptions (filters) are propagated to every broker leaving state along the path • Matching notification follow (backwards) the path set by subscriptions • Reducing subscription propagation • Avoiding subscription propagation when a filter including the subscription has been already forwarded MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  30. s1: price > 600 SIENAFiltering-Based RoutingSubscription Forwarding s1:1 a 2 s1:a 1 s1:2 s1:2 3 5 s1:1 4 6 s1:3 7 s1:3 8 s1:5 9 s1:6 MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  31. s2: price > 700 SIENAFiltering-Based Routing Subscription Merging s1 covers s2 s1:1 s2:5 s1:1 s1 covers s2 splitter a 2 s1:a s2:2 s1:a 1 splitter s1:2 s1:2 s2:8 s1:2 3 5 s1:1 4 6 s1:3 7 s1:3 8 b s1:5 s1:5 s2:b 9 s1:6 MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  32. n1:price = 899 SIENAFiltering-Based Routing Notification Delivery n1matches s1 n1matches s2 s1:1 s2:5 a 2 s1:a s2:2 1 s1:2 s1:2 s2:8 3 5 s1:1 4 6 s1:3 7 s1:3 8 b s1:5 s2:b 9 s1:6 MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  33. Filtering based routing • Tradeoff between subscription redundancy and subscription flooding • The more the brokers aware of any subscription the earlier notifications that do not match any subscription are filtered out • Tradeoff between expressiveness of subscription language and scalability • Expressiveness complexity brings to flood filters • Limited complexity better scalability MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  34. Rendez Vous Routing • Each event of the event space is owned by one or more (reliability reasons) brokers • Three functions: • SN(): 2N (rendez-vous nodes of ) • EN(e): 2N (rendez-vous nodes of e) • If e  SN() thenSN()  EN(e)   (intersection rule) • Main characteristics: • Controlled subscriptions distribution • Controlled matching • Better load balance MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  35. σ σ σ σ σ e Rendezvous-based routing • Each node is responsible for a partition of the event space • Storing subscriptions, matching events e σ Problem: difficult to define mapping functions when the set of nodes changes over time MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  36. Rendez-Vous routing based on DHT • A matching publisher & subscriber must come up with the same hash keys based on the content Distributed Hash Table buckets distributed publish/subscribe system MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  37. Rendez-Vous routing based on DHT • A matching publisher & subscriber must come up with the same hash keys based on the content Distributed Hash Table buckets home node subscriber subscription MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  38. Rendez-Vous routing based on DHT • A matching publisher & subscriber must come up with the same hash keys based on the content Distributed Hash Table buckets home node subscription subscriber publisher publication MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  39. Rendez-Vous routing based on DHT • A matching publisher & subscriber must come up with the same hash keys based on the content Distributed Hash Table buckets home node subscription publication subscriber publisher MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  40. Rendez-Vous routing based on DHT • A matching publisher & subscriber must come up with the same hash keys based on the content Distributed Hash Table buckets home node subscription subscriber publisher publication MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  41. unsub() sub() send() pub() leave() join() notify() delivery() Rendez-Vous routing [Baldoni et al 2005] Application Event space is mapped into the universe of keys (fixed) CB-pub/sub Subs ak-mapping • Stateless mapping: • Does not depend on execution history (subscriptions, node joins and leaves) DHT Overlay maintains consistency of KN mapping kn-mapping MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  42. Rendez Vous Routing • Problem: • bottleneck at hash bucket nodes • Structuring subscribers as a multicast diffusion tree • Restrictions on the subscription language: • Mapping between multidimensional multi-typed subscriptions to the uni or bi-dimensional space of a structured overlay is not straightforward (e.g., string manipulation) • Strong points: • Handle inherently dynamic changes in the overlay MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  43. Summary of routing • Filtering based routing: • Order of hundreds of brokers • Managed environment • Rendez-vous routing based on DHT • Order of thousands of nodes • Unmanaged environment MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  44. A few pub/sub systems MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  45. Pub/Sub Sysems: Tib/RV [Oki et al 03] • Topic Based • Two level hierarchical architecture of brokers (deamons) on TCP/IP • Event routing is realized through one diffusion tree per subject • Each broker knows the entire network topology and current subscription configuration MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  46. Pub/Sub systems: Gryphon [IBM 00] • Content based • Hierarchical tree from publishers to subscribers • Filtering-based routing • Mapping content-based to network level multicast MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  47. Pub/Sub Systems: SCRIBE [Castro et al. 02] • Topic Based • Based on DHT (Pastry) • Rendez-vous event routing • A random identifier is assigned to each topic • The pastry node with the identifier closest to the one of the topic becomes responsible for that topic MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  48. MEGHDOOT • Content Based • Based on Structured Overlay CAN • Mapping the subscription language and the event space to CAN space • Subscription and event Routing exploit CAN routing algorithms MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  49. Mobile Environment MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

  50. Potential Advantages of Pub/Sub for Mobile Wireless • Decoupling of publishers and subscribers aids mobility • Decoupling of publishers and subscribers aids disconnected operation • Multicast delivery can exploit intrinsic broadcast properties of wireless MINEMA Summer School - Klagenfurt (Austria) July 11-15, 2005

More Related