1 / 31

Composite Subscriptions in Content-based Pub/Sub Systems

Composite Subscriptions in Content-based Pub/Sub Systems. Guoli Li. Agenda. Motivation PADRES System Composite Subscriptions Case Study: Workflow Management Evaluations Conclusions and Future Work. Motivation. A large number of atomic events Lack of event correlation

phelan-olson
Download Presentation

Composite Subscriptions in Content-based Pub/Sub Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Composite Subscriptions in Content-based Pub/Sub Systems Guoli Li

  2. Agenda • Motivation • PADRES System • Composite Subscriptions • Case Study: Workflow Management • Evaluations • Conclusions and Future Work Middleware 2005 Grenoble France

  3. Motivation • A large number of atomic events • Lack of event correlation • Potential Applications: • Workflow processing and business process execution • Network management • … Middleware 2005 Grenoble France

  4. The PADRES System • PADRES: Publish/Subscribe Applied to Distributed REsource Scheduling • Features: • Predicate-based subscription language • (attribute, op, value) pair • Extended to supporting composite subscriptions • Rule-based publication/subscription matching • Publications are mapped to facts • Subscriptions are mapped to rules • Advertisement-based routing Middleware 2005 Grenoble France

  5. Content-based Routing • Advertising Advertisement Distributed Overlay Broker Network … … PADRES Middleware 2005 Grenoble France

  6. Content-based Routing • Subscribing Subscription Distributed Overlay Broker Network … … PADRES Middleware 2005 Grenoble France

  7. Content-based Routing • Publishing Publication Distributed Overlay Broker Network … … PADRES Middleware 2005 Grenoble France

  8. Several Concepts • Message • Advertisement • Subscription • Publication • Event • Atomic event: publication in pub/sub literature • Composite event: a pattern of events • Subscriptions • Atomic subscription: atomic event • Composite subscription: composite event Middleware 2005 Grenoble France

  9. & S5 & || || S1 S2 S3 S4 Composite Subscriptions • Composite subscription consists of primitive subscriptions • Provide a higher level view for subscribers • Save network bandwidth • Subscription language features • Operators: AND (&), OR (||) • Variables: ($x) • Advanced event patterns CS={{S1 OR S2} AND{S3 OR S4} AND S5} Middleware 2005 Grenoble France

  10. Composite Subscriptions • Parallelization pattern: s1 & s2 • Alternation pattern: s1 || s2 • Sequence pattern: (s1;[timespan:ts]s2)[within:wi] • Repetition pattern: Repetition(S, n, attr, v) e.g. {rule ((job (appl = $X) (name = A) (state = succ)) & (job (appl = $X) (name = B) (state = succ))) => (forward a notification to proper destination)} Middleware 2005 Grenoble France

  11. Composite Subscriptions • Parallelization pattern: s1 & s2 • Alternation pattern: s1 || s2 • Sequence pattern: (s1;[timespan:ts]s2)[within:wi] • Repetition pattern: Repetition(S, n, attr, v) e.g. {rule ((job …. (name = A) (time = $Y)) & (job …. (name = B) (time > $Y+ts) (time < $Y+wi))) => (forward a notification to proper destination)} e.g. {rule ((job …. (name = A) (attr = $Z)) & (job …. (name = A) (attr = $Z+v)) & … & (job …. (name = A) (attr = $Z+(n-1)v)) ) => (forward a notification to proper destination)} Middleware 2005 Grenoble France

  12. & || S3 S1 S2 Composite Subscription Routing CS={{S1 AND S2} ANDS3} P Publishers Subscribers CS’={S1 AND S2} S P1 Distributed Overlay Broker Network CS B1 B5 S S1 B3 CS’ B4 S2 S3 B2 B6 P2 P3 Middleware 2005 Grenoble France

  13. AND AND S3 S1 S2 Composite Event Detection CS={{S1 AND S2} ANDS3} P Publishers Subscribers S CS’={S1 AND S2} P1 P1 Distributed Overlay Broker Network S1 CS B1 B5 S CS’ CS P12 B3 B4 P123 B2 B6 P2 S2 P2 S3 P3 P3 Middleware 2005 Grenoble France

  14. Advantages • Rule-based approach naturally enables the composite subscription • Event detection results can be shared • Network traffic is reduced • Composite subscription is forwarded into the network as far as possible before it is split • Composite events are detected close to the data source and not widely disseminated Middleware 2005 Grenoble France

  15. Case Study: Workflow Management • Decentralized Workflow Management based on Pub/Sub • Functions • Workflow transform • Workflow deployment • Workflow execution • Workflow control and monitoring Middleware 2005 Grenoble France

  16. Architecture Middleware 2005 Grenoble France

  17. E F Workflow Descriptions • Workflows are described using XML-based languages, such as BPEL • Activities defined in a workflow have dependencies among each other • A workflow instance is generated by a trigger • Failures can be handled in the workflow description trigger A B C D Middleware 2005 Grenoble France

  18. Workflow Transform • The first job(s) subscribe to trigger messages e.g. Job A subscribes to trigger publication: {[class,eq,trigger],[workflow,eq,payroll], [instanceID,=,$x]} • Job dependencies are implied by subscriptions e.g. Job E subscribes to job A failed: {[class,eq,job_info],[workflow,eq,payroll], [nstanceID,=,$x], [job,eq,A], [status,eq,fail]} e.g. Job D subscribes to B and C finished, which is a composite subscription • Advertisements are generated from a workflow specification file Middleware 2005 Grenoble France

  19. BPEL A B C D Workflow Deployment sub/advs Job D sub/advs Job C sub/advs Job B sub/advs Job A Deployer sub/advs Job D Distributed Overlay Broker Network Job D Agent B1 B5 B3 B4 sub/advs Job A B2 Job A Agent B6 sub/advs Job C sub/advs Job B Job C Agent Job B Agent Middleware 2005 Grenoble France

  20. BPEL A B C D Workflow Execution (Success) trigger Workflow Manager END Distributed Overlay Broker Network Job D Agent B1 B5 BC B3 A B4 A sub/advs Job D B2 Job A Agent B6 A sub/advs Job A Job C Agent sub/advs Job C Job B Agent B C sub/advs Job B Middleware 2005 Grenoble France

  21. BPEL A E B C F D Workflow Execution (Failure) trigger Workflow Manager sub/advs Job D Distributed Overlay Broker Network Job D Agent B1 B5 B3 B4 END B2 Job A Agent B6 Job F Agent A Job C Agent sub/advs Job F sub/advs Job A Job B Agent E Job E Agent sub/advs Job E sub/advs Job C sub/advs Job B Middleware 2005 Grenoble France

  22. Workflow Monitoring • Monitor is a Pub/Sub client • Monitor/control workflow execution status • Workflow level • Activity level • Support dynamic workflow modification • Visualize topology of broker overlay network • Visualize message routing Middleware 2005 Grenoble France

  23. Evaluations • Environment • Intel Xeon 3GHz, 1GB RAM • JDK 1.4.2 • Workload description • Predicates based on 20 attributes with random operators • Values are uniformly distributed in the value range • Two workflows with different number of jobs • Metrics • Matching time • Routing delay per composite subscription • Network traffic Middleware 2005 Grenoble France

  24. Publication Matching Time Middleware 2005 Grenoble France

  25. Composite Event Detection Middleware 2005 Grenoble France

  26. Routing Delay Middleware 2005 Grenoble France

  27. Number of Notifications Middleware 2005 Grenoble France

  28. Workflow Traffic Middleware 2005 Grenoble France

  29. Related Work • P.R. Pietzuch, B. Shand, and J. Bacon. Composite event detection as a generic middleware extension. IEEE Network Magazine, Special Issue on Middleware Technologies for Future Communication Networks, Jan 2004 • SIENA • REBECA Middleware 2005 Grenoble France

  30. Conclusions • An expressive subscription language • A rule-based approach of message matching and composite event detection • Case study: a decentralized workflow management system based on PADRES • Future Works • Support BPEL4WS • QoS based workflow execution Middleware 2005 Grenoble France

  31. Questions? Middleware 2005 Grenoble France

More Related