1 / 92

BPM in Cloud Architectures: Business Process Management with SLAs and Events

BPM in Cloud Architectures: Business Process Management with SLAs and Events. BPM 2010, Hoboken, NJ. Hans-Arno Jacobsen Bell University Laboratory Chair University of Toronto. http://www.padres.msrg.utoronto.ca. An e QoS ystem for declarative distributed applications with SLAs.

kayla
Download Presentation

BPM in Cloud Architectures: Business Process Management with SLAs and Events

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BPM in Cloud Architectures:Business Process Management withSLAs and Events BPM 2010, Hoboken, NJ Hans-Arno Jacobsen Bell University Laboratory Chair University of Toronto http://www.padres.msrg.utoronto.ca An eQoSystem for declarative distributed applications with SLAs • http://eqosystem.msrg.org/

  2. Business Process Example Pid=c1 Loan Application Processing service RTT < 100ms uptime > 99.99 % Store inDB … service time < 2s Reject < 0.3 gid=c001 Creditcheck 2 < 0.5 Pid=c001 Pid=c3301 Pid=c401 gid=c001 c001 end Checkscore Checkscore 2 Creditcheck Approve service cost < $0.02 > 0.7 Send toofficer else … else BPM 2010, Hoboken, NJ

  3. Large-scale Business Processes Vendor Goods selection Goods delivery Dispatch B Packaging Pick-up goods • Case Study (Chinese Electronics Manufacturer): • Department-level processes with 26 to 47 activities • Global processes that compose departmental ones • Thousands of concurrent instances • Hundreds of collaborating partners • Geographically distributed • Administrative boundaries Out-stock B FedEx Delivery Pick up Sale prediction Sign Contract Sale Fill order Determinate plan Process Check order CCC administrate Fill out-stock bill Check stock Manufactory Confirm features Design Fill dispatch bill Determinate plan Control Prototype Out Take Raw materials Execute plan Warehouse Material Out-stock B Pay Credit card Check Assign Audit Process control Make plan Target price Signature Raw Check dealer Check credit Finance Confirm Approval Approval Monitoring Feature selection Print receipt Validate Statistic Monitor Marketing Requirement collection Feedback Affirm order Chart Strategy Design Marketing Manufactory Order Payment BPM 2010, Hoboken, NJ

  4. What Support is Required ? • De-coupling and loose coupling • Fine-grained event filtering • In-network event processing • Composite event detection • Event correlation BPM 2010, Hoboken, NJ

  5. Conflicting Goals End user:Performance Administrator:Cost, utilization Developer:Flexibility, simplicity BPM 2010, Hoboken, NJ

  6. Agenda • Enabler • The PADRES approach • Event-driven BPM with PADRES • SLA-driven BPM with PADRES BPM 2010, Hoboken, NJ

  7. What Abstractions Enable BPM? BPM 2010, Hoboken, NJ

  8. What Abstractions Do Not Work? Cum grani salis • Databases • Great for managing historic data • But what about future data (e.g., events) • Data streams • Great for managing structured streams of tuples • But what about un-structured, multi-typed, sporadic, un-ordered events from many sources • Rule-based expert systems • Great for inference and reasoning • But what about managing large numbers of fined-grainedfilters in distributed environments BPM 2010, Hoboken, NJ

  9. What Abstractions Enable BPM? • It is our opinion that the afore-mentioned requirements can best be addressed by • The content-based publish/subscribe paradigm • Realized by content-based message routing • Events represent state transitions in the environment. • Conveyed as publications to the pub/sub system • Event filtering and correlation is based on • Subscriptions managed by the pub/sub system BPM 2010, Hoboken, NJ

  10. B B Matching Engine & B Routing Table Output queue d2 Input queue Notification Notification subscription dest Class = Loan d2 B Service RTT < 2s d3 B Output queue d3 Content-based Routing Content-based Publish/Subscribe 3. Publish Publisher Publisher Publications Service = credit check 1. Advertise class = Loan, status = approved, Event-Based Content Routing Flexible Service RTT < 2s amount > $500K 10 Decoupled Declarative Responsive Uptime > 99.99% Subscriptions Service RTT < 150ms 2. Subscribe Subscriber Subscriber BPM 2010, Hoboken, NJ

  11. Application Modeling • Advertisements (schema or types) • [class=Loan], [action=*], [customerID=*], [amount<$100K], [region=East] • Publications & events (data) • [class, Loan], [action, request], [customerID, 876594], [amount, $50K] • Subscriptions (query) • [class=Loan], [amount>$500K], [region=*] Application semantics is expressed via advertisements (data sources), publications (data sources), and subscriptions (data sinks) S A P BPM 2010, Hoboken, NJ

  12. Benefits of Publish/Subscribe • Simplifies IT development and maintenance by decoupling enterprise components • Supports sophisticated interactions among components using expressive subscription languages • Supports fine-grained subscriptions for event processing • Achieves scalability with in-network filtering and processing BPM 2010, Hoboken, NJ

  13. A E B C F D Many Applications are Event-based &Benefit from Publish/Subscribe Workflows, business processes and job scheduling Supply chain and logistics Job A done In flight Triggered Delivered Fault! Ordered Event-Based Callback RFID: Razor SKU Invoke Loan Light Temperature Transform Service oriented architectures RFID and sensor networks BPM 2010, Hoboken, NJ

  14. Our Approach SLA model An SLA model can communicate goals to cloud infrastructure providers. Process monitoring Distributed execution Service selection An event-based system is an enabling technology for a number of the listed features. Content-based routing & messaging middleware BPM 2010, Hoboken, NJ 14

  15. Our PADRES ESB for Event-driven BPM Enterprise Services Bus (Events & Services Bus) • First generation of students, when I looked away  • Peng Alex David aRno Eli Serge • PADRES is Publish/subscribe Applied to Distributed Resource Scheduling • PAdres is Distributed REsource Scheduling • http://www.padres.msrg.utoronto.ca Web start & open source padres.msrg.org Implemented in Java Acknowledgements (2004-present): 15 BPM 2010, Hoboken, NJ

  16. Our PADRES ESB Stack & Vision BPM 2010, Hoboken, NJ

  17. PADRES ESB is an Overlay of P/S Brokers dest2 Matching Engine B + Publications Routing Table B B dest1 output queue dest2 output queue dest1 input queue subscription dest B dest3 output queue dest3 S P service time < 3s dest2 service time < 2s dest3 service time = 3s service time = 2.5s service time = 1s S P = publisher = subscriber Try out and download at: http://www.padres.msrg.org S BPM 2010, Hoboken, NJ

  18. A E B C F D Innovative PADRES Features HistoricAccess CSRG TR 2009 ACM DEBS’2007 Management CompositeEvents ACM Middleware’2007 ACM Middleware’2004 IEEE ICDCS’2005 IEEE ICDCS’2009 ACM Middleware’2008 ACM DEBS’2007 Security Robustness IEEE ICDCS’2010 ACM Middleware’2006 LoadBalancing BPM 2010, Hoboken, NJ

  19. Event-driven BPM with PADRES But not as complex BPM 2010, Hoboken, NJ

  20. A B C D Exception A E E B C F F D B C D Modeling Business Processes • Dependency in processes and more complex process patterns require event correlation • Event correlation is enabled by the detection of composite events • Composite events are expressed via composite subscriptions • Composite subscription consists of atomic subscriptions • Subscription language features for BPM modeling • E.g., AND, OR, and variables ($x) • Example: D executes, if BandC have completed (D depends on B and C). BPM 2010, Hoboken, NJ

  21. Composite Subscription Examples • Expresses a structural property of a process [class = Activity Status], [cmd. = Archived],[Process ID = $X] AND [class = Activity Status], [cmd. = Signed Off],[Process ID = $X] • Expresses a performance property of a process [cmd. = Credit check request], [Process ID = $X] AND [status = Approved], [Process ID = $X] … Loan Process Archive Process Approve > $50K Checkscore Creditcheck Singed Off Reject BPM 2010, Hoboken, NJ

  22. E F Business Process Management • Transformation of process into pub/sub language • Deployment of transformed process • Execution of process by triggering instances • Monitor process & instance execution • Manage, i.e., control, version,… trigger multiple instances concurrently trigger Exception & compensation A B C D BPM 2010, Hoboken, NJ

  23. BPEL Receive Assign Flow Invoke Wait Reply Business Process Execution END WS Gateway Agent WS client PADRES ESB 6 Invoke 1 3 4 Web Service 2 5 Receive Assign Web Service Reply Wait pub/sub http/soap BPM 2010, Hoboken, NJ

  24. BPEL Transformation Example Sub1 (flow agent): [class = ACTIVITY_STATUS], [process = ’Process5’], [activityName = ’activity1’], [IID isPresent any], [status = ’SUCCESS’] Sub4 (actvity6 agent): ( [class = ACTIVITY_STATUS], [process = ’Process5’], [activityName = ’activity5’], [IID = $X], [status = ’SUCCESS’] ) AND ( [class = LINK_STATUS], [process = ’Process5’], [activityName = ’activity2’], [IID = $X], [status isPresent any] ) Sub1: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity1’], [IID,isPresent,any], [status,eq,’SUCCESS’] Pub3: [class,LINK_STATUS], [process,’Process5’], [activityName,’actitiy2’], [IID,’g001’], [status,’POSITIVE’] Sub5: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity4’], [IID,isPresent,any], [status,eq,’SUCCESS’] && [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity7’], [IID,isPresent,any], [status,eq,”SUCCESS”] Process 5 activity1 Pub1: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’flow1’], [IID,’g001’], [status,’STARTED’] Sub4: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’activity2’], [IID,eq,$X], [status,eq,’SUCCESS’] && [class,eq,LINK_STATUS], [process,eq,’Process5’], [activityName,eq,’activity2’], [IID,eq,$X], [status,isPresent,any] flow1 activity2 activity5 activity3 activity6 Sub2: [class,eq,ACTIVITY_STATUS], [process,eq,’Process5’], [activityName,eq,’flow1’], [IID,isPresent,any], [status,eq,’STARTED’] Pub5: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’actitiy7’], [IID,’g001’], [status,’SUCCESS’] activity4 activity7 activity8 Pub2: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’actitiy2’], [IID,’g001’], [status,’SUCCESS’] Pub4: [class, ACTIVITY_STATUS], [process,’Process5’], [activityName,’actitiy6’], [IID,’g001’], [status,’SUCCESS’] Cf. our ACM Trans Web’2010 for full BPEL mapping BPM 2010, Hoboken, NJ

  25. Evaluation: Changing Request Rate P/S Clustered 20 servers P/S Centralized P/S P/S P/S Distributed PADRES ESB BPM 2010, Hoboken, NJ

  26. SLA-driven BPM More organized! An eQoSystem for declarative distributed applications with SLAs • http://eqosystem.msrg.org/ BPM 2010, Hoboken, NJ

  27. Currently, business goals must be manually considered at every stage of the business process development cycle Only trusted partners service time < 3s Find flight Y Far? Validaterequest Getdestination Find train N cost < $0.02 BPM 2010, Hoboken, NJ

  28. Service Level Agreements (SLAs) SLAs are contracts between service consumers and providers that specify the expected behavior of each party and the penalties of violating the contract. SLAs specify business goals declaratively. BPM 2010, Hoboken, NJ

  29. p q A B C D Runtime Uses of SLAs Process Dynamic service discovery Discover services with capabilities that satisfy goals. MonitoringOnly monitor the business events related to goals.Feed back measurements to support runtime adaptations. Distributed executionFine-grained allocation of process to available resources.Move portions of process to strategic locations. ESB adaptationReconfigure the ESB topology to satisfy goals. ESB broker topology C A,B service time < 2s 1a M service time < 1s 1b D 2 Web service Execution engine ESB node (PADRES broker) M Monitor BPM 2010, Hoboken, NJ

  30. Distributed Execution Example • Execution of one process • instance incurs 5 events • 3 from P to T • 2 from T to S • In practice many concurrent • instances (a rate) Process Activity P T S Execution engine hosting activity P ESB broker topology T P 2 5 8 1 4 7 S 3 6 9 Broker Execution engine BPM 2010, Hoboken, NJ

  31. SLA & Cost Function Example • SLA is minimize traffic • SLA metric is message hops • Minimize message hops • f() = msg_hops_rate = msg_rate * engine_distance • Measure the msg_hops_rate between activities • Measure the distance between execution engines • Strive to find better and better deployments subject to execution constraints T P 2 5 1 4 BPM 2010, Hoboken, NJ

  32. RedeploymentManager (CASCON) SLAs, cost models Estimators Ranking algorithms SLA Management Stack Execution Engine Execution Engine Candidate Engine Discovery (ACM DEBS’2009) Activity Profiler P Q R Activities Engine Profiler P/S broker Activity Manager Latency Bandwidth Engine resource Activities (ACM TWEB’2010) Instance states Atomic Redeployer (IEEE ICDCS’2009) Input, output queues ... PADRES messaging layer BPM 2010, Hoboken, NJ

  33. Cost Model Components The cost of a process is based on metrics 1. Distribution cost Cdist = f(Cd3, Cd1, Cd2) Cd1 Message rate Cd2 Message size Cd3Latency 2. Engine cost Ceng = f(Ce1, Ce2,Ce3) Ce1 Load (number of instances) Ce2 Resources (CPU, memory, etc.) Ce3 Activity complexity 3. Service cost Cserv = f(Cs1, Cs2, Cs3) Cs1 Latency of external service Cs2 Execution time of external service Cs3 Marshalling/unmarshalling Cost(activity) = f(wiCi) Cost(process) = ∑cost(activity) These metrics can be weighted to achieve different objectives Optimize time wd1 = wd3 = we3 = wserv = 1, other wi = 0 Optimize network overhead wd2 = wd3 = 1, other wi = 0 Various optimization criteria can be specified Threshold criteria: ∑wiCi > x E.g., Report SLA violations within 3 s. Minimized criteria:min( ∑wiCi ) E.g., Minimize distribution overhead BPM 2010, Hoboken, NJ

  34. Redeployment Manager • Estimator: Computes an estimate of the metric cost ck(ai,ej) of hosting an activity ai at engine ej • Cost model: Computes an estimate of the cost c(ai,ej) of hosting activity ai on engine ej • Check deployment: Determines what to do with an activity ai • Determine best engine e • Compute benefit: c(ai) – c(ai,e) • Compute resident time at current engine • If resident long enough • If benefit is large enough move ai to e • Otherwise, apply pressure to other activities BPM 2010, Hoboken, NJ

  35. Evaluation and design issues BPM 2010, Hoboken, NJ

  36. Process Hotspot – Illustration Process p = 90% q = 10 % B G A D E F I C H Red activities are pinned to brokers ESB broker topology SLAMinimize traffic AB 2 5 8 GI D 1 4 7 F MetricMessage hops E C 3 6 9 H Broker Execution engine BPM 2010, Hoboken, NJ

  37. Process Hotspot – Results 10% of static 40 • Traffic with redeployment is 47% of the static case • Post-redeployment traffic is 10% • Redeployment triggered in about 30 sec BPM 2010, Hoboken, NJ

  38. Varying Hotspot – Illustration Process p q B G A D E F I C H ESB broker topology AB 2 5 8 GI Red activities are pinned to brokers D 1 4 7 F E C 3 6 9 H Broker Execution engine BPM 2010, Hoboken, NJ

  39. Varying Hotspot – Results Dynamic redeployment suffers temporarily after process hotspot moves Traffic with redeployment is 42% of the static case BPM 2010, Hoboken, NJ

  40. Larger Process - Results • Larger process with several parallel branches • Traffic with redeployment is 48% of the static case • 14% in steady state BPM 2010, Hoboken, NJ

  41. Summary on SLA-driven BPM • Distributed execution engine has qualitative and quantitative advantages • Redeployment algorithm can optimize SLA in many cases • Challenges and design issues • Deployment stuck in local optima • Coordination among re-deployment decisions • SLA granularity • Interference of processes BPM 2010, Hoboken, NJ

  42. Benefits of Content-based Publish/Subscribe for BPM • Naturally enables centralized and distributed business process coordination • Coordination can span administrative domains and physically distributed resources • Supports process orchestration and choreography • Monitoring & control is integral part of paradigm • Agile on the fly process adaptation and versioning • Correlation of application events with low-level infrastructure events BPM 2010, Hoboken, NJ

  43. Summary: The PADRES Stack Download PADRES @ http://padres.msrg.org BPM 2010, Hoboken, NJ

  44. Outlook Enabled by the PADRES Events & Services Bus • Ad hoc and dynamic business processes • Business performance management • Business impact assessment • Business entity management BPM 2010, Hoboken, NJ

  45. Conclusions • Many applications are inherently event-driven. • Effective BPM requires capable event processing abstractions. • Content-based publish/subscribe is a powerful event processing abstraction and paradigm. • PADRES is based on the pub/sub paradigm. • PADRES is an ESB targeted at event-based BPM. • PADRES enables real-time business analytics and business activity monitoring. BPM 2010, Hoboken, NJ

  46. Acknowledgements: Current PADRES Team • Chen Chen • Alex Cheung • Alton Chiu • Amer Farroukh • Patrick Lee • Guoli Li • Bala Maniymaran • Vinod Muthusamy • Reza Sherafat • Naweed Tajuddin • Chunyang Ye • Young Yoon Countless alumni (see our web site) http://padres.msrg.org BPM 2010, Hoboken, NJ

  47. P A D R E S Questions & Discussion? BPM 2010, Hoboken, NJ

  48. Selected Literature about PADRES • All our papers are available from • http://msrg.org/tags/padres & hyperlinked below • A Distributed Service Oriented Architecture for Business Process Execution. ACM Trans. on the Web, 2010. • Publisher Placement Algorithms in Content-based Publish/Subscribe. IEEE ICDCS, 2010. • Transactional Mobility in Distributed Content-Based Publish/Subscribe Systems. IEEE ICDCS, 2009. • Distributed Automatic Service Composition in Large-Scale Systems. ACM DEBS, 2008. • Adaptive Content-based Routing in General Overlay Topologies.ACM Middleware, 2008. • Historic Data Access in Publish/Subscribe. ACM DEBS, 2007. • Papers on managing SLAs with PADRES • http://msrg.org/tags/sla BPM 2010, Hoboken, NJ

  49. References DEBS Conference http://www.debs.org July 2011 @ IBM in New York, US • The PADRES ESB project home • http://padres.msrg.utoronto.ca • An eQoSystem for declarative distributed applications with SLAs • http://research.msrg.utoronto.ca/Eqosystem/ • The Micro-ToPSS event processing middleware for sensor networks (RFID) • http://microToPSS.msrg.utoronto.ca/ • Mobile-ToPSS – publish/subscribe for mobile and location-based applications • http://research.msrg.utoronto.ca/Mobile/ • ToPSS - the Toronto Publish/Subscribe System Family • http://www.ToPSS.biz (coming soon  ) • Quantifying events in software to increase modularity & customization in C-based systems and software-based product lines • http://www.AspeCtC.net (ACC - the AspeCt-oriented C compiler) • The Middleware Systems Research Group • http://www.msrg.utoronto.ca • My web site • http://www.eecg.toronto.edu/~jacobsen @ the University of Toronto BPM 2010, Hoboken, NJ

  50. BPM 2010, Hoboken, NJ

More Related