Querying and Monitoring Distributed Business Processes

1. Customer Logistics Bank HR System Supplier eComm CRM ERP Querying and Monitoring Distributed Business Processes Daniel Deutch, Tova Milo Tel-Aviv University

2. Students Select… From… Where… sname sid=sid cid=cid name=“Mary” Courses Takes Students Data model Design Query language Streams ... Optimization Indexing Transactions Files organization Distribution ... XML SOAP WSDL ... Querying and Monitoring Distributed BPs

3. Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

4. Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

5. Logically related activities that, when combined in a flow, achieve a business goal. Activities may either be local or remote Operates in a cross-organization, distributed environment Abstract representation, independent of implementation Standardsfacilitate design, deployment, and execution Introduction to BPs What is a Business Process? Querying and Monitoring Distributed BPs

6. BPs are designed by Non-programmers Specify combination of functionalities and flow thereof, to solve a complex problem Example: process an order The operations/functions are implemented by programmers (programming in the small) Example: fetch order document Introduction to BPs Programming In the large Querying and Monitoring Distributed BPs

7. Abstract Process Defines the publicly visible behavior of services that an executable process offers Executable Process Contain the partner’s business logic behind an external protocol Introduction to BPs Abstract vs. Executable process Querying and Monitoring Distributed BPs

8. Orchestration: Executable process, message exchange sequences are controlled by the orchestration designer. Choreography: Non-executable protocol for interactions. E.g., the legal sequences of messages exchanged, guaranteeing interoperability Introduction to BPs Orchestration vs.Choreography Querying and Monitoring Distributed BPs

9. Introduction to BPs Orchestration Querying and Monitoring Distributed BPs

10. Introduction to BPs Web Services Meet BPs Local to company A Web Service 1 Web Service 4 Web Service 2 Web Service 5 Web Service 3 Web Service n At company B On the Web Company A business process Querying and Monitoring Distributed BPs

11. Introduction to BPs Web-Based Travel Agency BP Travel request responses Airlines Websites queries Consolidate Results Confirmation responses queries Travel Service Hotels Websites Querying and Monitoring Distributed BPs

12. Introduction to BPs BP Management • Planning • Modeling & Design • Development & Deployment • Execution, Interacting & Monitoring • Analysis and Optimization Querying and Monitoring Distributed BPs Source: Microsoft BPM Description

13. Complex Systems Distributed Settings Interoperability issues Robustness Scale Web interface Legacy systems Introduction to BPs Modeling Challenges Querying and Monitoring Distributed BPs

14. Introduction to BPs BP market (BPTrends survey) Querying and Monitoring Distributed BPs

15. Over 20 million hits in google for “business process” Over 5 million hits for "business process management" Vast interest by analysts (e.g. Gartner) Rapidly growing interest in industry New Standards Introduction to BPs BP Management buzz Querying and Monitoring Distributed BPs

16. Introduction to BPs Standards History BPML (Intallio et al) BPSS (ebXML) WSCI (Sun et al) WS-Choreography (W3C) BPEL4 PEOPLE (Oracle et. Al) 2000/05 2001/03 2001/05 2001/06 2002/03 2002/06 2002/08 2003/01 2003/04 2007/04 2007/06 XLang (Micorsoft) WSFL (IBM) WSCL (HP) BPEL4WS 1.0(IBM, Microsoft) BPEL4WS 1.1(OASIS) WSBPEL 2.0 (OASIS) BPEL Querying and Monitoring Distributed BPs

17. Introduction to BPs Standards stack Querying and Monitoring Distributed BPs

18. Language for specifying BP behavior based on Web Services (WS) Define BPs as coordinated sets of Web service interactions Define both abstract and executable processes Specifies Web services Composition Introduction to BPs BPEL in a nutshell Querying and Monitoring Distributed BPs

19. <receive> <reply> <invoke> <assign> <throw> <wait> <sequence> <flow> (parallel) <if> <while> <repeatUntil> … Introduction to BPs BPEL constructs Communication-related constructs Flow-related constructs Querying and Monitoring Distributed BPs

20. BPEL is XML-based In general we could use XML editors for specification design Infeasible in practice Introduction to BPs BPEL as XML Querying and Monitoring Distributed BPs

21. Introduction to BPs BPEL as XML (cont.) <process> <!– Definition and roles of process participants --> <partnerLinks> ... </partnerLinks> <!- Data/state used within the process --> <variables> ... </variables> <!- Properties that enable conversations --> <correlationSets> ... </correlationSets> <!- Exception handling --> <faultHandlers> ... </faultHandlers><!- Error recovery – undoing actions --> <compensationHandlers>...</compensationHandlers> <!- Concurrent events with process itself --> <eventHandlers> ... </eventHandlers> <!- Business process flow --> (activities)* </process> Querying and Monitoring Distributed BPs

22. Introduction to BPs BPEL as XML (cont.) <if> <condition> bpel:getVariableProperty('shipRequest', 'props:shipComplete') </condition> <sequence> <assign> <copy> <from variable="shipRequest" property="props:shipOrderID" /> <to variable="shipNotice" property="props:shipOrderID" /> </copy> <copy> <from variable="shipRequest" property="props:itemsCount" /> <to variable="shipNotice" property="props:itemsCount" /> </copy> </assign> If (shipRequest = shipComplete) { shipNotice.OrderId = shipRequest.OrderId; shipNotice.itemsCnt = shipRequest.itemsCnt; } Querying and Monitoring Distributed BPs

23. Introduction to BPs Basic activities Querying and Monitoring Distributed BPs

24. Introduction to BPs Flow activities Querying and Monitoring Distributed BPs

25. Introduction to BPs BP specification Querying and Monitoring Distributed BPs

26. Introduction to BPs Travel Agency Process Flow Querying and Monitoring Distributed BPs

27. Introduction to BPs Zoom In Querying and Monitoring Distributed BPs

28. Introduction to BPs Example Editor (eclipse) Querying and Monitoring Distributed BPs

29. Introduction to BPs Example editor (Oracle) Querying and Monitoring Distributed BPs

30. Introduction to BPs Example editor (IBM) Querying and Monitoring Distributed BPs

31. Introduction to BPs Example editor (Microsoft VS) Querying and Monitoring Distributed BPs

32. So far: challenges & solutions for modeling BPs are hard to analyze, debug, and optimize Again, due to scale, distributed settings, legacy systems,….. Good modeling simplifies process specification But further analysis tools are required Introduction to BPs Challenges Querying and Monitoring Distributed BPs

33. Static Analysis “What kind of credit services are used (in)directly?” “How can I buy a plane ticket?” “Can one get a price quote without giving first credit card info?” Monitoring “Notify me when a user hacks the system and get a price quote without giving his credit card info” Log Analysis “Find all logs where a user bought a plane ticket” Introduction to BPs Analysis Types Querying and Monitoring Distributed BPs

34. Introduction to Business Processes Querying Monitoring Summary & Research Directions Outline Querying and Monitoring Distributed BPs

35. Statically analyze a Business Process “Find all ways in which a user can buy a plane ticket” Analysis needs Control flow analysis (Reachability, Cycle Detection, Temporal properties,…) Structural analysis Data analysis Messages Validation, Pointer analysis, Array bounds analysis,… Combined (data and flow) analysis Database approach Treat BPs as data Design a query language Querying BPs Querying Business Processes Querying and Monitoring Distributed BPs

36. Recall that BPEL is XML-based, then… Why not use XQuery?! Similar arguments to BPs / XML Need to handle complex technical constructs Complex, unintuitive queries (many joins & recursion) No abstraction No robustness to changes in BPEL standard Querying BPs Why not XQuery? Querying and Monitoring Distributed BPs

37. Uniform Graphical Scalable Flexible Similar to the specification design Can handle partial information and uncertainty Querying BPs An Ideal querying/analysis tool Querying and Monitoring Distributed BPs

38. Querying BPs Models & Query Languages Monadic Second Order Logic Context Free Graph Grammars Temporal Logic Recursive State Machine Mu-calculus CTL* First Order Logic Finite State Machine LTL CTL Querying and Monitoring Distributed BPs

39. Finite State Machines (FSM) for software specification Temporal Logic (TL) [Emerson,’90] for querying all possible behaviors Very common in software (and hardware) verification Typically Linear time evaluation (data complexity) Exponential time (query complexity) Querying BPs First Try Querying and Monitoring Distributed BPs

40. States and transition function (typically no “accepting” state) The system configuration is encoded within the states Interested in properties of possible traversal over the states Temporal Logics express such properties Finite State Machines (FSM) Querying BPs Finite State Machines (FSM) Querying and Monitoring Distributed BPs

41. “Flat” No functions No recursion Cycles allowed Execution path by traversal Querying BPs FSM (example) Login Search Reserve Payment Cash Credit Confirm Cancel Querying and Monitoring Distributed BPs

42. Predicates x=0? Was a reservation made? Logical operators (and, or, not) Queries Can a reservation be made without relaying a credit card number? Must one eventually login if he makes a trip search? Querying BPs Temporal Logic - Ingredients Querying and Monitoring Distributed BPs

43. Quantifiers over execution paths A φ - All: φ holds on all paths starting from the current state. E φ - Exists: φ holds in at least one path. Path-specific quantifiers X φ - Next: φ holds at the next state. G φ - Globally: φ has to hold on the entire subsequent path. F φ - Finally: φ eventually has to hold (somewhere). φ U ψ - Until: φ has to hold until at some position ψ holds, and ψ must hold eventually. φ W ψ - Weak until: φ has to hold until ψ holds. Querying BPs Temporal Operators Querying and Monitoring Distributed BPs

44. Queries Can a reservation be made without relaying a credit card number? E(F(Reserve) and not F(Credit)) Must one eventually login if he makes a trip search? A (F(login) or not F(search)) Logics Linear time Logic (LTL) No path quantifiers CTL* Allows path quantifiers Mu-calculus Introduces fix-point operators Querying BPs Temporal Logic First Order (Monadic) Second Order Querying and Monitoring Distributed BPs

45. Scale 2. Expressive Power (Specification) 3. Expressive Power (Query language) 4. (Un)Intuitive Formulation Querying BPs Features & Limitations Querying and Monitoring Distributed BPs

46. Evaluation is linear in FSM size, but… FSM size is huge for real-life specifications Call stack Data Unfeasible Evaluation Features and Limitations 1. Scale Login Search Reserve Payment Cash Credit Confirm Cancel Querying and Monitoring Distributed BPs

47. Bounded Model Checking [Biere et. Al ’99,’03] ,[Clarke et. Al ’04], … Summarization [Reps et. Al ’98], [Sagiv et. Al ‘05],… Compact representation StateCharts [Harel ’87] BDD [Bryant 86’, Lam et. Al ’05] Features and Limitations 1. Scale Solutions Querying and Monitoring Distributed BPs

48. Features and Limitations 1. Scale StateCharts • Graphical tool for designing state machine based specifications • Extend basic FSM concepts (super-states) • Simplifies design • UML standard Querying and Monitoring Distributed BPs

49. Features and Limitations 1. Scale BDD • Data structure that allows compact representation of data with high similarities • (b1^ b2) V (not(b1) ^ b3) b1 b2 b3 1 0 Querying and Monitoring Distributed BPs

50. Store commands as db relations command c is written at location l “Open” all possible contexts Exploit similarities and represent compactly by Binary Decision Diagrams Datalog queries Features and Limitations 1. Scale Relational DB approach [Lam ’05] Querying and Monitoring Distributed BPs