Supporting Data Consistency in Concurrent Process Execution with Assurance Points and Invariants*

1. Supporting Data Consistency in Concurrent Process Execution with Assurance Points and Invariants* • Andrew Courter • Masters Thesis Defense • Department of Computer Science 08/24/2010 • Committee Members: • Dr. Susan Urban (Chair) • Dr. Michael Shin • Dr. Susan Mengel • *This research is supported by NSF Grant No. CCF-0820152.

2. Overview of Presentation • Motivation • Research Objectives • Related Work • Background Research for the Use of Invariants • Overview of the Invariant Monitoring System • A Prototype of the Invariant Monitoring System • Testing and Evaluation of the Invariant Monitoring System • Summary and Contributions • Future Research 2

3. Motivation • Service-oriented computing and Web services are becoming a standard approach for developing distributed applications • Traditional distributed transactions have relied on ACID properties to protect data • Two-Phase Commit • Serializability • It is not always practical for distributed processes composed of Web services to lock data in service-oriented computing • Need to relax ACID properties and depend on user-defined constraints 3

4. Assurance Points • Shrestha, R. Using Assurance Points and Integration Rules for Recovery in Service Composition. M.S. Thesis, 2010. • Part of the Decentralized Data Dependency Analysis Project • Focused on enhancing a BPEL-like process with user-defined constraints • Supports user defined checkpoints at locations within a process • Three techniques for variable levels of recovery (APRollback, APRetry, APCC) S. D. Urban, L. Gao, R. Shrestha, And A. Courter, Achieving Recovery in Service Composition Through Assurance Points and Integration Rules, to appear in Proceedings of the International Conference on Cooperative Information Systems (COOPIS), Crete, Greece, October, 2010. 4

5. The Invariant Concept • APs provide checkpoints but cannot monitor conditions between APs • Enhancing APs with the concept of monitored invariants • Invariants allow stronger constraint monitoring for a specific duration of execution without locking data 5

6. Research Objectives • Develop the concept of invariants to monitor data in a concurrent environment that needs to enforce user-defined constraints • Extend the AP model to support the expression of invariants. • Design a prototype system for registration of invariants and analysis/extraction of data items to be monitored. • Design and prototype a system for monitoring data changes together with any necessary revisions to the DEGS system to support monitoring capabilities. • Design and prototype a notification and invariant re-evaluation system in response to data changes. • Test and evaluate the invariant re-evaluation system. 6

7. Related Work

8. Related Work • Transactional Workflows (Sheth, 1997) • Transactional Issues for Web Services (Papazoglou, 2003) • Tentative Holding (Limthanmaphon& Zhang, 2004) • Promises Approach (Chen, Fekete, Greenfield, & Jang, 2009) • Transactional Attitudes (Mikalsen, Tai, &Rouvellou, 2002) 8

9. Related Work • Reservation-Based Approach (Zhao, Moser, &Melliar-Smith, 2009) • Monitoring Extension to BPEL (Baresi& Guinea, 2005) • Aspect-Oriented Workflows (A. Charfi&Mezini, 2006) (Wu, et al., 2010) • The Invariant Concept allows users to define constraints and how long to monitor these constraints • Express user-defined constraints • Monitor the constraint conditions • Specify the monitoring time frame • Not reservation or promise based • Optimistic approach to concurrent execution 9

10. Background Research for the Use of Invariants

11. DeltaGrid Delta-Enabled Grid Service Invoke Process Execution Deltas Event Processor Process History Capture System OGSA-DAI Failure Recovery System Oracle Database Delta Repository Deltas Delta-Enabled Grid Services (DEGS) • Urban, S. D., Xiao, Y., Blake, L., & Dietrich, S. W. (2009). Monitoring Data Dependencies in Concurrent Process Execution through Delta-Enabled Grid Services. 5(1), pp85-106. • Key Components • Delta Repository • OGSA-DAI • Delta Notification • Delta-Enabled Grid Service • Delta Grid Delta Notification

12. Service Composition withAssurance Points (APs) • Note: • Condition (c) is always expressed in a negative form (not(C)). • The expression of a pre-condition, post-condition or any additional condition is optional. • Basic Use of AP and Integration Rules

13. Overview of the Invariant Monitoring System

14. The Invariant Monitoring System

15. Challenges for the Invariant Monitoring System • Register, Activate, and Deactivate Invariants • Capture and Filter Deltas Against Invariant Conditions • Re-evaluation of Invariants 15

16. Invariant Specification • CREATE RULE: Invariant_identifier::inv • EVENT: startAP ( endAP, Parameter1, Parameter2….) • CONDITION: rule condition specification • ACTION: recovery procedures • [ON RETRY]: additional/alternative recovery procedures • IRinv is expressed as an Event-Condition-Action (ECA) rule. • The action specification is executed if the condition evaluates to true. • During retry action, there is possibility to execute same integration rule for a second time  invoke ON RETRY

17. Hotel Room Monitoring Example 17

18. Bank Loan Monitoring Example 18

19. A Prototype of the Invariant Monitoring System

20. Monitored Objects • Monitored Objects are acquired from the SQL condition by extracting the table names together with the attributes and relevant conditions. • Invariant: select * from loan where loan.applicantID = ‘” + customerId + “’ and loan.status = ‘pre-qualified’ and loan.amount < (select 10*balance from account where account.customerID = ” + customerId + “’) • Monitored Objects: • (loan, applicantID, =, customerID) • (loan, status, =, ‘pre-qualified’) • (loan, amount, <, ‘calc’) • (account, balance, =, ‘calc’) • (account, customerID, =, customerID) • Invariant: select * from Rooms R where R.roomPrice < ‘” + price + “’ and R.roomType = ‘seaview’ and R.hid = “ + hotelID • Monitored Objects: • (Rooms, roomPrice, <, price) • (Rooms, roomType, =, ‘seaview’) • (Rooms, hid, =, hotelID) 20

21. XML Representation of Invariants 21

22. Invariant Registration Invariant Database Storage • InvariantCondition, MonitoredObjects, siteID, SQLEvalLocation, InvariantAction received in Invariant Agent • SQL Eval WS receives InvariantIdentifer and InvariantCondition and checks condition. • If there are no tuples, process is notified and Invariant is removed. Otherwise, MonitoredObjects are forwarded to Delta Analysis Agent. 22

23. Evaluation Web Service Evaluation Web Service Functionality Materialized View CREATE MATERIALIZED VIEW inv123 REFRESH FORCE ON COMMIT ENABLE QUERY REWRITE AS <select statement for invariant> 23

24. Extensions to DEGS No change needed in delta representation for Insert and Delete operations For Update operation, the DEGS was modified to capture all attributes of a tuple in a delta. 24

25. Invariant Storage Container Two hashtables are used, one containing the tablename and attribute, the other containing each Invariant and its related MonitoredObjects (MObj) 25

26. Overview of the Filtering Process • Check Attributes of Monitored Objects • Checking Insert/Delete requires conditions in a delta to satisfy all Monitored Object constraints in an Invariant • Checking Update requires conditions in a delta to satisfy most Monitored Object constraint and violate at least one Monitored Object constraint • The setValue and Relation values in monitored objects are checked against deltas to determine violations • Filtering Single and Multiple Tables • Single tables use the tupleCount and numViolations variables in each Invariant and when these are equal to each other the Invariant is re-evaluated • Multiple tables use the tupleCount and numViolations in addition to a threshold value to determine when a significant number of potential violations have been made. 26

27. Single Table Filtering • Invariant: “select r.price from room r where r.price < ‘30’ and r.roomType = ‘seaview’ and r.hotelid = ‘234’“ • Monitored Objects: [(room, price, <, ‘30’), (room, roomType, =, ‘seaview’), (room, hotelid, =, ‘234’)] • Number of Satisfying Tuples: 1 • Case 1: Single table insert • A tuple satisfying all of the monitored object conditions is inserted into the room table, so the number of tuples is incremented by one. • Case 2: Single table delete • The only satisfying tuple is deleted by an external process. • The number of violations are incremented and the number of tuples will equal the number of violations. • Case 3: Single table update (similar to single table delete) 27

28. Multiple Table Filtering • Invariant: “select r.price from room r, hotel h where r.price < ‘30’ and r.roomType = ‘seaview’ and r.hotelid = h.hotelid and h.state = ‘Texas’“ • Monitored Objects: [(room, price, <, ‘30’), (room, roomType, =, ‘seaview’), (hotel, state, =, ‘Texas’)] • Number of Satisfying Tuples: 25 Threshold: 25%  • Case 4: Multiple table insert • All inserts into multiple table invariants are ignored. • Inserting tuples can potentially increase the size of the number of tuples that satisfy the invariant condition, but will not cause a violation. • Case 5: Multiple table delete • 7 tuples from the room table satisfying the invariant condition are deleted one after another by an external process. • The number of violations are incremented each time and after the seventh deletion the number of violations will be greater than the threshold(7 > .25*25). • Case 5: Multiple table update (similar to multiple table delete) 28

29. Evaluation of the Invariant Monitoring System

30. Test Structure Hotel Example Database Structure • Primary focus for evaluation was on execution of the Invariant Evaluation Web Service • The Hotel example was used to demonstrate differences between execution times of the Evaluation Web Service for multiple table monitoring. • The database structure is shown above where multiple Rooms can be related to one Hotel 30

31. Evaluation of the Invariant Evaluation WS • Time Comparisons Twice as long to remove 4x as long 31

32. Evaluation of the Invariant Evaluation WS • There exists some overhead when creating the materialized view and when removing the Invariant monitoring. • Using a materialized view for SQL evaluation is more efficient when re-evaluation will potentially be frequent. 32

33. Summary and Contributions • Enhanced Assurance Points with the Invariant Monitoring System • Invariant Agent to register Invariants and handle activation and deactivation of multiple Invariant conditions. • Delta Analysis Agent to filter through deltas and determine if monitored object conditions have been violated. • Filtering distinguishes between single and multiple table filters and uses a threshold value to minimize execution of the invariant evaluation web service. • Designed and Evaluated an Invariant Evaluation Web Service to make use of Materialized Views to handle condition re-evaluations. • Invariants provide a way to monitor data consistency in an environment where the coordinated locking of data items across multiple service executions is not possible, thus providing better support for reliability and maintenance of user-defined correctness conditions among concurrent processes. 33

34. Future Research • Using DEGS to monitor the created materialized views • Statistical analysis to determine more accurate threshold values for different invariants. • Enhance the Invariant Evaluation Web Service to push changes to the Invariant Agent • Testing concurrent processes that use APs with Invariant monitoring capabilities 34

35. Invariant Monitoring System Questions?

36. Acknowledgements • Committee Members: • Dr. Susan D. Urban (Chair) • Dr. Michael Shin • Dr. Susan Mengel • Le Gao, Mary Shuman The End 36

37. Update Algorithm 37

38. Remove Algorithm 38

39. Insert/Delete Filtering 39

40. Update Filtering 40

41. Update Tuple Count 41