Business Monitoring Framework for Process Discovery with Real-Life Logs

1. Business Monitoring Framework for Process Discovery with Real-Life Logs Mari Abe maria@jp.ibm.com Michiharu Kudo kudo@jp.ibm.com IBM Research - Tokyo

2. Outline • Motivation • Proposed approach • Evaluation Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

3. Outline • Motivation • Proposed approach • Evaluation Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

4. Issue 1. Log semantics is implementation-dependent • Business analysts must asses whether it is possible to discover processes from their log Customers do NOT have process model & workflow engines process model & workflow engines logs Customers have only an implementation of application Application logs Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

5. 14 sec for calc_premium_page 61 sec for insurance_option_page (product = “life insurance”) user1 user1 2013-08-20T08:30:34 response pageid=“calc_premium_page” 9 sec for simulation_result_page 2013-08-20T08:30:48 request user1 source=“calc_premium_page” response user1 2013-08-20T08:30:49 pageid=“insurance_option_page” request source=“insurance_option_page” product = “life insurance, premium =“$50”, period=“10 years” 2013-08-20T08:31:50 user1 response 2013-08-20T08:31:51 user1 pageid=“simulation_result_page” request source = “simulation_result_page user1 2013-08-20T08:32:00 Log of Premium Calculation Application of Insurance • Real-life log includes complicated correlations with different keys in different event type for process discovery • The log contains valuable information for business analysis line # 1 2 3 4 5 6 7 8 9 10 user1 took 84 sec for premium calculation process with product “life insurance”, premium “$50”, period “10 years” 11 Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

6. analysis program1 analysis program2 analysis program3 Issue 2. Taking Long Time for Analysis is not Acceptable for Face-to-face Meeting • The customers desire to see discovered processes from the different view points of the metrics • Changing and executing of the analysis program takes long time to get the new result at the face-to-face meeting interactively application logs 1st iteration 2nd iteration 60 minutes for changing and executing of program weekday weekend 60 minutes for changing and executing of program 3rd iteration product A product B : Frustrating analysis! Time Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

7. Challenge • How to define implementation-dependent log semantics as a configuration • How to accelerate analysis cycle so that they can see the result from different view at the face-to-face meeting interactively Analysis Time Analysis Time ... ... iteration iteration 1st 1st 2nd 3rd 2nd 3rd Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

8. Our Goal and Approach • Goal • Design a process discovery framework to generate event base and their query expressions with metrics for agile analysis • Approach: • Employ MonitoringContext to configure correlation keys and metrics calculations • Propose runtime supporting dynamic matching between instances of monitoring contexts • Generate event base with metric annotations and their queries synthesize application log event base with metric annotations (MXML) Proposed Runtime query expressions with metrics (XSLT) review Configurations (Monitoring Contexts) business analysts select Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo business analysts customers

9. Outline • Motivation • Proposed approach • Evaluation Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

10. Components of Runtime • Class for reading log that • checks log format • converts lines of log to BusinessEvents • sends BusinessEvents to MonitoringContextManager LogReader Class for dispatching events MonitoringContextManager • Class for generating results with internal states of monitoring contexts that • knows output format of event base * * GenerationTemplate MonitoringContext • Class for processing events that knows • correlation keys, inbound/outbound events, metrics calculations, terminal conditions • knows abstraction level of monitoring Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

11. Event Processing Flow Event base fragment of a task with metrics annotation (e.g. MXML) Task Template (4）Finalize a MonitoringContext instance <AuditTrailEntry id=0> <!-- DURATION=14000 --> <!-- START_TYPE=new --> <!-- HELP_PAGE_ACCESS=0 --> <WorkflowModelElement>logon_page</WorkflowModelElement> <EventType>Complete</EventType> <TimeStamp>2013-08-20T08:30:33.408+0900</TimeStamp> <Originator>user1</Originator></AuditTrailEntry> ... (2) Dispatch the event (1) Generate an event Task Context LogReader Monitoring ContextManager ... Log (3)Generate an event of higher level Process Context (4) Dispatch the event to higher level of contexts ... Process fragment with metric annotation (e.g. MXML with reference) <WorkflowLog> <Process> <ProcessInstance id="0"> <!-- duration=314001 --> <!–- product=productA--> <!–- username=user1--> <AuditTrailEntryRef id=0/> ... Process Template Query Template Query for event base (e.g. XSLT) <xsl:template match= “//ProcessInstance[comment()[contains(., ‘product=productA’)]]”> ... </xsl:template> Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

12. Dynamic Matching Between MonitoringContext Instances • Parent-child relationship is built based on inclusion relationship of correlation keys between different abstraction level line # 1 user1 correlation key 2 process context userID=“user1” response user1 pageid=“calc_premium_page”... 3 parent 4 5 correlation key request user1 source=“calc_premium_page” 6 child userID = “user1” task context response user1 pageid=“insurance_option_page” 7 PageID = “calc_premium_page” 8 source=“insurance_option_page” product = “life insurance, premium =“$50”, period=“10 years” request user1 9 user1 response pageid=“simulation_result_page” 10 11 request user1 source = “simulation_result_page Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

13. Outline • Motivation • Proposed approach • Evaluation Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

14. Evaluation • Goal: Test feasibility of the framework with real applications • Tested data • Logs generated from applications (30 text files, 206 MB in total) • Output format: MXML with metric annotation Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

15. Results Generated context instances Execution time Process model (all log included) Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

16. Process Models Queried by Metrics • Metric: count of help-pages accessed • The result became one of data sources for customers to verify whether or not the help pages were effectively used Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo 3 and more times less than 3 times

17. Conclusion • Challenge • How to define implementation-dependent log semantics as a configuration • How to accelerate analysis cycle so that they can see the result from the different view at the face-to-face meeting interactively • Goal • Design a process discovery framework for agile analysis • Approach • Employ monitoring contexts to configure log analysis with metrics calculations • Propose runtime supporting dynamic matching between instances of monitoring contexts • Generate event base with metric annotations and their queries • Results • The result showed 18.4 min to generate MXML file, 2.7 sec to query with a metric for real-life log • It enabled to accelerate analysis cycle Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo

18. Q&A Monitoring Framework for Process Discovery, Mari Abe and Michiharu Kudo