180 likes | 297 Views
ENHANCING PRECISION IN PROCESS CONFORMANCE Stability, Confidence And Severity. JORGE MUNOZ-GAMA and JOSEP CARMONA Universitat Politecnica de Catalunya Barcelona, Spain. Conformance: precision. Information System. Process. Model. Discovery. Logs. ?. Fitness. Precision. Conformance.
E N D
ENHANCING PRECISION IN PROCESS CONFORMANCE Stability, Confidence And Severity JORGE MUNOZ-GAMA and JOSEP CARMONA UniversitatPolitecnica de Catalunya Barcelona, Spain
Conformance: precision Information System Process Model Discovery Logs ? Fitness Precision Conformance Generalization Structure
(1) Log Behavior • Prefix automaton of log behavior D F A # Instances Log Traces A B D E A 1435 54 54 54 54 H A C D G H F A 946 A C G D H F A 764 D H F A A C G H D F A 54 A C D G G H F A 1 818 764 764 764 764 764 G A C D G H F A 3145 3200 3199 2381 1435 2381 1435 3145 3200 3199 1765 1764 1710 946 946 947 947 946 946 946 946 G H F A B 1 1 1 1 D E A 1435 1435 1435 1435
(2) Log-based Model Exploration • Extend with tasks availed by the model in each state D F A 54 54 54 54 E B H D H F A A D A 818 764 764 764 764 C G F H G A C D G H F A G G 3200 3200 1765 947 947 G 946 946 946 H F A 0 0 B 1 1 1 1 D E A H H 1435 1435 1435 1435 0 G 0 0
(3) Comparing Log and Model • Imprecisions = in the model but not in the log • Threshold ( ) for robustness D F A 54 54 54 54 H D H F A 764 764 764 764 G C D G H F A G G 818 946 946 946 0 0 A 3200 G 3200 1765 H 947 H 947 H F A B 0 G 0 1 1 1 1 D E A 1435 0 1435 1435 1435
Metric • Counts and weights imprecisions according to their frequencies • Estimating the effort needed to achieve a model completely precise D F A 54 54 54 54 H D H F A G G 818 764 764 764 764 G * Extension of Munoz-Gama and Carmona BPM 2010 0 0 A C D G H F A 3200 3200 1765 H 947 H 947 G 946 946 946 H F A B 0 G 0 1 1 1 1 D E A 1435 0 1435 1435 1435
Confidence log K High Confidence Low Confidence
Confidence: Upper Estimation D F A • BIP Formulation • Best scenario = coveringimprecisions K = 3 54 54 54 54 H D H F A G G 818 764 764 764 764 G 0 0 A C D G H F A 3200 3200 1765 H 947 H 947 G 946 946 946 B 0 0 1 • Upper Bound D E A 1435 1435 1435 1435 • Cost of an imprecision (C): • Gain of an imprecision (G):
Confidence: Lower Estimation • Worst scenario = new escaping states 54 54 54 54 H D F A K = 1 D H F A • new states with escaping states each • e.g. G G 818 764 764 764 764 G 0 0 A C D G H F A 3200 3200 1765 H 947 H 947 G 946 946 946 0 0 1 1 1 1 1 • Lower bound
Severity D F A D F A 0 0 0 0 H H 54 54 54 54 54 54 54 54 H H H H H H D F A D F A H H D H F A D H F A 0 0 0 0 54 54 54 54 54 54 54 54 H H G G G G 818 818 764 764 764 764 764 764 764 764 G D H F A D H F A 0 0 0 0 G G G G 818 818 764 764 764 764 764 764 764 764 sever G A C D G H F A 0 0 0 0 3200 3200 1765 H 947 H 947 G 946 946 946 mid • Subjective and multifactor • Frequency, Alternation, Stability, Criticality H F A A C D G H F A B 0 G 0 3200 3200 1765 H 947 H 947 G 1 1 1 1 946 946 946 H H low D E A H F A B 0 G 0 H H 1435 1 1 1 1 0 0 0 1435 1435 1435 H H H H D E A 0 0 0 0 0 1435 0 0 1435 1435 1435 H H 0 0 • All imprecisions equally important?
Severity: Frequency • Imprecision in frequent parts more sever 3 3000 sever sever 10 7 10000 7000 0 0
Severity: Alternation • More chances to make a mistake more sever sever sever
Severity: Stability • Apply perturbation • increase the number of instances in that point • proportional to the current occurrence number • Measure how easy is to overpass the threshold • Imprecision stable to perturbation more sever sever sever 3000 3000 10000 6901 10000 7000 0 99
Severity: Criticality • Importance of the task involved in the imprecision • Inspired on Cost-based Fitness in Conformance Checking by Adriansyah, Sidorova and van Dongen, ACSD 2011 sever sever Bank Transfer Check Date Format
Severity Results * Benchmarks produced by PLG by Andrea Burattin and Alessandro Sperduti
Implementation ETConformance Plug-in
Not addressed in this presentation • Non fitting traces • Invisible and Duplicate tasks Conclusions • Metric to measure the precision • Confidence interval over the metric • Severity assessments over the imprecisions • Implemented in an open-source framework