1 / 18

ENHANCING PRECISION IN PROCESS CONFORMANCE Stability, Confidence And Severity

This study explores methods for enhancing precision in process conformance using log data. We present frameworks for measuring fitness, precision, and generalization in process models in relation to the underlying logs. Our approach incorporates concepts of stability, confidence, and severity to evaluate imprecisions within process models, addressing their impact on the accuracy of performance assessments. Key metrics and methodologies are demonstrated, paving the way for better conformance checking and model discovery. Our findings are backed by quantitative analysis and offer practical solutions for operational improvement.

brina
Download Presentation

ENHANCING PRECISION IN PROCESS CONFORMANCE Stability, Confidence And Severity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ENHANCING PRECISION IN PROCESS CONFORMANCE Stability, Confidence And Severity JORGE MUNOZ-GAMA and JOSEP CARMONA UniversitatPolitecnica de Catalunya Barcelona, Spain

  2. Conformance: precision Information System Process Model Discovery Logs ? Fitness Precision Conformance Generalization Structure

  3. (1) Log Behavior • Prefix automaton of log behavior D F A # Instances Log Traces A B D E A 1435 54 54 54 54 H A C D G H F A 946 A C G D H F A 764 D H F A A C G H D F A 54 A C D G G H F A 1 818 764 764 764 764 764 G A C D G H F A 3145 3200 3199 2381 1435 2381 1435 3145 3200 3199 1765 1764 1710 946 946 947 947 946 946 946 946 G H F A B 1 1 1 1 D E A 1435 1435 1435 1435

  4. (2) Log-based Model Exploration • Extend with tasks availed by the model in each state D F A 54 54 54 54 E B H D H F A A D A 818 764 764 764 764 C G F H G A C D G H F A G G 3200 3200 1765 947 947 G 946 946 946 H F A 0 0 B 1 1 1 1 D E A H H 1435 1435 1435 1435 0 G 0 0

  5. (3) Comparing Log and Model • Imprecisions = in the model but not in the log • Threshold ( ) for robustness D F A 54 54 54 54 H D H F A 764 764 764 764 G C D G H F A G G 818 946 946 946 0 0 A 3200 G 3200 1765 H 947 H 947 H F A B 0 G 0 1 1 1 1 D E A 1435 0 1435 1435 1435

  6. Metric • Counts and weights imprecisions according to their frequencies • Estimating the effort needed to achieve a model completely precise D F A 54 54 54 54 H D H F A G G 818 764 764 764 764 G * Extension of Munoz-Gama and Carmona BPM 2010 0 0 A C D G H F A 3200 3200 1765 H 947 H 947 G 946 946 946 H F A B 0 G 0 1 1 1 1 D E A 1435 0 1435 1435 1435

  7. Confidence log K High Confidence Low Confidence

  8. Confidence: Upper Estimation D F A • BIP Formulation • Best scenario = coveringimprecisions K = 3 54 54 54 54 H D H F A G G 818 764 764 764 764 G 0 0 A C D G H F A 3200 3200 1765 H 947 H 947 G 946 946 946 B 0 0 1 • Upper Bound D E A 1435 1435 1435 1435 • Cost of an imprecision (C): • Gain of an imprecision (G):

  9. Confidence: Lower Estimation • Worst scenario = new escaping states 54 54 54 54 H D F A K = 1 D H F A • new states with escaping states each • e.g. G G 818 764 764 764 764 G 0 0 A C D G H F A 3200 3200 1765 H 947 H 947 G 946 946 946 0 0 1 1 1 1 1 • Lower bound

  10. Confidence Results

  11. Severity D F A D F A 0 0 0 0 H H 54 54 54 54 54 54 54 54 H H H H H H D F A D F A H H D H F A D H F A 0 0 0 0 54 54 54 54 54 54 54 54 H H G G G G 818 818 764 764 764 764 764 764 764 764 G D H F A D H F A 0 0 0 0 G G G G 818 818 764 764 764 764 764 764 764 764 sever G A C D G H F A 0 0 0 0 3200 3200 1765 H 947 H 947 G 946 946 946 mid • Subjective and multifactor • Frequency, Alternation, Stability, Criticality H F A A C D G H F A B 0 G 0 3200 3200 1765 H 947 H 947 G 1 1 1 1 946 946 946 H H low D E A H F A B 0 G 0 H H 1435 1 1 1 1 0 0 0 1435 1435 1435 H H H H D E A 0 0 0 0 0 1435 0 0 1435 1435 1435 H H 0 0 • All imprecisions equally important?

  12. Severity: Frequency • Imprecision in frequent parts more sever 3 3000 sever sever 10 7 10000 7000 0 0

  13. Severity: Alternation • More chances to make a mistake more sever sever sever

  14. Severity: Stability • Apply perturbation • increase the number of instances in that point • proportional to the current occurrence number • Measure how easy is to overpass the threshold • Imprecision stable to perturbation more sever sever sever 3000 3000 10000 6901 10000 7000 0 99

  15. Severity: Criticality • Importance of the task involved in the imprecision • Inspired on Cost-based Fitness in Conformance Checking by Adriansyah, Sidorova and van Dongen, ACSD 2011 sever sever Bank Transfer Check Date Format

  16. Severity Results * Benchmarks produced by PLG by Andrea Burattin and Alessandro Sperduti

  17. Implementation ETConformance Plug-in

  18. Not addressed in this presentation • Non fitting traces • Invisible and Duplicate tasks Conclusions • Metric to measure the precision • Confidence interval over the metric • Severity assessments over the imprecisions • Implemented in an open-source framework

More Related