Evaluating conformance measures in process mining using conformance propositions

Anja F. Syring, Niek Tax, Wil M.P. van der Aalst

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Process mining sheds new light on the relationship between process models and real-life processes. Process discovery can be used to learn process models from event logs. Conformance checking is concerned with quantifying the quality of a business process model in relation to event data that was logged during the execution of the business process. There exist different categories of conformance measures. Recall, also called fitness, is concerned with quantifying how much of the behavior that was observed in the event log fits the process model. Precision is concerned with quantifying how much behavior a process model allows for that was never observed in the event log. Generalization is concerned with quantifying how well a process model generalizes to behavior that is possible in the business process but was never observed in the event log. Many recall, precision, and generalization measures have been developed throughout the years, but they are often defined in an ad-hoc manner without formally defining the desired properties up front. To address these problems, we formulate 21 conformance propositions and we use these propositions to evaluate current and existing conformance measures. The goal is to trigger a discussion by clearly formulating the challenges and requirements (rather than proposing new measures). Additionally, this paper serves as an overview of the conformance checking measures that are available in the process mining area.

Original languageEnglish
Title of host publicationTransactions on Petri Nets and Other Models of Concurrency XIV
EditorsMaciej Koutny, Lucia Pomello, Lars Michael Kristensen
PublisherSpringer
Pages192-221
Number of pages30
ISBN (Print)9783662606506
DOIs
Publication statusPublished - 21 Nov 2019
Event39th International Conference on Application and Theory of Petri Nets and Concurrency, Petri Nets 2018, and the 18th International Conference on Application of Concurrency to System Design, ACSD 2018 - Bratislava, Slovakia
Duration: 24 Jun 201929 Jun 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11790 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference39th International Conference on Application and Theory of Petri Nets and Concurrency, Petri Nets 2018, and the 18th International Conference on Application of Concurrency to System Design, ACSD 2018
CountrySlovakia
CityBratislava
Period24/06/1929/06/19

    Fingerprint

Keywords

  • Conformance checking
  • Evaluation measures
  • Process mining

Cite this

Syring, A. F., Tax, N., & van der Aalst, W. M. P. (2019). Evaluating conformance measures in process mining using conformance propositions. In M. Koutny, L. Pomello, & L. M. Kristensen (Eds.), Transactions on Petri Nets and Other Models of Concurrency XIV (pp. 192-221). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11790 LNCS). Springer. https://doi.org/10.1007/978-3-662-60651-3_8