TY - GEN
T1 - Evaluating conformance measures in process mining using conformance propositions
AU - Syring, Anja F.
AU - Tax, Niek
AU - van der Aalst, Wil M.P.
PY - 2019/11/21
Y1 - 2019/11/21
N2 - Process mining sheds new light on the relationship between process models and real-life processes. Process discovery can be used to learn process models from event logs. Conformance checking is concerned with quantifying the quality of a business process model in relation to event data that was logged during the execution of the business process. There exist different categories of conformance measures. Recall, also called fitness, is concerned with quantifying how much of the behavior that was observed in the event log fits the process model. Precision is concerned with quantifying how much behavior a process model allows for that was never observed in the event log. Generalization is concerned with quantifying how well a process model generalizes to behavior that is possible in the business process but was never observed in the event log. Many recall, precision, and generalization measures have been developed throughout the years, but they are often defined in an ad-hoc manner without formally defining the desired properties up front. To address these problems, we formulate 21 conformance propositions and we use these propositions to evaluate current and existing conformance measures. The goal is to trigger a discussion by clearly formulating the challenges and requirements (rather than proposing new measures). Additionally, this paper serves as an overview of the conformance checking measures that are available in the process mining area.
AB - Process mining sheds new light on the relationship between process models and real-life processes. Process discovery can be used to learn process models from event logs. Conformance checking is concerned with quantifying the quality of a business process model in relation to event data that was logged during the execution of the business process. There exist different categories of conformance measures. Recall, also called fitness, is concerned with quantifying how much of the behavior that was observed in the event log fits the process model. Precision is concerned with quantifying how much behavior a process model allows for that was never observed in the event log. Generalization is concerned with quantifying how well a process model generalizes to behavior that is possible in the business process but was never observed in the event log. Many recall, precision, and generalization measures have been developed throughout the years, but they are often defined in an ad-hoc manner without formally defining the desired properties up front. To address these problems, we formulate 21 conformance propositions and we use these propositions to evaluate current and existing conformance measures. The goal is to trigger a discussion by clearly formulating the challenges and requirements (rather than proposing new measures). Additionally, this paper serves as an overview of the conformance checking measures that are available in the process mining area.
KW - Conformance checking
KW - Evaluation measures
KW - Process mining
UR - http://www.scopus.com/inward/record.url?scp=85076497114&partnerID=8YFLogxK
U2 - 10.1007/978-3-662-60651-3_8
DO - 10.1007/978-3-662-60651-3_8
M3 - Conference contribution
AN - SCOPUS:85076497114
SN - 9783662606506
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 192
EP - 221
BT - Transactions on Petri Nets and Other Models of Concurrency XIV
A2 - Koutny, Maciej
A2 - Pomello, Lucia
A2 - Kristensen, Lars Michael
PB - Springer
T2 - 39th International Conference on Application and Theory of Petri Nets and Concurrency, Petri Nets 2018, and the 18th International Conference on Application of Concurrency to System Design, ACSD 2018
Y2 - 24 June 2019 through 29 June 2019
ER -