What Averages Do Not Tell

Process Mining concerns discovering insights on business processes from their execution data that are logged by supporting information systems. The logged data (event log) is formed of event sequences (traces) that correspond to executions of a process. Many Deep Learning techniques have been successfully adapted for predictive Process Mining that aims to predict outcomes of processes, remaining time of the process execution, the next event, or even the entire suffix of running traces. Traces are multimodal sequences and very differently structured than the natural language sentences or images often used in Deep Learning. This may require a different approach to processing. Relevant challenges may be the skewness of trace-length distribution and of the activity distribution in real-life logs. So far, there has been little focus on these differences. For suffix prediction, the performance of Deep Learning models was evaluated only on average measures and for a small number of real-life logs with different pre-processing and evaluation strategies, which makes a comparison difficult. We provide an unified end-to-end framework and compare the performance of seven state-of-the-art sequential architectures in common settings. Results show that sequence modeling still has a lot of room for improvement for the majority of the more complex datasets. Further research is required to get consistent performance not just in average measures but over all the prefixes.