Workflow management systems coordinate and allocate work through the various stages of executing business processes. The benefits of such systems appear pervasive, but no hard data is available that confirms that their implementation improves organizational performance. In part, this is due to the difficulty of measuring the effects of enterprise-wide initiatives in general. In this paper, the results are presented of a longitudinal, multi-case study into the effectiveness of workflow management technology. The study builds on a novel methodology that combines field work and computer simulations. Through its application, the contribution of this technology to conduct business processes faster and with less effort could be quantitatively assessed. Surprisingly, only a fraction of the projects that were followed in this longitudinal study led to a fully operational implementation of a workflow management system at all. Even so, in most of the projects where such a system was introduced this resulted in substantial improvements. We present success and fail factors for the implementation of this technology within organizations, which we inferred from a follow-up analysis. The novel methodology presented in this paper is thought to be of value to track the performance effects of introducing other technologies or organization concepts as well.