Human-computer interaction analysis for RPA support: Framework and new horizons

In the last decade, industry has wholeheartedly embraced RPA. A large number of RPA projects have been conducted, which all involve the mimicking of human behavior by software. Although vendor-specific frameworks exist to support all aspects of RPA solutions - from the analysis of the system until the enactment and control of the developed robots - most of them base their solution on workflow diagrams (e. g., BluePrism or UlPath). This chapter explores the benefits of a conceptual RPA framework that is based on log information related to the exact human-computer interaction. While the observation of employees' computers (i. e., clicks and keystrokes) may lead to the desired data, it is by no means trivial to convert such events into a meaningful log. Our approach shows how to create a so-called user interface (UI) log based on image similarity techniques. This UI log can then be used to provide support for all stages of the RPA lifecycle. Specifically, we will describe a use case in which the UI log is used (1) to support the analysis of the underlying process and (2) to generate a test platform which checks whether or not the developed robots behave in accordance with the analyzed process. We will also discuss how other stages of the RPA lifecycle can leverage the existence of such a UI log, which leads to the identification of new research lines.