A Process-Oriented Approach to Analyze Analysts' Use of Visualizations: Revealing Insights into the What, When, and How

Despite Visual Analytics (VA) tools being essential for supporting data analysis, evaluating their use in real-world analytical processes remains challenging. Traditional evaluation methods often overlook the nuanced and evolving nature of analysis processes and are not always suitable for investigating scenarios in which analysts combine multiple tools and visualization types. In this paper, we propose a flexible analysis approach for studying analysts' use of visualizations within and across VA tools. Our qualitative method allows researchers to extract user behavior and cognitive steps from screen recordings and think-aloud data and generate event sequences that capture analytic processes. This enables the analysis of usage patterns from multiple perspectives and levels of granularity and allows for the evaluation of effectiveness measures, such as efficiency and accuracy. We demonstrate our approach in the domain of process mining, where our findings provide insights into the use of existing visualizations, and we reflect on lessons learned from this application.