Abstract
Guidance can support users during the exploration and analysis of complex data. Previous research focused on characterizing the theoretical aspects of guidance in visual analytics and implementing guidance in different scenarios. However, the evaluation of guidance-enhanced visual analytics solutions remains an open research question. We tackle this question by introducing and validating a practical evaluation methodology for guidance in visual analytics. We identify eight quality criteria to be fulfilled and collect expert feedback on their validity. To facilitate actual evaluation studies, we derive two sets of heuristics. The first set targets heuristic evaluations conducted by expert evaluators. The second set facilitates end-user studies where participants actually use a guidance-enhanced system. By following such a dual approach, the different quality criteria of guidance can be examined from two different perspectives, enhancing the overall value of evaluation studies. To test the practical utility of our methodology, we employ it in two studies to gain insight into the quality of two guidance-enhanced visual analytics solutions, one being a work-in-progress research prototype, and the other being a publicly available visualization recommender system. Based on these two evaluations, we derive good practices for conducting evaluations of guidance in visual analytics and identify pitfalls to be avoided during such studies.
Original language | English |
---|---|
Article number | 10301693 |
Pages (from-to) | 997-1007 |
Number of pages | 11 |
Journal | IEEE Transactions on Visualization and Computer Graphics |
Volume | 30 |
Issue number | 1 |
DOIs | |
Publication status | Published - 1 Jan 2024 |
Externally published | Yes |
Funding
This work was funded by Vienna Science and Technology Fund (WWTF) under grant [10.47379/ICT19047] and by DoRIAH (FFG Grant #880883). We also want to thank Markus Bögl and Emily Wall for their support with the evaluation, and the anonymous reviewers for their constructive feedback.
Keywords
- evaluation
- Guidance
- heuristics
- visual analytics