A Heuristic Approach for Dual Expert/End-User Evaluation of Guidance in Visual Analytics

Davide Ceneda, Christopher Collins, Mennatallah El-Assady, Silvia Miksch, Christian Tominski, Alessio Arleo

Research output: Contribution to journalArticleAcademicpeer-review

2 Citations (Scopus)

Abstract

Guidance can support users during the exploration and analysis of complex data. Previous research focused on characterizing the theoretical aspects of guidance in visual analytics and implementing guidance in different scenarios. However, the evaluation of guidance-enhanced visual analytics solutions remains an open research question. We tackle this question by introducing and validating a practical evaluation methodology for guidance in visual analytics. We identify eight quality criteria to be fulfilled and collect expert feedback on their validity. To facilitate actual evaluation studies, we derive two sets of heuristics. The first set targets heuristic evaluations conducted by expert evaluators. The second set facilitates end-user studies where participants actually use a guidance-enhanced system. By following such a dual approach, the different quality criteria of guidance can be examined from two different perspectives, enhancing the overall value of evaluation studies. To test the practical utility of our methodology, we employ it in two studies to gain insight into the quality of two guidance-enhanced visual analytics solutions, one being a work-in-progress research prototype, and the other being a publicly available visualization recommender system. Based on these two evaluations, we derive good practices for conducting evaluations of guidance in visual analytics and identify pitfalls to be avoided during such studies.

Original languageEnglish
Article number10301693
Pages (from-to)997-1007
Number of pages11
JournalIEEE Transactions on Visualization and Computer Graphics
Volume30
Issue number1
DOIs
Publication statusPublished - 1 Jan 2024
Externally publishedYes

Funding

This work was funded by Vienna Science and Technology Fund (WWTF) under grant [10.47379/ICT19047] and by DoRIAH (FFG Grant #880883). We also want to thank Markus Bögl and Emily Wall for their support with the evaluation, and the anonymous reviewers for their constructive feedback.

Keywords

  • evaluation
  • Guidance
  • heuristics
  • visual analytics

Fingerprint

Dive into the research topics of 'A Heuristic Approach for Dual Expert/End-User Evaluation of Guidance in Visual Analytics'. Together they form a unique fingerprint.

Cite this