A Wizard of Oz Study of Guidance Strategies and Dynamics

Co-adaptive guidance in visual analytics is a mixed-initiative process in which both the user and the system work together to support each other in solving a given analysis task. While previous studies show the effectiveness of guidance, the impact of guidance design decisions (e.g., the suggestions' timing, contextualization, or adaptation) and misguidance often remain under-investigated. To investigate these aspects and examine a variety of guidance interaction patterns in a realistic analysis scenario, we present a Wizard of Oz (WOz) study setup in which pairs of participants take the user's and the system's roles, respectively. As users perform their analysis tasks, they are observed by wizards who provide just-in-time guidance as they see fit. Moreover, we designed the study so that wizards would occasionally and unknowingly provide misguidance during the analysis to investigate the users' confidence in guidance systems. We recruited two groups of participants (12 wizards and 12 users) and paired each participant with two from the other group, obtaining 48 observations. We report insights on interactions between users and wizards. By analyzing these interaction dynamics and the guidance strategies the wizards apply, we derive recommendations for implementing and evaluating future co-adaptive guidance systems.