TY - GEN
T1 - Making Time Fly: Using Fillers to Improve Perceived Latency in Crowd-Powered Conversational Systems
AU - Abbas, Tahir
AU - Markopoulos, Panos
AU - Khan, Javed
AU - Gadiraju, Ujwal
PY - 2021/10/4
Y1 - 2021/10/4
N2 - Crowd-Powered Conversational Systems (CPCS) are gaining traction due to their potential utility in a range of application fields where automated conversational interfaces are still inadequate. Currently, long response times negatively impact CPCSs, limiting their potential application as conversational partners. Related research has focused on developing algorithms for swiftly hiring workers and synchronous crowd coordination techniques to ensure high-quality work. Evaluation studies typically concern system reaction times and performance measurements, but have so far not examined the effects of extended wait times on users. The goal of this study, based on time perception models, is to explore how effective different time fillers are at reducing the negative impacts of waiting in CPCSs. To this end, we conducted a rigorous simulation-based between-subjects (N = 930) study on the Prolific crowdsourcing platform to assess the influence of different filler types across three levels of delay (8, 16 & 32s) for Information Retrieval (IR) and stress management tasks. Our results show that asking users to perform secondary tasks (e.g., microtasks or breathing exercises) while waiting for longer periods of time helped divert their attention away from timekeeping, increased their engagement, and resulted in shorter perceived waiting times. For shorter delays, conversational fillers generated more intense immersion and contributed to shorten the perception of time.
AB - Crowd-Powered Conversational Systems (CPCS) are gaining traction due to their potential utility in a range of application fields where automated conversational interfaces are still inadequate. Currently, long response times negatively impact CPCSs, limiting their potential application as conversational partners. Related research has focused on developing algorithms for swiftly hiring workers and synchronous crowd coordination techniques to ensure high-quality work. Evaluation studies typically concern system reaction times and performance measurements, but have so far not examined the effects of extended wait times on users. The goal of this study, based on time perception models, is to explore how effective different time fillers are at reducing the negative impacts of waiting in CPCSs. To this end, we conducted a rigorous simulation-based between-subjects (N = 930) study on the Prolific crowdsourcing platform to assess the influence of different filler types across three levels of delay (8, 16 & 32s) for Information Retrieval (IR) and stress management tasks. Our results show that asking users to perform secondary tasks (e.g., microtasks or breathing exercises) while waiting for longer periods of time helped divert their attention away from timekeeping, increased their engagement, and resulted in shorter perceived waiting times. For shorter delays, conversational fillers generated more intense immersion and contributed to shorten the perception of time.
KW - Crowd-powered Conversational System
KW - Conversational Agents
KW - Perceived Latency
KW - Time Fillers
KW - Time Perceptions
UR - http://www.scopus.com/inward/record.url?scp=85129787629&partnerID=8YFLogxK
U2 - 10.1609/hcomp.v9i1.18935
DO - 10.1609/hcomp.v9i1.18935
M3 - Conference contribution
SN - 9781577358725
VL - 9
SP - 2
EP - 14
BT - HCOMP 2021 - Proceedings of the 9th AAAI Conference on Human Computation and Crowdsourcing
A2 - Kamar, Ece
A2 - Luther, Kurt
PB - AAAI Press
ER -