The Trust Recovery Journey: The Effect of Timing of Errors on the Willingness to Follow AI Advice

Patricia Kahr; Gerrit Rooks; Chris C.P. Snijders; Martijn C. Willemsen

doi:10.1145/3640543.3645167

The Trust Recovery Journey: The Effect of Timing of Errors on the Willingness to Follow AI Advice

Patricia Kahr, Gerrit Rooks, Chris C.P. Snijders, Martijn C. Willemsen

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

4 Citaten (Scopus)

68 Downloads (Pure)

Samenvatting

Complementing human decision-making with AI advice offers substantial advantages. However, humans do not always trust AI advice appropriately and are overly sensitive to incidental AI errors, even in cases with overall good performance. Today’s research still needs to uncover the underlying aspects of trust decline and recovery over time in repeated human-AI interactions. Our work investigates the consequences of incidental AI error on (self-reported) trust and participants’ reliance on AI advice. Results from our experiment, where 208 participants evaluated 14 legal cases before and after receiving algorithmic advice, showed that trust significantly decreased after early and late errors but was rapidly restored in both scenarios. Reliance significantly dropped only for early errors but not for late errors. In both scenarios, reliance was able to be restored. Results suggest that late (compared to early) errors are less drastic in trust loss and allow quicker recovery. These findings align with an interpretation in which humans can build up trust over time if a system is performing well, making them more tolerant of incidental AI errors.

Originele taal-2	Engels
Titel	IUI '24
Subtitel	Proceedings of the 29th International Conference on Intelligent User Interfaces
Uitgeverij	Association for Computing Machinery, Inc.
Pagina's	609-622
Aantal pagina's	14
ISBN van elektronische versie	979-8-4007-0508-3
DOI's	https://doi.org/10.1145/3640543.3645167
Status	Gepubliceerd - 5 apr. 2024
Evenement	IUI 2024 – International Conference on Intelligent User Interface (IUI '24) - Greenville, Greenville, Verenigde Staten van Amerika Duur: 18 mrt. 2024 → 21 mrt. 2024

Congres

Congres	IUI 2024 – International Conference on Intelligent User Interface (IUI '24)
Verkorte titel	IUI
Land/Regio	Verenigde Staten van Amerika
Stad	Greenville
Periode	18/03/24 → 21/03/24

Financiering

We thank Claudia Sprenger (Eindhoven University of Technology) for supporting the study setup, as well as Ralf Schmidt (Eindhoven University of Technology) for his guidance on the technical implementation of our study. Our gratitude extends to the European Supply Chain Forum (ESCF), the Department of Industrial Engineering and Innovation Sciences (IE&IS), the Eindhoven Artificial Intelligence Systems Institute (EAISI), and the Logistics Community Brabant for their sponsorship of the research project \"AI Planner of the Future\", which supports the PhD project \"Trust in AI over time\".

Financiers	Financiernummer
Technische Universiteit Eindhoven

Toegang tot document

10.1145/3640543.3645167Licentie: CC BY

pdfDefinitieve gepubliceerde versie, 1,35 MBLicentie: CC BY

Andere bestanden en links

Link naar publicatie in Scopus

Citeer dit

@inproceedings{0093339825054cd4b4894054ac25be32,

title = "The Trust Recovery Journey: The Effect of Timing of Errors on the Willingness to Follow AI Advice",

abstract = "Complementing human decision-making with AI advice offers substantial advantages. However, humans do not always trust AI advice appropriately and are overly sensitive to incidental AI errors, even in cases with overall good performance. Today{\textquoteright}s research still needs to uncover the underlying aspects of trust decline and recovery over time in repeated human-AI interactions. Our work investigates the consequences of incidental AI error on (self-reported) trust and participants{\textquoteright} reliance on AI advice. Results from our experiment, where 208 participants evaluated 14 legal cases before and after receiving algorithmic advice, showed that trust significantly decreased after early and late errors but was rapidly restored in both scenarios. Reliance significantly dropped only for early errors but not for late errors. In both scenarios, reliance was able to be restored. Results suggest that late (compared to early) errors are less drastic in trust loss and allow quicker recovery. These findings align with an interpretation in which humans can build up trust over time if a system is performing well, making them more tolerant of incidental AI errors.",

keywords = "Collaborative Decision-Making, Trust Recovery, Trust in AI Over Time",

author = "Patricia Kahr and Gerrit Rooks and Snijders, {Chris C.P.} and Willemsen, {Martijn C.}",

year = "2024",

month = apr,

day = "5",

doi = "10.1145/3640543.3645167",

language = "English",

pages = "609--622",

booktitle = "IUI '24",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

note = "IUI 2024 – International Conference on Intelligent User Interface (IUI '24), IUI ; Conference date: 18-03-2024 Through 21-03-2024",

}

Kahr, P , Rooks, G , Snijders, CCP & Willemsen, MC 2024, The Trust Recovery Journey: The Effect of Timing of Errors on the Willingness to Follow AI Advice. in IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces. Association for Computing Machinery, Inc., blz. 609-622, IUI 2024 – International Conference on Intelligent User Interface (IUI '24), Greenville, South Carolina, Verenigde Staten van Amerika, 18/03/24. https://doi.org/10.1145/3640543.3645167

The Trust Recovery Journey: The Effect of Timing of Errors on the Willingness to Follow AI Advice. / Kahr, Patricia ; Rooks, Gerrit ; Snijders, Chris C.P. et al.
IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces. Association for Computing Machinery, Inc., 2024. blz. 609-622.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - The Trust Recovery Journey

T2 - IUI 2024 – International Conference on Intelligent User Interface (IUI '24)

AU - Kahr, Patricia

AU - Rooks, Gerrit

AU - Snijders, Chris C.P.

AU - Willemsen, Martijn C.

PY - 2024/4/5

Y1 - 2024/4/5

N2 - Complementing human decision-making with AI advice offers substantial advantages. However, humans do not always trust AI advice appropriately and are overly sensitive to incidental AI errors, even in cases with overall good performance. Today’s research still needs to uncover the underlying aspects of trust decline and recovery over time in repeated human-AI interactions. Our work investigates the consequences of incidental AI error on (self-reported) trust and participants’ reliance on AI advice. Results from our experiment, where 208 participants evaluated 14 legal cases before and after receiving algorithmic advice, showed that trust significantly decreased after early and late errors but was rapidly restored in both scenarios. Reliance significantly dropped only for early errors but not for late errors. In both scenarios, reliance was able to be restored. Results suggest that late (compared to early) errors are less drastic in trust loss and allow quicker recovery. These findings align with an interpretation in which humans can build up trust over time if a system is performing well, making them more tolerant of incidental AI errors.

AB - Complementing human decision-making with AI advice offers substantial advantages. However, humans do not always trust AI advice appropriately and are overly sensitive to incidental AI errors, even in cases with overall good performance. Today’s research still needs to uncover the underlying aspects of trust decline and recovery over time in repeated human-AI interactions. Our work investigates the consequences of incidental AI error on (self-reported) trust and participants’ reliance on AI advice. Results from our experiment, where 208 participants evaluated 14 legal cases before and after receiving algorithmic advice, showed that trust significantly decreased after early and late errors but was rapidly restored in both scenarios. Reliance significantly dropped only for early errors but not for late errors. In both scenarios, reliance was able to be restored. Results suggest that late (compared to early) errors are less drastic in trust loss and allow quicker recovery. These findings align with an interpretation in which humans can build up trust over time if a system is performing well, making them more tolerant of incidental AI errors.

KW - Collaborative Decision-Making

KW - Trust Recovery

KW - Trust in AI Over Time

UR - http://www.scopus.com/inward/record.url?scp=85190969348&partnerID=8YFLogxK

U2 - 10.1145/3640543.3645167

DO - 10.1145/3640543.3645167

M3 - Conference contribution

SP - 609

EP - 622

BT - IUI '24

PB - Association for Computing Machinery, Inc.

Y2 - 18 March 2024 through 21 March 2024

ER -

The Trust Recovery Journey: The Effect of Timing of Errors on the Willingness to Follow AI Advice

Samenvatting

Congres

Financiering

Toegang tot document

Andere bestanden en links

Vingerafdruk

Citeer dit