It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task

Patricia K. Kahr; Gerrit Rooks; Martijn C. Willemsen; Chris C.P. Snijders

doi:10.1145/3581641.3584058

It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

29 Citaten (Scopus)

196 Downloads (Pure)

Samenvatting

Humans increasingly interact with AI systems, and successful interactions rely on individuals trusting such systems (when appropriate). Considering that trust is fragile and often cannot be restored quickly, we focus on how trust develops over time in a human-AI-interaction scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low) and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex decision-making task in which individuals estimate jail time for 20 criminal law cases with AI advice. Results show that trust is significantly higher for high-accuracy models. Also, behavioral trust does not decline, and subjective trust even increases significantly with high accuracy. Human-like explanations did not generally affect trust but boosted trust in high-accuracy models.

Originele taal-2	Engels
Titel	IUI 2023 - Proceedings of the 28th International Conference on Intelligent User Interfaces
Plaats van productie	New York
Uitgeverij	Association for Computing Machinery, Inc.
Pagina's	528–539
Aantal pagina's	12
ISBN van elektronische versie	9798400701061
ISBN van geprinte versie	979-8-4007-0106-1
DOI's	https://doi.org/10.1145/3581641.3584058
Status	Gepubliceerd - 27 mrt. 2023
Evenement	28th International Conference on Intelligent User Interfaces, IUI 2023 - Aerial Center, University of Technology Sydney (UTS), Sydney, Australië Duur: 27 mrt. 2023 → 31 mrt. 2023 Congresnummer: 28

Congres

Congres	28th International Conference on Intelligent User Interfaces, IUI 2023
Verkorte titel	IUI 2023
Land/Regio	Australië
Stad	Sydney
Periode	27/03/23 → 31/03/23

Duurzame ontwikkelingsdoelstellingen van de VN

Deze output draagt bij aan de volgende duurzame ontwikkelingsdoelstelling(en)

SDG 9 – Industrie, innovatie en infrastructuur
SDG 16 – Vrede, rechtvaardigheid en sterke instellingen

Trefwoorden

Trust Development
Human-AI Interaction
Collaborative Decision-Making
Trustworthy AI

Toegang tot document

10.1145/3581641.3584058

pdfDefinitieve gepubliceerde versie, 690 KBLicentie: CC BY

Andere bestanden en links

Link naar publicatie in Scopus

28th International Conference on Intelligent User Interfaces, IUI 2023
Kahr, P. (Deelnemer)
27 mrt. 2023 → 31 mrt. 2023
Activiteit: Types deelname aan of organisatie van een evenement › Congres › Wetenschappelijk

Citeer dit

Kahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2023). It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task. In IUI 2023 - Proceedings of the 28th International Conference on Intelligent User Interfaces (blz. 528–539). Association for Computing Machinery, Inc.. https://doi.org/10.1145/3581641.3584058

@inproceedings{1b9ddcd49b984fd3b58e7aec267bd538,

title = "It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task",

abstract = "Humans increasingly interact with AI systems, and successful interactions rely on individuals trusting such systems (when appropriate). Considering that trust is fragile and often cannot be restored quickly, we focus on how trust develops over time in a human-AI-interaction scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low) and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex decision-making task in which individuals estimate jail time for 20 criminal law cases with AI advice. Results show that trust is significantly higher for high-accuracy models. Also, behavioral trust does not decline, and subjective trust even increases significantly with high accuracy. Human-like explanations did not generally affect trust but boosted trust in high-accuracy models.",

keywords = "Trust Development, Human-AI Interaction, Collaborative Decision-Making, Trustworthy AI, Collaborative Decision-Making, Human-AI Interaction, Trust Development, Trustworthy AI",

author = "Kahr, \{Patricia K.\} and Gerrit Rooks and Willemsen, \{Martijn C.\} and Snijders, \{Chris C.P.\}",

year = "2023",

month = mar,

day = "27",

doi = "10.1145/3581641.3584058",

language = "English",

isbn = "979-8-4007-0106-1",

pages = "528–539",

booktitle = "IUI 2023 - Proceedings of the 28th International Conference on Intelligent User Interfaces",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

note = "28th International Conference on Intelligent User Interfaces, IUI 2023, IUI 2023 ; Conference date: 27-03-2023 Through 31-03-2023",

}

Kahr, PK , Rooks, G , Willemsen, MC & Snijders, CCP 2023, It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task. in IUI 2023 - Proceedings of the 28th International Conference on Intelligent User Interfaces. Association for Computing Machinery, Inc., New York, blz. 528–539, 28th International Conference on Intelligent User Interfaces, IUI 2023, Sydney, Nieuw-Zuid-Wales, Australië, 27/03/23. https://doi.org/10.1145/3581641.3584058

It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task. / Kahr, Patricia K.; Rooks, Gerrit ; Willemsen, Martijn C. et al.
IUI 2023 - Proceedings of the 28th International Conference on Intelligent User Interfaces. New York: Association for Computing Machinery, Inc., 2023. blz. 528–539.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - It Seems Smart, but It Acts Stupid

T2 - 28th International Conference on Intelligent User Interfaces, IUI 2023

AU - Kahr, Patricia K.

AU - Rooks, Gerrit

AU - Willemsen, Martijn C.

AU - Snijders, Chris C.P.

N1 - Conference code: 28

PY - 2023/3/27

Y1 - 2023/3/27

N2 - Humans increasingly interact with AI systems, and successful interactions rely on individuals trusting such systems (when appropriate). Considering that trust is fragile and often cannot be restored quickly, we focus on how trust develops over time in a human-AI-interaction scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low) and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex decision-making task in which individuals estimate jail time for 20 criminal law cases with AI advice. Results show that trust is significantly higher for high-accuracy models. Also, behavioral trust does not decline, and subjective trust even increases significantly with high accuracy. Human-like explanations did not generally affect trust but boosted trust in high-accuracy models.

AB - Humans increasingly interact with AI systems, and successful interactions rely on individuals trusting such systems (when appropriate). Considering that trust is fragile and often cannot be restored quickly, we focus on how trust develops over time in a human-AI-interaction scenario. In a 2x2 between-subject experiment, we test how model accuracy (high vs. low) and type of explanation (human-like vs. not) affect trust in AI over time. We study a complex decision-making task in which individuals estimate jail time for 20 criminal law cases with AI advice. Results show that trust is significantly higher for high-accuracy models. Also, behavioral trust does not decline, and subjective trust even increases significantly with high accuracy. Human-like explanations did not generally affect trust but boosted trust in high-accuracy models.

KW - Trust Development

KW - Human-AI Interaction

KW - Collaborative Decision-Making

KW - Trustworthy AI

KW - Collaborative Decision-Making

KW - Human-AI Interaction

KW - Trust Development

KW - Trustworthy AI

UR - https://www.scopus.com/pages/publications/85152144514

U2 - 10.1145/3581641.3584058

DO - 10.1145/3581641.3584058

M3 - Conference contribution

SN - 979-8-4007-0106-1

SP - 528

EP - 539

BT - IUI 2023 - Proceedings of the 28th International Conference on Intelligent User Interfaces

PB - Association for Computing Machinery, Inc.

CY - New York

Y2 - 27 March 2023 through 31 March 2023

ER -

It Seems Smart, but It Acts Stupid: Development of Trust in AI Advice in a Repeated Legal Decision-Making Task

Samenvatting

Congres

Duurzame ontwikkelingsdoelstellingen van de VN

Trefwoorden

Toegang tot document

Andere bestanden en links

Vingerafdruk

Activiteiten

28th International Conference on Intelligent User Interfaces, IUI 2023

Citeer dit