Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running

Heleen Muijlwijk; Martijn C. Willemsen; Barry Smyth; Wijnand A. IJsselsteijn

doi:10.1145/3640543.3645205

Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running

Heleen Muijlwijk, Martijn C. Willemsen, Barry Smyth, Wijnand A. IJsselsteijn

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

1 Citaat (Scopus)

62 Downloads (Pure)

Samenvatting

Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

Originele taal-2	Engels
Titel	IUI '24
Subtitel	Proceedings of the 29th International Conference on Intelligent User Interfaces
Plaats van productie	New York
Uitgeverij	Association for Computing Machinery, Inc.
Pagina's	245–258
Aantal pagina's	14
ISBN van elektronische versie	979-8-4007-0508-3
DOI's	https://doi.org/10.1145/3640543.3645205
Status	Gepubliceerd - 5 apr. 2024
Evenement	IUI 2024 – International Conference on Intelligent User Interface (IUI '24) - Greenville, Greenville, Verenigde Staten van Amerika Duur: 18 mrt. 2024 → 21 mrt. 2024

Congres

Congres	IUI 2024 – International Conference on Intelligent User Interface (IUI '24)
Verkorte titel	IUI
Land/Regio	Verenigde Staten van Amerika
Stad	Greenville
Periode	18/03/24 → 21/03/24

Financiering

We would like to thank our student Ylja van Miltenburg for her extensive contribution on building the interactive tool and running the user study. We also thank the participants for sharing their time and insights with us. This research was performed within the framework of the strategic joint research program on Data Science between TU/e and Philips Electronics Nederland B.V.

Trefwoorden

AI-assisted decision making
Case-Based Reasoning (CBR)
Human-AI collaboration
Interactive Machine Learning (IML)
domain experts
marathon running
subject matter expertise
task familiarity
user trust

Toegang tot document

10.1145/3640543.3645205

pdfDefinitieve gepubliceerde versie, 852 KBLicentie: CC BY

Andere bestanden en links

Link naar publicatie in Scopus

Citeer dit

Muijlwijk, H., Willemsen, M. C., Smyth, B., & IJsselsteijn, W. A. (2024). Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running. In IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces (blz. 245–258). Association for Computing Machinery, Inc.. https://doi.org/10.1145/3640543.3645205

@inproceedings{ae31d0acca114aad80dad498daab6ca3,

title = "Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running",

abstract = "Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners{\textquoteright} next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users{\textquoteright} domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.",

keywords = "AI-assisted decision making, Case-Based Reasoning (CBR), Human-AI collaboration, Interactive Machine Learning (IML), domain experts, marathon running, subject matter expertise, task familiarity, user trust, AI-assisted decision making, Case-Based Reasoning (CBR), domain experts, Human-AI collaboration, Interactive Machine Learning (IML), marathon running, subject matter expertise, task familiarity, user trust",

author = "Heleen Muijlwijk and Willemsen, {Martijn C.} and Barry Smyth and IJsselsteijn, {Wijnand A.}",

year = "2024",

month = apr,

day = "5",

doi = "10.1145/3640543.3645205",

language = "English",

pages = "245–258",

booktitle = "IUI '24",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

note = "IUI 2024 – International Conference on Intelligent User Interface (IUI '24), IUI ; Conference date: 18-03-2024 Through 21-03-2024",

}

Muijlwijk, H , Willemsen, MC, Smyth, B & IJsselsteijn, WA 2024, Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running. in IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces. Association for Computing Machinery, Inc., New York, blz. 245–258, IUI 2024 – International Conference on Intelligent User Interface (IUI '24), Greenville, South Carolina, Verenigde Staten van Amerika, 18/03/24. https://doi.org/10.1145/3640543.3645205

Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running. / Muijlwijk, Heleen ; Willemsen, Martijn C.; Smyth, Barry et al.
IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces. New York: Association for Computing Machinery, Inc., 2024. blz. 245–258.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models

T2 - IUI 2024 – International Conference on Intelligent User Interface (IUI '24)

AU - Muijlwijk, Heleen

AU - Willemsen, Martijn C.

AU - Smyth, Barry

AU - IJsselsteijn, Wijnand A.

PY - 2024/4/5

Y1 - 2024/4/5

N2 - Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

AB - Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

KW - AI-assisted decision making

KW - Case-Based Reasoning (CBR)

KW - Human-AI collaboration

KW - Interactive Machine Learning (IML)

KW - domain experts

KW - marathon running

KW - subject matter expertise

KW - task familiarity

KW - user trust

KW - AI-assisted decision making

KW - Case-Based Reasoning (CBR)

KW - domain experts

KW - Human-AI collaboration

KW - Interactive Machine Learning (IML)

KW - marathon running

KW - subject matter expertise

KW - task familiarity

KW - user trust

UR - http://www.scopus.com/inward/record.url?scp=85191003836&partnerID=8YFLogxK

U2 - 10.1145/3640543.3645205

DO - 10.1145/3640543.3645205

M3 - Conference contribution

SP - 245

EP - 258

BT - IUI '24

PB - Association for Computing Machinery, Inc.

CY - New York

Y2 - 18 March 2024 through 21 March 2024

ER -

Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running

Samenvatting

Congres

Financiering

Trefwoorden

Toegang tot document

Andere bestanden en links

Vingerafdruk

Citeer dit