Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running

Heleen Muijlwijk; Martijn C. Willemsen; Barry Smyth; Wijnand A. IJsselsteijn

doi:10.1145/3640543.3645205

Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running

Heleen Muijlwijk, Martijn C. Willemsen, Barry Smyth, Wijnand A. IJsselsteijn

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Citation (Scopus)

60 Downloads (Pure)

Abstract

Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

Original language	English
Title of host publication	IUI '24
Subtitle of host publication	Proceedings of the 29th International Conference on Intelligent User Interfaces
Place of Publication	New York
Publisher	Association for Computing Machinery, Inc
Pages	245–258
Number of pages	14
ISBN (Electronic)	979-8-4007-0508-3
DOIs	https://doi.org/10.1145/3640543.3645205
Publication status	Published - 5 Apr 2024
Event	IUI 2024 – International Conference on Intelligent User Interface (IUI '24) - Greenville, Greenville, United States Duration: 18 Mar 2024 → 21 Mar 2024

Conference

Conference	IUI 2024 – International Conference on Intelligent User Interface (IUI '24)
Abbreviated title	IUI
Country/Territory	United States
City	Greenville
Period	18/03/24 → 21/03/24

Funding

We would like to thank our student Ylja van Miltenburg for her extensive contribution on building the interactive tool and running the user study. We also thank the participants for sharing their time and insights with us. This research was performed within the framework of the strategic joint research program on Data Science between TU/e and Philips Electronics Nederland B.V.

Keywords

AI-assisted decision making
Case-Based Reasoning (CBR)
domain experts
Human-AI collaboration
Interactive Machine Learning (IML)
marathon running
subject matter expertise
task familiarity
user trust

Access to Document

10.1145/3640543.3645205

pdfFinal published version, 852 KBLicence: CC BY

Cite this

Muijlwijk, H., Willemsen, M. C., Smyth, B., & IJsselsteijn, W. A. (2024). Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running. In IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces (pp. 245–258). Association for Computing Machinery, Inc. https://doi.org/10.1145/3640543.3645205

@inproceedings{ae31d0acca114aad80dad498daab6ca3,

title = "Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running",

abstract = "Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners{\textquoteright} next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users{\textquoteright} domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.",

keywords = "AI-assisted decision making, Case-Based Reasoning (CBR), Human-AI collaboration, Interactive Machine Learning (IML), domain experts, marathon running, subject matter expertise, task familiarity, user trust, AI-assisted decision making, Case-Based Reasoning (CBR), domain experts, Human-AI collaboration, Interactive Machine Learning (IML), marathon running, subject matter expertise, task familiarity, user trust",

author = "Heleen Muijlwijk and Willemsen, {Martijn C.} and Barry Smyth and IJsselsteijn, {Wijnand A.}",

year = "2024",

month = apr,

day = "5",

doi = "10.1145/3640543.3645205",

language = "English",

pages = "245–258",

booktitle = "IUI '24",

publisher = "Association for Computing Machinery, Inc",

address = "United States",

note = "IUI 2024 – International Conference on Intelligent User Interface (IUI '24), IUI ; Conference date: 18-03-2024 Through 21-03-2024",

}

Muijlwijk, H , Willemsen, MC, Smyth, B & IJsselsteijn, WA 2024, Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running. in IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces. Association for Computing Machinery, Inc, New York, pp. 245–258, IUI 2024 – International Conference on Intelligent User Interface (IUI '24), Greenville, South Carolina, United States, 18/03/24. https://doi.org/10.1145/3640543.3645205

Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running. / Muijlwijk, Heleen ; Willemsen, Martijn C.; Smyth, Barry et al.
IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces. New York: Association for Computing Machinery, Inc, 2024. p. 245–258.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models

T2 - IUI 2024 – International Conference on Intelligent User Interface (IUI '24)

AU - Muijlwijk, Heleen

AU - Willemsen, Martijn C.

AU - Smyth, Barry

AU - IJsselsteijn, Wijnand A.

PY - 2024/4/5

Y1 - 2024/4/5

N2 - Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

AB - Users with large domain knowledge can be reluctant to use prediction models. This also applies to the sports domain, where running coaches rarely rely on marathon prediction tools for race-plan advice for their runners’ next marathon. This paper studies the effect of adding interactivity to such prediction models, to incorporate and acknowledge users’ domain knowledge. In think-aloud sessions and an online study, we tested an interactive machine learning tool that allowed coaches to indicate the importance of earlier races feeding into the model. Our results show that coaches deploy rich knowledge when working with the model on runners familiar to them, and their adaptations improved model accuracy. Those coaches who could interact with the model displayed more trust and acceptance in the resulting predictions.

KW - AI-assisted decision making

KW - Case-Based Reasoning (CBR)

KW - Human-AI collaboration

KW - Interactive Machine Learning (IML)

KW - domain experts

KW - marathon running

KW - subject matter expertise

KW - task familiarity

KW - user trust

KW - AI-assisted decision making

KW - Case-Based Reasoning (CBR)

KW - domain experts

KW - Human-AI collaboration

KW - Interactive Machine Learning (IML)

KW - marathon running

KW - subject matter expertise

KW - task familiarity

KW - user trust

UR - http://www.scopus.com/inward/record.url?scp=85191003836&partnerID=8YFLogxK

U2 - 10.1145/3640543.3645205

DO - 10.1145/3640543.3645205

M3 - Conference contribution

SP - 245

EP - 258

BT - IUI '24

PB - Association for Computing Machinery, Inc

CY - New York

Y2 - 18 March 2024 through 21 March 2024

ER -

Benefits of Human-AI Interaction for Expert Users Interacting with Prediction Models: a Study on Marathon Running

Abstract

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this