How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale

Jeanette Falk Olesen; Yiyi Chen; Janet Rafner; Mike Zhang; Johannes Bjerva; Alexander Nolte

doi:10.1145/3706598.3713447

How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale

Jeanette Falk Olesen
, Yiyi Chen
, Janet Rafner
, Mike Zhang
, Johannes Bjerva
, Alexander Nolte

Social Software Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

3 Citations (Scopus)

44 Downloads (Pure)

Abstract

Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes across domains such as industry, education, and research. However, there are no large-scale studies on creativity in hackathons which can advance theory on how hackathon formats lead to creative outcomes. We conducted a computational analysis of 193,353 hackathon projects. By operationalizing creativity through usefulness and novelty, we refined our dataset to 10,363 projects, allowing us to analyze how participant characteristics, collaboration patterns, and hackathon setups influence the development of creative projects. The contribution of our paper is twofold: We identified means for organizers to foster creativity in hackathons. We also explore the use of large language models (LLMs) to augment the evaluation of creative outcomes and discuss challenges and opportunities of doing this, which has implications for creativity research at large.

Original language	English
Title of host publication	CHI '25
Subtitle of host publication	Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems
Editors	Naomi Yamashita, Vanessa Evers, Koji Yatani, Xianghua (Sharon) Ding, Bongshin Lee, Marshini Chetty, Phoebe Toups-Dugas
Place of Publication	New York
Publisher	Association for Computing Machinery, Inc.
Number of pages	23
ISBN (Electronic)	979-8-4007-1394-1
DOIs	https://doi.org/10.1145/3706598.3713447
Publication status	Published - 25 Apr 2025
Event	2025 CHI Conference on Human Factors in Computing Systems, CHI 2025 - Yokohama, Japan Duration: 26 Apr 2025 → 1 May 2025 https://chi2025.acm.org/

Conference

Conference	2025 CHI Conference on Human Factors in Computing Systems, CHI 2025
Abbreviated title	CHI 2025
Country/Territory	Japan
City	Yokohama
Period	26/04/25 → 1/05/25
Internet address	https://chi2025.acm.org/

Funding

YC and JB are funded by the Carlsberg Foundation, under the Semper Ardens: Accelerate Programme (project nr. CF21-0454). MZ and JB are funded by the Villum Foundation (project nr. VIL57392). We would further like to thank the Aarhus University Research Foundation and Center for Shaping digital citizenship.

Keywords

Hackathons
creativity
human-centered AI
large language models
quantitative methods

Access to Document

10.1145/3706598.3713447

pdfFinal published version, 1.32 MBLicence: CC BY

Cite this

Olesen, J. F., Chen, Y., Rafner, J., Zhang, M., Bjerva, J., & Nolte, A. (2025). How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale. In N. Yamashita, V. Evers, K. Yatani, X. Ding, B. Lee, M. Chetty, & P. Toups-Dugas (Eds.), CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems Article 198 Association for Computing Machinery, Inc.. https://doi.org/10.1145/3706598.3713447

Olesen, Jeanette Falk ; Chen, Yiyi ; Rafner, Janet et al. / How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale. CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. editor / Naomi Yamashita ; Vanessa Evers ; Koji Yatani ; Xianghua (Sharon) Ding ; Bongshin Lee ; Marshini Chetty ; Phoebe Toups-Dugas. New York : Association for Computing Machinery, Inc., 2025.

@inproceedings{bf0ef9e7e42a43ac9d5cb05771fe4138,

title = "How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale",

abstract = "Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes across domains such as industry, education, and research. However, there are no large-scale studies on creativity in hackathons which can advance theory on how hackathon formats lead to creative outcomes. We conducted a computational analysis of 193,353 hackathon projects. By operationalizing creativity through usefulness and novelty, we refined our dataset to 10,363 projects, allowing us to analyze how participant characteristics, collaboration patterns, and hackathon setups influence the development of creative projects. The contribution of our paper is twofold: We identified means for organizers to foster creativity in hackathons. We also explore the use of large language models (LLMs) to augment the evaluation of creative outcomes and discuss challenges and opportunities of doing this, which has implications for creativity research at large.",

keywords = "Hackathons, creativity, human-centered AI, large language models, quantitative methods",

author = "Olesen, \{Jeanette Falk\} and Yiyi Chen and Janet Rafner and Mike Zhang and Johannes Bjerva and Alexander Nolte",

year = "2025",

month = apr,

day = "25",

doi = "10.1145/3706598.3713447",

language = "English",

editor = "Naomi Yamashita and Vanessa Evers and Koji Yatani and Ding, \{Xianghua (Sharon)\} and Bongshin Lee and Marshini Chetty and Phoebe Toups-Dugas",

booktitle = "CHI '25",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

note = "2025 CHI Conference on Human Factors in Computing Systems, CHI 2025, CHI 2025 ; Conference date: 26-04-2025 Through 01-05-2025",

url = "https://chi2025.acm.org/",

}

Olesen, JF, Chen, Y, Rafner, J, Zhang, M, Bjerva, J & Nolte, A 2025, How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale. in N Yamashita, V Evers, K Yatani, X Ding, B Lee, M Chetty & P Toups-Dugas (eds), CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems., 198, Association for Computing Machinery, Inc., New York, 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025, Yokohama, Japan, 26/04/25. https://doi.org/10.1145/3706598.3713447

How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale. / Olesen, Jeanette Falk; Chen, Yiyi; Rafner, Janet et al.
CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. ed. / Naomi Yamashita; Vanessa Evers; Koji Yatani; Xianghua (Sharon) Ding; Bongshin Lee; Marshini Chetty; Phoebe Toups-Dugas. New York: Association for Computing Machinery, Inc., 2025. 198.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale

AU - Olesen, Jeanette Falk

AU - Chen, Yiyi

AU - Rafner, Janet

AU - Zhang, Mike

AU - Bjerva, Johannes

AU - Nolte, Alexander

PY - 2025/4/25

Y1 - 2025/4/25

N2 - Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes across domains such as industry, education, and research. However, there are no large-scale studies on creativity in hackathons which can advance theory on how hackathon formats lead to creative outcomes. We conducted a computational analysis of 193,353 hackathon projects. By operationalizing creativity through usefulness and novelty, we refined our dataset to 10,363 projects, allowing us to analyze how participant characteristics, collaboration patterns, and hackathon setups influence the development of creative projects. The contribution of our paper is twofold: We identified means for organizers to foster creativity in hackathons. We also explore the use of large language models (LLMs) to augment the evaluation of creative outcomes and discuss challenges and opportunities of doing this, which has implications for creativity research at large.

AB - Hackathons have become popular collaborative events for accelerating the development of creative ideas and prototypes. There are several case studies showcasing creative outcomes across domains such as industry, education, and research. However, there are no large-scale studies on creativity in hackathons which can advance theory on how hackathon formats lead to creative outcomes. We conducted a computational analysis of 193,353 hackathon projects. By operationalizing creativity through usefulness and novelty, we refined our dataset to 10,363 projects, allowing us to analyze how participant characteristics, collaboration patterns, and hackathon setups influence the development of creative projects. The contribution of our paper is twofold: We identified means for organizers to foster creativity in hackathons. We also explore the use of large language models (LLMs) to augment the evaluation of creative outcomes and discuss challenges and opportunities of doing this, which has implications for creativity research at large.

KW - Hackathons

KW - creativity

KW - human-centered AI

KW - large language models

KW - quantitative methods

UR - https://www.scopus.com/pages/publications/105005771402

U2 - 10.1145/3706598.3713447

DO - 10.1145/3706598.3713447

M3 - Conference contribution

BT - CHI '25

A2 - Yamashita, Naomi

A2 - Evers, Vanessa

A2 - Yatani, Koji

A2 - Ding, Xianghua (Sharon)

A2 - Lee, Bongshin

A2 - Chetty, Marshini

A2 - Toups-Dugas, Phoebe

PB - Association for Computing Machinery, Inc.

CY - New York

T2 - 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025

Y2 - 26 April 2025 through 1 May 2025

ER -

Olesen JF, Chen Y, Rafner J, Zhang M, Bjerva J, Nolte A. How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale. In Yamashita N, Evers V, Yatani K, Ding X, Lee B, Chetty M, Toups-Dugas P, editors, CHI '25: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, Inc. 2025. 198 doi: 10.1145/3706598.3713447

How Do Hackathons Foster Creativity? Towards Automated Evaluation of Creativity at Scale

Abstract

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this