Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akhtar; Omar Benjelloun; Costanza Conforti; Pieter Gijsbers; Joan Giner-Miguelez; Nitisha Jain; Michael Kuchnik; Quentin Lhoest; Pierre Marcenac; Manil Maskey; Peter Mattson; Luis Oala; Pierre Ruyssen; Rajat Shinde; Elena Simperl; Goeffry Thomas; Slava Tykhonov; Joaquin Vanschoren; Jos van der Velde; Steffen Vogler; Carole-Jean Wu

doi:10.1145/3650203.3663326

Croissant: A Metadata Format for ML-Ready Datasets

Mubashara Akhtar
, Omar Benjelloun
, Costanza Conforti
, Pieter Gijsbers
, Joan Giner-Miguelez
, Nitisha Jain
, Michael Kuchnik
, Quentin Lhoest
, Pierre Marcenac
, Manil Maskey
, Peter Mattson
, Luis Oala
, Pierre Ruyssen
, Rajat Shinde
, Elena Simperl
, Goeffry Thomas
, Slava Tykhonov
, Joaquin Vanschoren
, Jos van der Velde
, Steffen Vogler

Carole-Jean Wu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

34 Citations (Scopus)

133 Downloads (Pure)

Abstract

Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.

Original language	English
Title of host publication	DEEM '24
Subtitle of host publication	Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning
Publisher	Association for Computing Machinery, Inc.
Pages	1-6
Number of pages	6
ISBN (Electronic)	9798400706110
ISBN (Print)	979-8-4007-0611-0
DOIs	https://doi.org/10.1145/3650203.3663326
Publication status	Published - 9 Jun 2024

Funding

Joan Giner-Miguelez is supported by the AIDOaRt project, which is funded by the ECSEL Joint Undertaking (JU) under grant agreement No 101007350. The JU receives support from the European Union s Horizon 2020 research and innovation programme and Sweden, Austria, Czech Republic, Finland, France, Italy, and Spain. Pieter Gijsbers, Joaquin Vanschoren, and Jos van der Velde would like to acknowledge funding by EU s Horizon Europe research and innovation program under grant agreement No. 952215 (TAILOR) and No. 101070000 (AI4EUROPE).

Funders	Funder number
European Union's Horizon 2020 - Research and Innovation Framework Programme
Electronic Components and Systems for European Leadership	101007350
European Union's Horizon 2020 - Research and Innovation Framework Programme	101070000, AI4EUROPE, 952215

Keywords

ML datasets
discoverability
reproducibility
responsible AI

Access to Document

10.1145/3650203.3663326

pdfFinal published version, 1.42 MB

Cite this

Akhtar, M., Benjelloun, O., Conforti, C., Gijsbers, P., Giner-Miguelez, J., Jain, N., Kuchnik, M., Lhoest, Q., Marcenac, P., Maskey, M., Mattson, P., Oala, L., Ruyssen, P., Shinde, R., Simperl, E., Thomas, G., Tykhonov, S., Vanschoren, J., van der Velde, J., ... Wu, C.-J. (2024). Croissant: A Metadata Format for ML-Ready Datasets. In DEEM '24: Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning (pp. 1-6). Association for Computing Machinery, Inc.. https://doi.org/10.1145/3650203.3663326

@inproceedings{16a4df1988414d998df0eeb6d585f138,

title = "Croissant: A Metadata Format for ML-Ready Datasets",

abstract = "Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.",

keywords = "ML datasets, discoverability, reproducibility, responsible AI",

author = "Mubashara Akhtar and Omar Benjelloun and Costanza Conforti and Pieter Gijsbers and Joan Giner-Miguelez and Nitisha Jain and Michael Kuchnik and Quentin Lhoest and Pierre Marcenac and Manil Maskey and Peter Mattson and Luis Oala and Pierre Ruyssen and Rajat Shinde and Elena Simperl and Goeffry Thomas and Slava Tykhonov and Joaquin Vanschoren and \{van der Velde\}, Jos and Steffen Vogler and Carole-Jean Wu",

year = "2024",

month = jun,

day = "9",

doi = "10.1145/3650203.3663326",

language = "English",

isbn = "979-8-4007-0611-0",

pages = "1--6",

booktitle = "DEEM '24",

publisher = "Association for Computing Machinery, Inc.",

address = "United States",

}

Akhtar, M, Benjelloun, O, Conforti, C, Gijsbers, P, Giner-Miguelez, J, Jain, N, Kuchnik, M, Lhoest, Q, Marcenac, P, Maskey, M, Mattson, P, Oala, L, Ruyssen, P, Shinde, R, Simperl, E, Thomas, G, Tykhonov, S, Vanschoren, J , van der Velde, J, Vogler, S & Wu, C-J 2024, Croissant: A Metadata Format for ML-Ready Datasets. in DEEM '24: Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning. Association for Computing Machinery, Inc., pp. 1-6. https://doi.org/10.1145/3650203.3663326

Croissant: A Metadata Format for ML-Ready Datasets. / Akhtar, Mubashara; Benjelloun, Omar; Conforti, Costanza et al.
DEEM '24: Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning. Association for Computing Machinery, Inc., 2024. p. 1-6.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Croissant: A Metadata Format for ML-Ready Datasets

AU - Akhtar, Mubashara

AU - Benjelloun, Omar

AU - Conforti, Costanza

AU - Gijsbers, Pieter

AU - Giner-Miguelez, Joan

AU - Jain, Nitisha

AU - Kuchnik, Michael

AU - Lhoest, Quentin

AU - Marcenac, Pierre

AU - Maskey, Manil

AU - Mattson, Peter

AU - Oala, Luis

AU - Ruyssen, Pierre

AU - Shinde, Rajat

AU - Simperl, Elena

AU - Thomas, Goeffry

AU - Tykhonov, Slava

AU - Vanschoren, Joaquin

AU - van der Velde, Jos

AU - Vogler, Steffen

AU - Wu, Carole-Jean

PY - 2024/6/9

Y1 - 2024/6/9

N2 - Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.

AB - Data is a critical resource for Machine Learning (ML), yet working with data remains a key friction point. This paper introduces Croissant, a metadata format for datasets that simplifies how data is used by ML tools and frameworks. Croissant makes datasets more discoverable, portable and interoperable, thereby addressing significant challenges in ML data management and responsible AI. Croissant is already supported by several popular dataset repositories, spanning hundreds of thousands of datasets, ready to be loaded into the most popular ML frameworks.

KW - ML datasets

KW - discoverability

KW - reproducibility

KW - responsible AI

UR - https://www.scopus.com/pages/publications/85196652122

U2 - 10.1145/3650203.3663326

DO - 10.1145/3650203.3663326

M3 - Conference contribution

SN - 979-8-4007-0611-0

SP - 1

EP - 6

BT - DEEM '24

PB - Association for Computing Machinery, Inc.

ER -

Croissant: A Metadata Format for ML-Ready Datasets

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this