The BRAVO Semantic Segmentation Challenge Results in UNCV2024

Tuan-Hung Vu; Eduardo Valle; Andrei Bursuc; Tommie Kerssies; Daan de Geus; Gijs Dubbelman; Long  Qian; Bingke Zhu; Yingying Chen; Ming Tang; Jinqiao Wang; Tomáš Vojíř; Jan  Šochman; Jiří  Matas; Michael  Smith; Frank  Ferrie; Shamik  Basu; Christos  Sakaridis; Luc Van Gool

doi:10.1007/978-3-031-91585-7_18

The BRAVO Semantic Segmentation Challenge Results in UNCV2024

Tuan-Hung Vu (Corresponding author), Eduardo Valle, Andrei Bursuc, Tommie Kerssies, Daan de Geus, Gijs Dubbelman, Long Qian, Bingke Zhu, Yingying Chen, Ming Tang, Jinqiao Wang, Tomáš Vojíř, Jan Šochman, Jiří Matas, Michael Smith, Frank Ferrie, Shamik Basu, Christos Sakaridis, Luc Van Gool

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model’s accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model’s ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.

Original language	English
Title of host publication	Computer Vision – ECCV 2024 Workshops
Subtitle of host publication	Milan, Italy, September 29–October 4, 2024, Proceedings, Part XVII
Editors	Alessio Del Bue, Cristian Canton, Jordi Pont-Tuset, Tatiana Tommasi
Place of Publication	Cham
Publisher	Springer
Pages	290-306
Number of pages	17
ISBN (Electronic)	978-3-031-91585-7
ISBN (Print)	978-3-031-91584-0
DOIs	https://doi.org/10.1007/978-3-031-91585-7_18
Publication status	Published - 12 May 2025
Event	18th European Conference on Computer Vision, ECCV 2024 - Milan, Italy Duration: 29 Sept 2024 → 4 Oct 2024

Publication series

Name	Lecture Notes in Computer Science (LNCS)
Volume	15639
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	18th European Conference on Computer Vision, ECCV 2024
Country/Territory	Italy
City	Milan
Period	29/09/24 → 4/10/24

Access to Document

10.1007/978-3-031-91585-7_18

https://arxiv.org/pdf/2409.15107

Cite this

Vu, T.-H., Valle, E., Bursuc, A., Kerssies, T., de Geus, D., Dubbelman, G., Qian, L., Zhu, B., Chen, Y., Tang, M., Wang, J., Vojíř, T., Šochman, J., Matas, J., Smith, M., Ferrie, F., Basu, S., Sakaridis, C., & Van Gool, L. (2025). The BRAVO Semantic Segmentation Challenge Results in UNCV2024. In A. Del Bue, C. Canton, J. Pont-Tuset, & T. Tommasi (Eds.), Computer Vision – ECCV 2024 Workshops: Milan, Italy, September 29–October 4, 2024, Proceedings, Part XVII (pp. 290-306). (Lecture Notes in Computer Science (LNCS); Vol. 15639). Springer. https://doi.org/10.1007/978-3-031-91585-7_18

Vu, Tuan-Hung ; Valle, Eduardo ; Bursuc, Andrei et al. / The BRAVO Semantic Segmentation Challenge Results in UNCV2024. Computer Vision – ECCV 2024 Workshops: Milan, Italy, September 29–October 4, 2024, Proceedings, Part XVII. editor / Alessio Del Bue ; Cristian Canton ; Jordi Pont-Tuset ; Tatiana Tommasi. Cham : Springer, 2025. pp. 290-306 (Lecture Notes in Computer Science (LNCS)).

@inproceedings{49d06f3bbffe4bdaba746f728eda635f,

title = "The BRAVO Semantic Segmentation Challenge Results in UNCV2024",

abstract = "We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model{\textquoteright}s accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model{\textquoteright}s ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.",

author = "Tuan-Hung Vu and Eduardo Valle and Andrei Bursuc and Tommie Kerssies and \{de Geus\}, Daan and Gijs Dubbelman and Long Qian and Bingke Zhu and Yingying Chen and Ming Tang and Jinqiao Wang and Tom{\'a}{\v s} Voj{\'i}{\v r} and Jan {\v S}ochman and Ji{\v r}{\'i} Matas and Michael Smith and Frank Ferrie and Shamik Basu and Christos Sakaridis and \{Van Gool\}, Luc",

year = "2025",

month = may,

day = "12",

doi = "10.1007/978-3-031-91585-7\_18",

language = "English",

isbn = "978-3-031-91584-0",

series = "Lecture Notes in Computer Science (LNCS)",

publisher = "Springer",

pages = "290--306",

editor = "\{Del Bue\}, Alessio and Cristian Canton and Jordi Pont-Tuset and Tatiana Tommasi",

booktitle = "Computer Vision – ECCV 2024 Workshops",

address = "Germany",

note = "18th European Conference on Computer Vision, ECCV 2024 ; Conference date: 29-09-2024 Through 04-10-2024",

}

Vu, T-H, Valle, E, Bursuc, A, Kerssies, T , de Geus, D , Dubbelman, G, Qian, L, Zhu, B, Chen, Y, Tang, M, Wang, J, Vojíř, T, Šochman, J, Matas, J, Smith, M, Ferrie, F, Basu, S, Sakaridis, C & Van Gool, L 2025, The BRAVO Semantic Segmentation Challenge Results in UNCV2024. in A Del Bue, C Canton, J Pont-Tuset & T Tommasi (eds), Computer Vision – ECCV 2024 Workshops: Milan, Italy, September 29–October 4, 2024, Proceedings, Part XVII. Lecture Notes in Computer Science (LNCS), vol. 15639, Springer, Cham, pp. 290-306, 18th European Conference on Computer Vision, ECCV 2024, Milan, Italy, 29/09/24. https://doi.org/10.1007/978-3-031-91585-7_18

The BRAVO Semantic Segmentation Challenge Results in UNCV2024. / Vu, Tuan-Hung (Corresponding author); Valle, Eduardo; Bursuc, Andrei et al.
Computer Vision – ECCV 2024 Workshops: Milan, Italy, September 29–October 4, 2024, Proceedings, Part XVII. ed. / Alessio Del Bue; Cristian Canton; Jordi Pont-Tuset; Tatiana Tommasi. Cham: Springer, 2025. p. 290-306 (Lecture Notes in Computer Science (LNCS); Vol. 15639).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - The BRAVO Semantic Segmentation Challenge Results in UNCV2024

AU - Vu, Tuan-Hung

AU - Valle, Eduardo

AU - Bursuc, Andrei

AU - Kerssies, Tommie

AU - de Geus, Daan

AU - Dubbelman, Gijs

AU - Qian, Long

AU - Zhu, Bingke

AU - Chen, Yingying

AU - Tang, Ming

AU - Wang, Jinqiao

AU - Vojíř, Tomáš

AU - Šochman, Jan

AU - Matas, Jiří

AU - Smith, Michael

AU - Ferrie, Frank

AU - Basu, Shamik

AU - Sakaridis, Christos

AU - Van Gool, Luc

PY - 2025/5/12

Y1 - 2025/5/12

N2 - We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model’s accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model’s ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.

AB - We propose the unified BRAVO challenge to benchmark the reliability of semantic segmentation models under realistic perturbations and unknown out-of-distribution (OOD) scenarios. We define two categories of reliability: (1) semantic reliability, which reflects the model’s accuracy and calibration when exposed to various perturbations; and (2) OOD reliability, which measures the model’s ability to detect object classes that are unknown during training. The challenge attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal interesting insights into the importance of large-scale pre-training and minimal architectural design in developing robust and reliable semantic segmentation models.

UR - http://www.scopus.com/inward/record.url?scp=105007226091&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-91585-7_18

DO - 10.1007/978-3-031-91585-7_18

M3 - Conference contribution

SN - 978-3-031-91584-0

T3 - Lecture Notes in Computer Science (LNCS)

SP - 290

EP - 306

BT - Computer Vision – ECCV 2024 Workshops

A2 - Del Bue, Alessio

A2 - Canton, Cristian

A2 - Pont-Tuset, Jordi

A2 - Tommasi, Tatiana

PB - Springer

CY - Cham

T2 - 18th European Conference on Computer Vision, ECCV 2024

Y2 - 29 September 2024 through 4 October 2024

ER -

Vu TH, Valle E, Bursuc A, Kerssies T , de Geus D , Dubbelman G et al. The BRAVO Semantic Segmentation Challenge Results in UNCV2024. In Del Bue A, Canton C, Pont-Tuset J, Tommasi T, editors, Computer Vision – ECCV 2024 Workshops: Milan, Italy, September 29–October 4, 2024, Proceedings, Part XVII. Cham: Springer. 2025. p. 290-306. (Lecture Notes in Computer Science (LNCS)). doi: 10.1007/978-3-031-91585-7_18

The BRAVO Semantic Segmentation Challenge Results in UNCV2024

Abstract

Publication series

Conference

Access to Document

Other files and links

Embargoed Document

Fingerprint

Cite this