Single network panoptic segmentation for street scene understanding

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

2 Downloads (Pure)

Uittreksel

In this work, we propose a single deep neural network for panoptic segmentation, for which the goal is to provide each individual pixel of an input image with a class label, as in semantic segmentation, as well as a unique identifier for specific objects in an image, following instance segmentation. Our network makes joint semantic and instance segmentation predictions and combines these to form an output in the panoptic format. This has two main benefits: firstly, the entire panoptic prediction is made in one pass, reducing the required computation time and resources; secondly, by learning the tasks jointly, information is shared between the two tasks, thereby improving performance. Our network is evaluated on two street scene datasets: Cityscapes and Mapillary Vistas. By leveraging information exchange and improving the merging heuristics, we increase the performance of the single network, and achieve a score of 23.9 on the Panoptic Quality (PQ) metric on Mapillary Vistas validation, with an input resolution of 640 × 900 pixels. On Cityscapes validation, our method achieves a PQ score of 45.9 with an input resolution of 512 × 1024 pixels. Moreover, our method decreases the prediction time by a factor of 2 with respect to separate networks.

Originele taal-2Engels
Titel2019 IEEE Intelligent Vehicles Symposium, IV 2019
Plaats van productiePiscataway
UitgeverijInstitute of Electrical and Electronics Engineers
Pagina's709-715
Aantal pagina's7
ISBN van elektronische versie978-1-7281-0560-4
DOI's
StatusGepubliceerd - 1 jun 2019
Evenement30th IEEE Intelligent Vehicles Symposium, IV 2019 - Paris, Frankrijk
Duur: 9 jun 201912 jun 2019

Congres

Congres30th IEEE Intelligent Vehicles Symposium, IV 2019
LandFrankrijk
StadParis
Periode9/06/1912/06/19

Vingerafdruk

Segmentation
Pixels
Pixel
Semantics
Prediction
Merging
Labels
Entire
Heuristics
Neural Networks
Metric
Decrease
Resources
Output
Deep neural networks

Citeer dit

de Geus, D., Meletis, P., & Dubbelman, G. (2019). Single network panoptic segmentation for street scene understanding. In 2019 IEEE Intelligent Vehicles Symposium, IV 2019 (blz. 709-715). [8813788] Piscataway: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/IVS.2019.8813788
de Geus, Daan ; Meletis, Panagiotis ; Dubbelman, Gijs. / Single network panoptic segmentation for street scene understanding. 2019 IEEE Intelligent Vehicles Symposium, IV 2019. Piscataway : Institute of Electrical and Electronics Engineers, 2019. blz. 709-715
@inproceedings{0ac964fe45314cf78c29ad1749741bd1,
title = "Single network panoptic segmentation for street scene understanding",
abstract = "In this work, we propose a single deep neural network for panoptic segmentation, for which the goal is to provide each individual pixel of an input image with a class label, as in semantic segmentation, as well as a unique identifier for specific objects in an image, following instance segmentation. Our network makes joint semantic and instance segmentation predictions and combines these to form an output in the panoptic format. This has two main benefits: firstly, the entire panoptic prediction is made in one pass, reducing the required computation time and resources; secondly, by learning the tasks jointly, information is shared between the two tasks, thereby improving performance. Our network is evaluated on two street scene datasets: Cityscapes and Mapillary Vistas. By leveraging information exchange and improving the merging heuristics, we increase the performance of the single network, and achieve a score of 23.9 on the Panoptic Quality (PQ) metric on Mapillary Vistas validation, with an input resolution of 640 × 900 pixels. On Cityscapes validation, our method achieves a PQ score of 45.9 with an input resolution of 512 × 1024 pixels. Moreover, our method decreases the prediction time by a factor of 2 with respect to separate networks.",
author = "{de Geus}, Daan and Panagiotis Meletis and Gijs Dubbelman",
year = "2019",
month = "6",
day = "1",
doi = "10.1109/IVS.2019.8813788",
language = "English",
pages = "709--715",
booktitle = "2019 IEEE Intelligent Vehicles Symposium, IV 2019",
publisher = "Institute of Electrical and Electronics Engineers",
address = "United States",

}

de Geus, D, Meletis, P & Dubbelman, G 2019, Single network panoptic segmentation for street scene understanding. in 2019 IEEE Intelligent Vehicles Symposium, IV 2019., 8813788, Institute of Electrical and Electronics Engineers, Piscataway, blz. 709-715, Paris, Frankrijk, 9/06/19. https://doi.org/10.1109/IVS.2019.8813788

Single network panoptic segmentation for street scene understanding. / de Geus, Daan; Meletis, Panagiotis; Dubbelman, Gijs.

2019 IEEE Intelligent Vehicles Symposium, IV 2019. Piscataway : Institute of Electrical and Electronics Engineers, 2019. blz. 709-715 8813788.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

TY - GEN

T1 - Single network panoptic segmentation for street scene understanding

AU - de Geus, Daan

AU - Meletis, Panagiotis

AU - Dubbelman, Gijs

PY - 2019/6/1

Y1 - 2019/6/1

N2 - In this work, we propose a single deep neural network for panoptic segmentation, for which the goal is to provide each individual pixel of an input image with a class label, as in semantic segmentation, as well as a unique identifier for specific objects in an image, following instance segmentation. Our network makes joint semantic and instance segmentation predictions and combines these to form an output in the panoptic format. This has two main benefits: firstly, the entire panoptic prediction is made in one pass, reducing the required computation time and resources; secondly, by learning the tasks jointly, information is shared between the two tasks, thereby improving performance. Our network is evaluated on two street scene datasets: Cityscapes and Mapillary Vistas. By leveraging information exchange and improving the merging heuristics, we increase the performance of the single network, and achieve a score of 23.9 on the Panoptic Quality (PQ) metric on Mapillary Vistas validation, with an input resolution of 640 × 900 pixels. On Cityscapes validation, our method achieves a PQ score of 45.9 with an input resolution of 512 × 1024 pixels. Moreover, our method decreases the prediction time by a factor of 2 with respect to separate networks.

AB - In this work, we propose a single deep neural network for panoptic segmentation, for which the goal is to provide each individual pixel of an input image with a class label, as in semantic segmentation, as well as a unique identifier for specific objects in an image, following instance segmentation. Our network makes joint semantic and instance segmentation predictions and combines these to form an output in the panoptic format. This has two main benefits: firstly, the entire panoptic prediction is made in one pass, reducing the required computation time and resources; secondly, by learning the tasks jointly, information is shared between the two tasks, thereby improving performance. Our network is evaluated on two street scene datasets: Cityscapes and Mapillary Vistas. By leveraging information exchange and improving the merging heuristics, we increase the performance of the single network, and achieve a score of 23.9 on the Panoptic Quality (PQ) metric on Mapillary Vistas validation, with an input resolution of 640 × 900 pixels. On Cityscapes validation, our method achieves a PQ score of 45.9 with an input resolution of 512 × 1024 pixels. Moreover, our method decreases the prediction time by a factor of 2 with respect to separate networks.

UR - http://www.scopus.com/inward/record.url?scp=85072284908&partnerID=8YFLogxK

U2 - 10.1109/IVS.2019.8813788

DO - 10.1109/IVS.2019.8813788

M3 - Conference contribution

AN - SCOPUS:85072284908

SP - 709

EP - 715

BT - 2019 IEEE Intelligent Vehicles Symposium, IV 2019

PB - Institute of Electrical and Electronics Engineers

CY - Piscataway

ER -

de Geus D, Meletis P, Dubbelman G. Single network panoptic segmentation for street scene understanding. In 2019 IEEE Intelligent Vehicles Symposium, IV 2019. Piscataway: Institute of Electrical and Electronics Engineers. 2019. blz. 709-715. 8813788 https://doi.org/10.1109/IVS.2019.8813788