Panoptic segmentation with a joint semantic and instance segmentation network

Research output: Working paperAcademic

13 Downloads (Pure)

Abstract

We present a single network method for panoptic segmentation. This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics. Joint training is the first step towards an end-to-end panoptic segmentation network and is faster and more memory efficient than training and predicting with two networks, as done in previous work. The architecture consists of a ResNet-50 feature extractor shared by the semantic segmentation and instance segmentation branch. For instance segmentation, a Mask R-CNN type of architecture is used, while the semantic segmentation branch is augmented with a Pyramid Pooling Module. Results for this method are submitted to the COCO and Mapillary Joint Recognition Challenge 2018. Our approach achieves a PQ score of 17.6 on the Mapillary Vistas validation set and 27.2 on the COCO test-dev set.
Original languageEnglish
PublisherarXiv.org
Pages1-6
Number of pages6
Publication statusUnpublished - 6 Sep 2018

Fingerprint

Semantics
Masks
Data storage equipment

Bibliographical note

Technical report

Keywords

  • cs.CV

Cite this

@techreport{efb7792bd15a48dcb0740a995e019c34,
title = "Panoptic segmentation with a joint semantic and instance segmentation network",
abstract = "We present a single network method for panoptic segmentation. This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics. Joint training is the first step towards an end-to-end panoptic segmentation network and is faster and more memory efficient than training and predicting with two networks, as done in previous work. The architecture consists of a ResNet-50 feature extractor shared by the semantic segmentation and instance segmentation branch. For instance segmentation, a Mask R-CNN type of architecture is used, while the semantic segmentation branch is augmented with a Pyramid Pooling Module. Results for this method are submitted to the COCO and Mapillary Joint Recognition Challenge 2018. Our approach achieves a PQ score of 17.6 on the Mapillary Vistas validation set and 27.2 on the COCO test-dev set.",
keywords = "cs.CV",
author = "{de Geus}, Daan and Panagiotis Meletis and Gijs Dubbelman",
note = "Technical report",
year = "2018",
month = "9",
day = "6",
language = "English",
pages = "1--6",
publisher = "arXiv.org",
type = "WorkingPaper",
institution = "arXiv.org",

}

TY - UNPB

T1 - Panoptic segmentation with a joint semantic and instance segmentation network

AU - de Geus, Daan

AU - Meletis, Panagiotis

AU - Dubbelman, Gijs

N1 - Technical report

PY - 2018/9/6

Y1 - 2018/9/6

N2 - We present a single network method for panoptic segmentation. This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics. Joint training is the first step towards an end-to-end panoptic segmentation network and is faster and more memory efficient than training and predicting with two networks, as done in previous work. The architecture consists of a ResNet-50 feature extractor shared by the semantic segmentation and instance segmentation branch. For instance segmentation, a Mask R-CNN type of architecture is used, while the semantic segmentation branch is augmented with a Pyramid Pooling Module. Results for this method are submitted to the COCO and Mapillary Joint Recognition Challenge 2018. Our approach achieves a PQ score of 17.6 on the Mapillary Vistas validation set and 27.2 on the COCO test-dev set.

AB - We present a single network method for panoptic segmentation. This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics. Joint training is the first step towards an end-to-end panoptic segmentation network and is faster and more memory efficient than training and predicting with two networks, as done in previous work. The architecture consists of a ResNet-50 feature extractor shared by the semantic segmentation and instance segmentation branch. For instance segmentation, a Mask R-CNN type of architecture is used, while the semantic segmentation branch is augmented with a Pyramid Pooling Module. Results for this method are submitted to the COCO and Mapillary Joint Recognition Challenge 2018. Our approach achieves a PQ score of 17.6 on the Mapillary Vistas validation set and 27.2 on the COCO test-dev set.

KW - cs.CV

M3 - Working paper

SP - 1

EP - 6

BT - Panoptic segmentation with a joint semantic and instance segmentation network

PB - arXiv.org

ER -