Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model

Xin Liu; Herman G.J. Groot; Egor Bondarev; Peter H.N. de With

doi:10.2352/ISSN.2470-1173.2020.10.IPAS-095

Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

In this paper, we investigate person re-identification (re-ID) in a multi-camera network for surveillance applications. To this end, we create a Spatio-Temporal Multi-Camera model (ST-MC model), which exploits statistical data on a person’s entry/exit points in the multi-camera network, to predict in which camera view a person will re-appear. The created ST-MC model is used as a novel extension to the Multiple Granularity Network (MGN) [1], which is the current state of the art in person re-ID. Compared to existing approaches that are solely based on Convolutional Neural Networks (CNNs), our approach helps to improve the re-ID performance by considering not only appearance-based features of a person from a CNN, but also contextual information. The latter serves as scene understanding information complimentary to person re-ID. Experimental results show that for the DukeMTMC-reID dataset [2][3], introduction of our ST-MC model substantially increases the mean Average Precision (mAP) and Rank-1 score from 77.2% to 84.1%, and from 88.6% to 96.2%, respectively.

Original language	English
Title of host publication	Proceedings IS&T International Symposium on Electronic Imaging
Subtitle of host publication	Image Processing: Algorithms and Systems XVIII, 2020
Place of Publication	Springfield
Publisher	Society for Imaging Science and Technology (IS&T)
Number of pages	13
DOIs	https://doi.org/10.2352/ISSN.2470-1173.2020.10.IPAS-095
Publication status	Published - 26 Jan 2020
Event	18th Image Processing: Algorithms and Systems Conference, IPAS 2020 - Burlingame, United States Duration: 26 Jan 2020 → 30 Jan 2020

Publication series

Name	Electronic Imaging
Volume	32

Conference

Conference	18th Image Processing: Algorithms and Systems Conference, IPAS 2020
Country/Territory	United States
City	Burlingame
Period	26/01/20 → 30/01/20

Funding

The work in this paper is funded by the European PS-Crimson project, in the framework of the ITEA research program.

Keywords

CNN
Context information DukeMTMC
DukeMTMC-reID
Person re-identification
Scene understanding
Spatial constraints
Temporal constraints

Access to Document

10.2352/ISSN.2470-1173.2020.10.IPAS-095

Cite this

Liu, X., Groot, H. G. J., Bondarev, E., & de With, P. H. N. (2020). Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model. In Proceedings IS&T International Symposium on Electronic Imaging: Image Processing: Algorithms and Systems XVIII, 2020 Article art00010 (Electronic Imaging; Vol. 32). Society for Imaging Science and Technology (IS&T). https://doi.org/10.2352/ISSN.2470-1173.2020.10.IPAS-095

Liu, Xin ; Groot, Herman G.J. ; Bondarev, Egor et al. / Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model. Proceedings IS&T International Symposium on Electronic Imaging: Image Processing: Algorithms and Systems XVIII, 2020. Springfield : Society for Imaging Science and Technology (IS&T), 2020. (Electronic Imaging).

@inproceedings{19a426532a5743d49da5d5064b6a50ea,

title = "Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model",

abstract = "In this paper, we investigate person re-identification (re-ID) in a multi-camera network for surveillance applications. To this end, we create a Spatio-Temporal Multi-Camera model (ST-MC model), which exploits statistical data on a person{\textquoteright}s entry/exit points in the multi-camera network, to predict in which camera view a person will re-appear. The created ST-MC model is used as a novel extension to the Multiple Granularity Network (MGN) [1], which is the current state of the art in person re-ID. Compared to existing approaches that are solely based on Convolutional Neural Networks (CNNs), our approach helps to improve the re-ID performance by considering not only appearance-based features of a person from a CNN, but also contextual information. The latter serves as scene understanding information complimentary to person re-ID. Experimental results show that for the DukeMTMC-reID dataset [2][3], introduction of our ST-MC model substantially increases the mean Average Precision (mAP) and Rank-1 score from 77.2\% to 84.1\%, and from 88.6\% to 96.2\%, respectively.",

keywords = "CNN, Context information DukeMTMC, DukeMTMC-reID, Person re-identification, Scene understanding, Spatial constraints, Temporal constraints",

author = "Xin Liu and Groot, \{Herman G.J.\} and Egor Bondarev and \{de With\}, \{Peter H.N.\}",

year = "2020",

month = jan,

day = "26",

doi = "10.2352/ISSN.2470-1173.2020.10.IPAS-095",

language = "English",

series = "Electronic Imaging",

publisher = "Society for Imaging Science and Technology (IS\&T)",

booktitle = "Proceedings IS\&T International Symposium on Electronic Imaging",

address = "United States",

note = "18th Image Processing: Algorithms and Systems Conference, IPAS 2020 ; Conference date: 26-01-2020 Through 30-01-2020",

}

Liu, X, Groot, HGJ , Bondarev, E & de With, PHN 2020, Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model. in Proceedings IS&T International Symposium on Electronic Imaging: Image Processing: Algorithms and Systems XVIII, 2020., art00010 , Electronic Imaging, vol. 32, Society for Imaging Science and Technology (IS&T), Springfield, 18th Image Processing: Algorithms and Systems Conference, IPAS 2020, Burlingame, United States, 26/01/20. https://doi.org/10.2352/ISSN.2470-1173.2020.10.IPAS-095

Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model. / Liu, Xin; Groot, Herman G.J.; Bondarev, Egor et al.
Proceedings IS&T International Symposium on Electronic Imaging: Image Processing: Algorithms and Systems XVIII, 2020. Springfield: Society for Imaging Science and Technology (IS&T), 2020. art00010 (Electronic Imaging; Vol. 32).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model

AU - Liu, Xin

AU - Groot, Herman G.J.

AU - Bondarev, Egor

AU - de With, Peter H.N.

PY - 2020/1/26

Y1 - 2020/1/26

N2 - In this paper, we investigate person re-identification (re-ID) in a multi-camera network for surveillance applications. To this end, we create a Spatio-Temporal Multi-Camera model (ST-MC model), which exploits statistical data on a person’s entry/exit points in the multi-camera network, to predict in which camera view a person will re-appear. The created ST-MC model is used as a novel extension to the Multiple Granularity Network (MGN) [1], which is the current state of the art in person re-ID. Compared to existing approaches that are solely based on Convolutional Neural Networks (CNNs), our approach helps to improve the re-ID performance by considering not only appearance-based features of a person from a CNN, but also contextual information. The latter serves as scene understanding information complimentary to person re-ID. Experimental results show that for the DukeMTMC-reID dataset [2][3], introduction of our ST-MC model substantially increases the mean Average Precision (mAP) and Rank-1 score from 77.2% to 84.1%, and from 88.6% to 96.2%, respectively.

AB - In this paper, we investigate person re-identification (re-ID) in a multi-camera network for surveillance applications. To this end, we create a Spatio-Temporal Multi-Camera model (ST-MC model), which exploits statistical data on a person’s entry/exit points in the multi-camera network, to predict in which camera view a person will re-appear. The created ST-MC model is used as a novel extension to the Multiple Granularity Network (MGN) [1], which is the current state of the art in person re-ID. Compared to existing approaches that are solely based on Convolutional Neural Networks (CNNs), our approach helps to improve the re-ID performance by considering not only appearance-based features of a person from a CNN, but also contextual information. The latter serves as scene understanding information complimentary to person re-ID. Experimental results show that for the DukeMTMC-reID dataset [2][3], introduction of our ST-MC model substantially increases the mean Average Precision (mAP) and Rank-1 score from 77.2% to 84.1%, and from 88.6% to 96.2%, respectively.

KW - CNN

KW - Context information DukeMTMC

KW - DukeMTMC-reID

KW - Person re-identification

KW - Scene understanding

KW - Spatial constraints

KW - Temporal constraints

UR - https://www.scopus.com/pages/publications/85094896246

U2 - 10.2352/ISSN.2470-1173.2020.10.IPAS-095

DO - 10.2352/ISSN.2470-1173.2020.10.IPAS-095

M3 - Conference contribution

AN - SCOPUS:85094896246

T3 - Electronic Imaging

BT - Proceedings IS&T International Symposium on Electronic Imaging

PB - Society for Imaging Science and Technology (IS&T)

CY - Springfield

T2 - 18th Image Processing: Algorithms and Systems Conference, IPAS 2020

Y2 - 26 January 2020 through 30 January 2020

ER -

Liu X, Groot HGJ , Bondarev E , de With PHN. Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model. In Proceedings IS&T International Symposium on Electronic Imaging: Image Processing: Algorithms and Systems XVIII, 2020. Springfield: Society for Imaging Science and Technology (IS&T). 2020. art00010 . (Electronic Imaging). doi: 10.2352/ISSN.2470-1173.2020.10.IPAS-095

Introducing Scene Understanding to Person Re-Identification using a Spatio-Temporal Multi-Camera Model

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this