Retrieving specific vehicle tracks by Natural Language (NL)-based descriptions is a convenient way to monitor vehicle movement patterns and traffic-related events. NL-based image retrieval has several applications in smart cities, traffic control, etc. In this work, we propose TIED, a text-to-image encoder-decoder model for the simultaneous extraction of visual and textual information for vehicle track retrieval. The model consists of an encoder network that enforces the two modalities into a common latent space and a decoder network that performs an inverse mapping to the text descriptions. The method exploits visual semantic attributes of a target vehicle along with a cycle-consistency loss. The proposed method employs both intra-modal and inter-modal relationships to improve retrieval performance. Our system yields competitive performance achieving the 7th position in the Natural Language-Based Vehicle Retrieval public track of the 2021 NVIDIA AI City Challenge. We demonstrate that the proposed TIED model obtains six times higher Mean Reciprocal Rank (MRR) than the baseline, achieving an MRR of 15.48. The code and models will be made publicly available.
|Title of host publication||Proceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021|
|Publisher||Institute of Electrical and Electronics Engineers|
|Number of pages||9|
|Publication status||Published - 1 Sep 2021|
|Event||2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021 - Nashville, United States|
Duration: 19 Jun 2021 → 25 Jun 2021
|Conference||2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021|
|Abbreviated title||CVPRW 2021|
|Period||19/06/21 → 25/06/21|
Bibliographical notePublisher Copyright:
© 2021 IEEE.
- Image retrieval
- Vehicle re-identification
- Text-to-image retrieval