TIED: A Cycle Consistent Encoder-Decoder Model for Text-to-Image Retrieval

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

1 Citation (Scopus)

Abstract

Retrieving specific vehicle tracks by Natural Language (NL)-based descriptions is a convenient way to monitor vehicle movement patterns and traffic-related events. NL-based image retrieval has several applications in smart cities, traffic control, etc. In this work, we propose TIED, a text-to-image encoder-decoder model for the simultaneous extraction of visual and textual information for vehicle track retrieval. The model consists of an encoder network that enforces the two modalities into a common latent space and a decoder network that performs an inverse mapping to the text descriptions. The method exploits visual semantic attributes of a target vehicle along with a cycle-consistency loss. The proposed method employs both intra-modal and inter-modal relationships to improve retrieval performance. Our system yields competitive performance achieving the 7th position in the Natural Language-Based Vehicle Retrieval public track of the 2021 NVIDIA AI City Challenge. We demonstrate that the proposed TIED model obtains six times higher Mean Reciprocal Rank (MRR) than the baseline, achieving an MRR of 15.48. The code and models will be made publicly available.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
PublisherInstitute of Electrical and Electronics Engineers
Pages4133-4141
Number of pages9
ISBN (Electronic)9781665448994
DOIs
Publication statusPublished - 1 Sep 2021
Event2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021 - Nashville, United States
Duration: 19 Jun 202125 Jun 2021

Conference

Conference2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2021
Abbreviated titleCVPRW 2021
Country/TerritoryUnited States
CityNashville
Period19/06/2125/06/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Keywords

  • NLP
  • Image retrieval
  • Vehicle re-identification
  • Text-to-image retrieval

Fingerprint

Dive into the research topics of 'TIED: A Cycle Consistent Encoder-Decoder Model for Text-to-Image Retrieval'. Together they form a unique fingerprint.

Cite this