Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

Narsimlu Kemsaram; Anweshan Das; Gijs Dubbelman

doi:10.1109/ITSC45102.2020.9294435

Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

Narsimlu Kemsaram, Anweshan Das, Gijs Dubbelman

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

4 Citations (Scopus)

Abstract

In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object’s depth and radial distance. It also estimates the object’s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.

Original language	English
Title of host publication	Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece
Publisher	Institute of Electrical and Electronics Engineers
Number of pages	8
ISBN (Electronic)	978-1-7281-4149-7
DOIs	https://doi.org/10.1109/ITSC45102.2020.9294435
Publication status	Published - 24 Dec 2020
Event	23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020 - Rhodes, Greece Duration: 20 Sept 2020 → 23 Sept 2020 https://www.ieee-itsc2020.org/

Conference

Conference	23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020
Abbreviated title	ITSC2020
Country/Territory	Greece
City	Rhodes
Period	20/09/20 → 23/09/20
Internet address	https://www.ieee-itsc2020.org/

Funding

The authors would like to give recognition to the remainder of the i-CAVE team without whom the project would not have been possible. Moreover, we would like to thank Varun Khattar for his support in testing and analysis with the Radar system. This research work is part of the i-CAVE research programme within the Sensing, Mapping and Localization project (project number 10024085). This i-CAVE programme is funded by NWO (Netherlands Organisation for Scientific Research).

Funders	Funder number
Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Keywords

Artificial intelligence
cooperative automated vehicles
deep neural network
stereo vision system

Access to Document

10.1109/ITSC45102.2020.9294435

Cite this

Kemsaram, N., Das, A., & Dubbelman, G. (2020). Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. In Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece Article 9294435 Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ITSC45102.2020.9294435

@inproceedings{85b54de87bf94c70a57dda25bbbbcf24,

title = "Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles",

abstract = "In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object{\textquoteright}s depth and radial distance. It also estimates the object{\textquoteright}s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.",

keywords = "Artificial intelligence, cooperative automated vehicles, deep neural network, stereo vision system",

author = "Narsimlu Kemsaram and Anweshan Das and Gijs Dubbelman",

year = "2020",

month = dec,

day = "24",

doi = "10.1109/ITSC45102.2020.9294435",

language = "English",

booktitle = "Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020, ITSC2020 ; Conference date: 20-09-2020 Through 23-09-2020",

url = "https://www.ieee-itsc2020.org/",

}

Kemsaram, N, Das, A & Dubbelman, G 2020, Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. in Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece., 9294435, Institute of Electrical and Electronics Engineers, 23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020, Rhodes, Greece, 20/09/20. https://doi.org/10.1109/ITSC45102.2020.9294435

Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. / Kemsaram, Narsimlu; Das, Anweshan ; Dubbelman, Gijs.
Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece. Institute of Electrical and Electronics Engineers, 2020. 9294435.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

AU - Kemsaram, Narsimlu

AU - Das, Anweshan

AU - Dubbelman, Gijs

PY - 2020/12/24

Y1 - 2020/12/24

N2 - In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object’s depth and radial distance. It also estimates the object’s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.

AB - In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object’s depth and radial distance. It also estimates the object’s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.

KW - Artificial intelligence

KW - cooperative automated vehicles

KW - deep neural network

KW - stereo vision system

UR - http://www.scopus.com/inward/record.url?scp=85099655835&partnerID=8YFLogxK

U2 - 10.1109/ITSC45102.2020.9294435

DO - 10.1109/ITSC45102.2020.9294435

M3 - Conference contribution

BT - Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece

PB - Institute of Electrical and Electronics Engineers

T2 - 23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020

Y2 - 20 September 2020 through 23 September 2020

ER -

Kemsaram N, Das A , Dubbelman G. Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. In Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece. Institute of Electrical and Electronics Engineers. 2020. 9294435 doi: 10.1109/ITSC45102.2020.9294435

Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

Abstract

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this