Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

Narsimlu Kemsaram; Anweshan Das; Gijs Dubbelman

doi:10.1109/ITSC45102.2020.9294435

Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

Narsimlu Kemsaram, Anweshan Das, Gijs Dubbelman

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

4 Citaten (Scopus)

Samenvatting

In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object’s depth and radial distance. It also estimates the object’s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.

Originele taal-2	Engels
Titel	Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece
Uitgeverij	Institute of Electrical and Electronics Engineers
Aantal pagina's	8
ISBN van elektronische versie	978-1-7281-4149-7
DOI's	https://doi.org/10.1109/ITSC45102.2020.9294435
Status	Gepubliceerd - 24 dec. 2020
Evenement	23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020 - Rhodes, Griekenland Duur: 20 sep. 2020 → 23 sep. 2020 https://www.ieee-itsc2020.org/

Congres

Congres	23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020
Verkorte titel	ITSC2020
Land/Regio	Griekenland
Stad	Rhodes
Periode	20/09/20 → 23/09/20
Internet adres	https://www.ieee-itsc2020.org/

Financiering

The authors would like to give recognition to the remainder of the i-CAVE team without whom the project would not have been possible. Moreover, we would like to thank Varun Khattar for his support in testing and analysis with the Radar system. This research work is part of the i-CAVE research programme within the Sensing, Mapping and Localization project (project number 10024085). This i-CAVE programme is funded by NWO (Netherlands Organisation for Scientific Research).

Financiers	Financiernummer
Nederlandse Organisatie voor Wetenschappelijk Onderzoek

Toegang tot document

10.1109/ITSC45102.2020.9294435

Andere bestanden en links

Link to publication in Scopus

Citeer dit

Kemsaram, N., Das, A., & Dubbelman, G. (2020). Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. In Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece Artikel 9294435 Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ITSC45102.2020.9294435

@inproceedings{85b54de87bf94c70a57dda25bbbbcf24,

title = "Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles",

abstract = "In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object{\textquoteright}s depth and radial distance. It also estimates the object{\textquoteright}s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.",

keywords = "Artificial intelligence, cooperative automated vehicles, deep neural network, stereo vision system",

author = "Narsimlu Kemsaram and Anweshan Das and Gijs Dubbelman",

year = "2020",

month = dec,

day = "24",

doi = "10.1109/ITSC45102.2020.9294435",

language = "English",

booktitle = "Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020, ITSC2020 ; Conference date: 20-09-2020 Through 23-09-2020",

url = "https://www.ieee-itsc2020.org/",

}

Kemsaram, N, Das, A & Dubbelman, G 2020, Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. in Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece., 9294435, Institute of Electrical and Electronics Engineers, 23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020, Rhodes, Griekenland, 20/09/20. https://doi.org/10.1109/ITSC45102.2020.9294435

Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. / Kemsaram, Narsimlu; Das, Anweshan ; Dubbelman, Gijs.
Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece. Institute of Electrical and Electronics Engineers, 2020. 9294435.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/Congresprocedure › Conferentiebijdrage › Academic › peer review

TY - GEN

T1 - Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

AU - Kemsaram, Narsimlu

AU - Das, Anweshan

AU - Dubbelman, Gijs

PY - 2020/12/24

Y1 - 2020/12/24

N2 - In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object’s depth and radial distance. It also estimates the object’s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.

AB - In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object’s depth and radial distance. It also estimates the object’s relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.

KW - Artificial intelligence

KW - cooperative automated vehicles

KW - deep neural network

KW - stereo vision system

UR - http://www.scopus.com/inward/record.url?scp=85099655835&partnerID=8YFLogxK

U2 - 10.1109/ITSC45102.2020.9294435

DO - 10.1109/ITSC45102.2020.9294435

M3 - Conference contribution

BT - Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece

PB - Institute of Electrical and Electronics Engineers

T2 - 23rd IEEE International Conference on Intelligent Transportation Systems, ITSC 2020

Y2 - 20 September 2020 through 23 September 2020

ER -

Kemsaram N, Das A , Dubbelman G. Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles. In Proceedings of 23rd IEEE International Conference on Intelligent Transportation Systems 2020 (IEEE ITSC 2020), Rhodes, Greece. Institute of Electrical and Electronics Engineers. 2020. 9294435 doi: 10.1109/ITSC45102.2020.9294435

Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

Samenvatting

Congres

Financiering

Toegang tot document

Andere bestanden en links

Vingerafdruk

Citeer dit