Learn from IoT: pedestrian detection and intention prediction for autonomous driving

Gürkan Solmaz, Everton Luís Berz, Marzieh Farahani Dolatabadi, Samet Aytaç, Jonathan Fürst, Bin Cheng, Jos den Ouden

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

Uittreksel

This paper explores the potential of machine learning (ML) systems which use data from in-vehicle sensors as well as external IoT data sources to enhance autonomous driving for efficiency and safety in urban environments. We propose a system which combines sensor data from autonomous vehicles and IoT data collected from pedestrians' mobile devices. Our approach includes two methods for vulnerable road user (VRU) detection and pedestrian movement intention prediction, and a model to combine the two outputs for potentially improving the autonomous decision-making. The first method creates a world model (WM) and accurately localizes VRUs using in-vehicle cameras and external mobile device data. The second method is a deep learning model to predict pedestrian's next movement steps using real-time trajectory and training with historical mobile device data. To test the system, we conduct three pilot tests at a university campus with a custom-built autonomous car and mobile devices carried by pedestrians. The results from our controlled experiments show that VRU detection can more accurately distinguish locations of pedestrians using IoT data. Furthermore, up to five future steps of pedestrians can be predicted within 2 m.
Originele taal-2Engels
TitelSMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019
Plaats van productieNew York
UitgeverijACM/IEEE
Pagina's27-32
Aantal pagina's6
ISBN van elektronische versie978-1-4503-6930-5
DOI's
StatusGepubliceerd - 4 okt 2019
EvenementSMAS '19 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability - Los Cabos, Mexico
Duur: 21 okt 201921 okt 2019

Congres

CongresSMAS '19 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability
LandMexico
StadLos Cabos
Periode21/10/1921/10/19

Vingerafdruk

Mobile devices
Learning systems
Sensors
Railroad cars
Decision making
Cameras
Trajectories
Internet of things
Experiments

Citeer dit

Solmaz, G., Berz, E. L., Dolatabadi, M. F., Aytaç, S., Fürst, J., Cheng, B., & Ouden, J. D. (2019). Learn from IoT: pedestrian detection and intention prediction for autonomous driving. In SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019 (blz. 27-32). New York: ACM/IEEE. https://doi.org/10.1145/3349622.3355446
Solmaz, Gürkan ; Berz, Everton Luís ; Dolatabadi, Marzieh Farahani ; Aytaç, Samet ; Fürst, Jonathan ; Cheng, Bin ; Ouden, Jos den. / Learn from IoT : pedestrian detection and intention prediction for autonomous driving. SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019. New York : ACM/IEEE, 2019. blz. 27-32
@inproceedings{2912959f271d429e84bdcf7ed5001497,
title = "Learn from IoT: pedestrian detection and intention prediction for autonomous driving",
abstract = "This paper explores the potential of machine learning (ML) systems which use data from in-vehicle sensors as well as external IoT data sources to enhance autonomous driving for efficiency and safety in urban environments. We propose a system which combines sensor data from autonomous vehicles and IoT data collected from pedestrians' mobile devices. Our approach includes two methods for vulnerable road user (VRU) detection and pedestrian movement intention prediction, and a model to combine the two outputs for potentially improving the autonomous decision-making. The first method creates a world model (WM) and accurately localizes VRUs using in-vehicle cameras and external mobile device data. The second method is a deep learning model to predict pedestrian's next movement steps using real-time trajectory and training with historical mobile device data. To test the system, we conduct three pilot tests at a university campus with a custom-built autonomous car and mobile devices carried by pedestrians. The results from our controlled experiments show that VRU detection can more accurately distinguish locations of pedestrians using IoT data. Furthermore, up to five future steps of pedestrians can be predicted within 2 m.",
keywords = "autonomous vehicles, deep neural networks, internet of things, vulnerable road user detection, Internet of things, Deep neural networks, Vulnerable road user detection, Autonomous vehicles",
author = "G{\"u}rkan Solmaz and Berz, {Everton Lu{\'i}s} and Dolatabadi, {Marzieh Farahani} and Samet Ayta{\cc} and Jonathan F{\"u}rst and Bin Cheng and Ouden, {Jos den}",
year = "2019",
month = "10",
day = "4",
doi = "10.1145/3349622.3355446",
language = "English",
pages = "27--32",
booktitle = "SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019",
publisher = "ACM/IEEE",

}

Solmaz, G, Berz, EL, Dolatabadi, MF, Aytaç, S, Fürst, J, Cheng, B & Ouden, JD 2019, Learn from IoT: pedestrian detection and intention prediction for autonomous driving. in SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019. ACM/IEEE, New York, blz. 27-32, SMAS '19 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, Los Cabos, Mexico, 21/10/19. https://doi.org/10.1145/3349622.3355446

Learn from IoT : pedestrian detection and intention prediction for autonomous driving. / Solmaz, Gürkan; Berz, Everton Luís; Dolatabadi, Marzieh Farahani; Aytaç, Samet; Fürst, Jonathan; Cheng, Bin; Ouden, Jos den.

SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019. New York : ACM/IEEE, 2019. blz. 27-32.

Onderzoeksoutput: Hoofdstuk in Boek/Rapport/CongresprocedureConferentiebijdrageAcademicpeer review

TY - GEN

T1 - Learn from IoT

T2 - pedestrian detection and intention prediction for autonomous driving

AU - Solmaz, Gürkan

AU - Berz, Everton Luís

AU - Dolatabadi, Marzieh Farahani

AU - Aytaç, Samet

AU - Fürst, Jonathan

AU - Cheng, Bin

AU - Ouden, Jos den

PY - 2019/10/4

Y1 - 2019/10/4

N2 - This paper explores the potential of machine learning (ML) systems which use data from in-vehicle sensors as well as external IoT data sources to enhance autonomous driving for efficiency and safety in urban environments. We propose a system which combines sensor data from autonomous vehicles and IoT data collected from pedestrians' mobile devices. Our approach includes two methods for vulnerable road user (VRU) detection and pedestrian movement intention prediction, and a model to combine the two outputs for potentially improving the autonomous decision-making. The first method creates a world model (WM) and accurately localizes VRUs using in-vehicle cameras and external mobile device data. The second method is a deep learning model to predict pedestrian's next movement steps using real-time trajectory and training with historical mobile device data. To test the system, we conduct three pilot tests at a university campus with a custom-built autonomous car and mobile devices carried by pedestrians. The results from our controlled experiments show that VRU detection can more accurately distinguish locations of pedestrians using IoT data. Furthermore, up to five future steps of pedestrians can be predicted within 2 m.

AB - This paper explores the potential of machine learning (ML) systems which use data from in-vehicle sensors as well as external IoT data sources to enhance autonomous driving for efficiency and safety in urban environments. We propose a system which combines sensor data from autonomous vehicles and IoT data collected from pedestrians' mobile devices. Our approach includes two methods for vulnerable road user (VRU) detection and pedestrian movement intention prediction, and a model to combine the two outputs for potentially improving the autonomous decision-making. The first method creates a world model (WM) and accurately localizes VRUs using in-vehicle cameras and external mobile device data. The second method is a deep learning model to predict pedestrian's next movement steps using real-time trajectory and training with historical mobile device data. To test the system, we conduct three pilot tests at a university campus with a custom-built autonomous car and mobile devices carried by pedestrians. The results from our controlled experiments show that VRU detection can more accurately distinguish locations of pedestrians using IoT data. Furthermore, up to five future steps of pedestrians can be predicted within 2 m.

KW - autonomous vehicles, deep neural networks, internet of things, vulnerable road user detection

KW - Internet of things

KW - Deep neural networks

KW - Vulnerable road user detection

KW - Autonomous vehicles

UR - http://www.scopus.com/inward/record.url?scp=85076404579&partnerID=8YFLogxK

U2 - 10.1145/3349622.3355446

DO - 10.1145/3349622.3355446

M3 - Conference contribution

SP - 27

EP - 32

BT - SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019

PB - ACM/IEEE

CY - New York

ER -

Solmaz G, Berz EL, Dolatabadi MF, Aytaç S, Fürst J, Cheng B et al. Learn from IoT: pedestrian detection and intention prediction for autonomous driving. In SMAS 2019 - Proceedings of the 1st ACM Workshop on Emerging Smart Technologies and Infrastructures for Smart Mobility and Sustainability, co-located with MobiCom 2019. New York: ACM/IEEE. 2019. blz. 27-32 https://doi.org/10.1145/3349622.3355446