Shunting trains with deep reinforcement learning

Evertjan Peer, V. Menkovski, Y. Zhang, Wan-Jui Lee

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

17 Citations (Scopus)
840 Downloads (Pure)


The Train Unit Shunting Problem (TUSP) is a difficult sequential decision making problem faced by Dutch Railways (NS). Current heuristic solutions under study at NS fall short in accounting for uncertainty during plan execution and do not efficiently support replanning. Furthermore, the resulting plans lack consistency. We approach the TUSP by formulating it as a Markov Decision Process and develop an image-like state space representation that allows us to develop a Deep Reinforcement Learning (DRL) solution. The Deep Q-Network efficiently reduces the state space and develops an on-line strategy for the TUSP capable of dealing with uncertainty and delivering significantly more consistent solutions compared to approaches currently being developed by NS.
Original languageEnglish
Title of host publication2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
Number of pages6
ISBN (Electronic)9781538666500
Publication statusPublished - 2018
Event2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC2018 - Miyazaki, Japan
Duration: 7 Oct 201810 Oct 2018


Conference2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC2018
Abbreviated titleSMC 2018
Internet address


Dive into the research topics of 'Shunting trains with deep reinforcement learning'. Together they form a unique fingerprint.

Cite this