Solving the Online Batching Problem using Deep Reinforcement Learning

Bram Cals, Yingqian Zhang (Corresponding author), Remco M. Dijkman, Claudy van Dorst

Onderzoeksoutput: Bijdrage aan tijdschriftTijdschriftartikelAcademicpeer review

13 Citaten (Scopus)


In e-commerce markets, on-time delivery is of great importance to customer satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL) approach, together with a heuristic, for deciding how and when arrived orders should be batched and picked in a warehouse to minimize the number of tardy orders. In particular, the technique facilitates making decisions on whether an order should be picked individually (pick-by-order) or picked in a batch with other orders (pick-by-batch), and if so, with which other orders. We approach the problem by formulating it as a semi-Markov decision process and developing a vector-based state representation that includes the characteristics of the warehouse system. This allows us to create a deep reinforcement learning solution that learns a strategy by interacting with the environment and solve the problem with a proximal policy optimization algorithm. We evaluate the performance of the proposed DRL approach by comparing it with several batching and sequencing heuristics in different problem settings. The results show that the DRL approach can develop a strategy that produces consistent, good solutions and performs better than the proposed heuristics in most of the tested cases. We show that the strategy learned by DRL is different from the hand-crafted heuristics.

In this paper, we demonstrate that the benefits from recent advancements of Deep Reinforcement Learning can be transferred to solve sequential decision-making problems in warehousing operations.

Deep reinforcement learningOrder batchingSequential decision makingMachine learningWarehousingE-commerce
Originele taal-2Engels
Aantal pagina's15
TijdschriftComputers & Industrial Engineering
Vroegere onlinedatum11 mrt. 2021
StatusGepubliceerd - jun. 2021


Duik in de onderzoeksthema's van 'Solving the Online Batching Problem using Deep Reinforcement Learning'. Samen vormen ze een unieke vingerafdruk.

Citeer dit