A reinforcement learning approach for transaction scheduling in a shuttle-based storage and retrieval system

Banu Yetkin Ekren (Corresponding author), Bartu Arslan

Research output: Contribution to journalArticleAcademicpeer-review

7 Citations (Scopus)
42 Downloads (Pure)


With recent Industry 4.0 developments, companies tend to automate their industries. Warehousing companies also take part in this trend. A shuttle-based storage and retrieval system (SBS/RS) is an automated storage and retrieval system technology experiencing recent drastic market growth. This technology is mostly utilized in large distribution centers processing mini-loads. With the recent increase in e-commerce practices, fast delivery requirements with low volume orders have increased. SBS/RS provides ultrahigh-speed load handling due to having an excess amount of shuttles in the system. However, not only the physical design of an automated warehousing technology but also the design of operational system policies would help with fast handling targets. In this work, in an effort to increase the performance of an SBS/RS, we apply a machine learning (ML) (i.e., Q-learning) approach on a newly proposed tier-to-tier SBS/RS design, redesigned from a traditional tier-captive SBS/RS. The novelty of this paper is twofold: First, we propose a novel SBS/RS design where shuttles can travel between tiers in the system; second, due to the complexity of operation of shuttles in that newly proposed design, we implement an ML-based algorithm for transaction selection in that system. The ML-based solution is compared with traditional scheduling approaches: first-in-first-out and shortest process time (i.e., travel) scheduling rules. The results indicate that in most cases, the Q-learning approach performs better than the two static scheduling approaches.
Original languageEnglish
Pages (from-to)274-295
Number of pages22
JournalInternational Transactions in Operational Research
Issue number1
Publication statusPublished - Jan 2024


FundersFunder number
Javna Agencija za Raziskovalno Dejavnost RS118M180
Türkiye Bilimsel ve Teknolojik Araştırma Kurumu


    • SBS/RS
    • reinforcement learning
    • q-learning
    • simulation
    • Warehousing
    • Automated storage/retrieval systems
    • Q-learning
    • automated storage
    • warehousing


    Dive into the research topics of 'A reinforcement learning approach for transaction scheduling in a shuttle-based storage and retrieval system'. Together they form a unique fingerprint.

    Cite this