Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study

M. Geurtsen; Ivo J.B.F. Adan; Zümbül Atan

doi:10.1109/WSC57314.2022.10015402

Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

5 Citations (Scopus)

90 Downloads (Pure)

Abstract

Scheduling in a semiconductor back-end factory is an extremely sophisticated and complex task. In semiconductor industry, more often than not, the scheduling of maintenance is underexposed to production scheduling. This is a missed opportunity as maintenance and production activities are deeply intertwined. This study considers the dynamic scheduling of maintenance activities on an assembly line. A policy is constructed to schedule a cleaning activity on the last machine of an assembly line such that the average production rate is maximized. The policy takes into account the given flexibility and the buffer content of the buffers in-between the machines in the assembly line. A Markov Decision Process is formulated for the problem and solved using Value Iteration and Reinforcement Learning Algorithms. In addition, for a real world case study, a simulation analysis is performed to evaluate the potential practical benefits

Original language	English
Title of host publication	Proceedings of the 2022 Winter Simulation Conference
Subtitle of host publication	Reimagine Tomorrow
Editors	B. Feng, G. Pedrielli, Y. Peng, S. Shashaani, E. Song, C.G. Corlu, L.H. Lee, E.P. Chew, T. Roeder, P. Lendermann
Publisher	Institute of Electrical and Electronics Engineers
Pages	3110-3121
Number of pages	12
ISBN (Electronic)	978-1-6654-7661-4
DOIs	https://doi.org/10.1109/WSC57314.2022.10015402
Publication status	Published - 23 Jan 2023
Event	2022 Winter Simulation Conference, WSC 2022: Reimagine Tomorrow - Singapore, Singapore Duration: 11 Dec 2022 → 14 Dec 2022 https://meetings.informs.org/wordpress/wsc2022/

Conference

Conference	2022 Winter Simulation Conference, WSC 2022
Country/Territory	Singapore
City	Singapore
Period	11/12/22 → 14/12/22
Internet address	https://meetings.informs.org/wordpress/wsc2022/

Access to Document

10.1109/WSC57314.2022.10015402

pdfFinal published version, 571 KBLicence: TAVERNE

Cite this

Geurtsen, M., Adan, I. J. B. F., & Atan, Z. (2023). Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study. In B. Feng, G. Pedrielli, Y. Peng, S. Shashaani, E. Song, C. G. Corlu, L. H. Lee, E. P. Chew, T. Roeder, & P. Lendermann (Eds.), Proceedings of the 2022 Winter Simulation Conference: Reimagine Tomorrow (pp. 3110-3121). Article 10015402 Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/WSC57314.2022.10015402

Geurtsen, M. ; Adan, Ivo J.B.F. ; Atan, Zümbül. / Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach : A Semiconductor Simulation Study. Proceedings of the 2022 Winter Simulation Conference: Reimagine Tomorrow. editor / B. Feng ; G. Pedrielli ; Y. Peng ; S. Shashaani ; E. Song ; C.G. Corlu ; L.H. Lee ; E.P. Chew ; T. Roeder ; P. Lendermann. Institute of Electrical and Electronics Engineers, 2023. pp. 3110-3121

@inproceedings{9e57106c194c4590b1e674849d0dbd58,

title = "Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study",

abstract = "Scheduling in a semiconductor back-end factory is an extremely sophisticated and complex task. In semiconductor industry, more often than not, the scheduling of maintenance is underexposed to production scheduling. This is a missed opportunity as maintenance and production activities are deeply intertwined. This study considers the dynamic scheduling of maintenance activities on an assembly line. A policy is constructed to schedule a cleaning activity on the last machine of an assembly line such that the average production rate is maximized. The policy takes into account the given flexibility and the buffer content of the buffers in-between the machines in the assembly line. A Markov Decision Process is formulated for the problem and solved using Value Iteration and Reinforcement Learning Algorithms. In addition, for a real world case study, a simulation analysis is performed to evaluate the potential practical benefits",

author = "M. Geurtsen and Adan, \{Ivo J.B.F.\} and Z{\"u}mb{\"u}l Atan",

year = "2023",

month = jan,

day = "23",

doi = "10.1109/WSC57314.2022.10015402",

language = "English",

pages = "3110--3121",

editor = "B. Feng and G. Pedrielli and Y. Peng and S. Shashaani and E. Song and C.G. Corlu and L.H. Lee and E.P. Chew and T. Roeder and P. Lendermann",

booktitle = "Proceedings of the 2022 Winter Simulation Conference",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "2022 Winter Simulation Conference, WSC 2022 : Reimagine Tomorrow ; Conference date: 11-12-2022 Through 14-12-2022",

url = "https://meetings.informs.org/wordpress/wsc2022/",

}

Geurtsen, M, Adan, IJBF & Atan, Z 2023, Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study. in B Feng, G Pedrielli, Y Peng, S Shashaani, E Song, CG Corlu, LH Lee, EP Chew, T Roeder & P Lendermann (eds), Proceedings of the 2022 Winter Simulation Conference: Reimagine Tomorrow., 10015402, Institute of Electrical and Electronics Engineers, pp. 3110-3121, 2022 Winter Simulation Conference, WSC 2022, Singapore, Singapore, 11/12/22. https://doi.org/10.1109/WSC57314.2022.10015402

Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study. / Geurtsen, M.; Adan, Ivo J.B.F.; Atan, Zümbül.
Proceedings of the 2022 Winter Simulation Conference: Reimagine Tomorrow. ed. / B. Feng; G. Pedrielli; Y. Peng; S. Shashaani; E. Song; C.G. Corlu; L.H. Lee; E.P. Chew; T. Roeder; P. Lendermann. Institute of Electrical and Electronics Engineers, 2023. p. 3110-3121 10015402.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach

T2 - 2022 Winter Simulation Conference, WSC 2022

AU - Geurtsen, M.

AU - Adan, Ivo J.B.F.

AU - Atan, Zümbül

PY - 2023/1/23

Y1 - 2023/1/23

N2 - Scheduling in a semiconductor back-end factory is an extremely sophisticated and complex task. In semiconductor industry, more often than not, the scheduling of maintenance is underexposed to production scheduling. This is a missed opportunity as maintenance and production activities are deeply intertwined. This study considers the dynamic scheduling of maintenance activities on an assembly line. A policy is constructed to schedule a cleaning activity on the last machine of an assembly line such that the average production rate is maximized. The policy takes into account the given flexibility and the buffer content of the buffers in-between the machines in the assembly line. A Markov Decision Process is formulated for the problem and solved using Value Iteration and Reinforcement Learning Algorithms. In addition, for a real world case study, a simulation analysis is performed to evaluate the potential practical benefits

AB - Scheduling in a semiconductor back-end factory is an extremely sophisticated and complex task. In semiconductor industry, more often than not, the scheduling of maintenance is underexposed to production scheduling. This is a missed opportunity as maintenance and production activities are deeply intertwined. This study considers the dynamic scheduling of maintenance activities on an assembly line. A policy is constructed to schedule a cleaning activity on the last machine of an assembly line such that the average production rate is maximized. The policy takes into account the given flexibility and the buffer content of the buffers in-between the machines in the assembly line. A Markov Decision Process is formulated for the problem and solved using Value Iteration and Reinforcement Learning Algorithms. In addition, for a real world case study, a simulation analysis is performed to evaluate the potential practical benefits

UR - https://www.scopus.com/pages/publications/85147427275

U2 - 10.1109/WSC57314.2022.10015402

DO - 10.1109/WSC57314.2022.10015402

M3 - Conference contribution

SP - 3110

EP - 3121

BT - Proceedings of the 2022 Winter Simulation Conference

A2 - Feng, B.

A2 - Pedrielli, G.

A2 - Peng, Y.

A2 - Shashaani, S.

A2 - Song, E.

A2 - Corlu, C.G.

A2 - Lee, L.H.

A2 - Chew, E.P.

A2 - Roeder, T.

A2 - Lendermann, P.

PB - Institute of Electrical and Electronics Engineers

Y2 - 11 December 2022 through 14 December 2022

ER -

Geurtsen M, Adan IJBF , Atan Z. Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study. In Feng B, Pedrielli G, Peng Y, Shashaani S, Song E, Corlu CG, Lee LH, Chew EP, Roeder T, Lendermann P, editors, Proceedings of the 2022 Winter Simulation Conference: Reimagine Tomorrow. Institute of Electrical and Electronics Engineers. 2023. p. 3110-3121. 10015402 doi: 10.1109/WSC57314.2022.10015402

Dynamic Scheduling of Maintenance by a Reinforcement Learning Approach: A Semiconductor Simulation Study

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this