Integrated Learning and Control for Critical Systems

We study the integrated optimization of condition-based maintenance and process control of critical manufacturing equipment that is used for production processes and is subject to failures. Thereto we assume that there is a process characteristic that is indicative of both the production output quality and the state of the equipment. To model the decreased production output quality, we assume that operational costs are increasing in the absolute deviation of the process characteristic from the perfect operational state, while the equipment fails if this absolute deviation exceeds a certain threshold. The evolution of the process characteristic is modelled as a Brownian motion with an a-priori unknown drift that needs to be learned from observations of the process at equidistant epochs. To this end, we propose a Bayesian procedure to infer this unknown parameter from data on-the-fly with increasing accuracy. A decision maker responsible for the critical manufacturing equipment faces a sequential decision problem: At each epoch, either preventive maintenance is performed or the production process is continued with the risk of sudden equipment failure and/or inferior production output. Using our Bayesian framework, we formulate this sequential replacement problem as a Bayesian Markov decision process so that learning and decision-making are integrated. We establish the optimality of an intuitive and easy-to-implement bandwidth policy, under both the average and discounted cost criterion. This policy prescribes to continue production as long as the process characteristic is within a bandwidth that is bounded by dynamic control limits. Numerical results show that this data-driven policy performs excellently compared to a policy that a-priori knows the true drift and that it is robust against misspecification of the initial parameters.