Dynamic server assignment in an extended machine-repair model

We consider an extension of the classical machine-repair problem. The machines, apart from receiving service from a single repairman, now also supply service themselves to queues of products. The extended model can be viewed as a two-layered queueing network, in which the queues of products in the ¿rst layer are generally correlated, due to the fact that the machines have to share the repairman’s capacity in the second layer. We are concerned with the dynamic control problem how the repairman should allocate his capacity to the machines at any point in time so that the long-term average (weighted) sum of the queue lengths of the ¿rst-layer queues is minimised. Since the optimal policy for the repairman cannot be found analytically due to the correlations in the queue lengths, we propose a near-optimal policy. We do this by combining intuition and results from queueing theory with techniques from Markov decision theory. Speci¿cally, we study the relative value functions for several policies for which the model can be decomposed in less complicated subsystems, and we combine the results with the classical one-step policy improvement algorithm. The resulting policy is easy to apply, scalable in the number of machines and is shown to be highly accurate over a wide range of parameter settings.