Unbounded-rate Markov decision processes: structural properties via a parametrisation approach

Vertaalde titel van de bijdrage: Markov beslissingsprocessen met onbegrensde sprongintensiteit: structuureigenschappen via parametrisatie

H. Blok

Onderzoeksoutput: ScriptieDissertatie 4 (Onderzoek NIET TU/e / Promotie NIET TU/e)


This research is interested in optimal control of Markov decision processes (MDPs). Herein a key role is played by structural properties. Properties such as monotonicity and convexity help in finding the optimal policy. Value iteration is a tool to derive such properties in discrete time processes. However, in queueing theory there arise problems that can best be modelled as a unbounded-rate continuous time MDP. These processes are not uniformisable and thus value iteration is not available. This thesis builds towards a systemic way for deriving properties, for both disounted and average cost. The procedure that is proposed consist of multiple steps. The first step is to make the MDP uniformisable a truncation needs to be made that keeps the properties intact, we have some recommendations for suitable truncations. In the second step, value iteration can be used to prove the desired structure, we have developed a list of results that can be used for these proofs. As the third step, taking the limit of the truncation to infinity, we have provided conditions that the structures for the truncated processes hold for the unbounded process as well. Applications of this method include the competing queues problem and a server farm problem.
Vertaalde titel van de bijdrageMarkov beslissingsprocessen met onbegrensde sprongintensiteit: structuureigenschappen via parametrisatie
Originele taal-2Engels
KwalificatieDoctor in de Filosofie
Toekennende instantie
  • Universiteit Leiden
  • den Hollander, W.T.F., Promotor
Datum van toekenning23 jun. 2016
Plaats van publicatieLeiden
StatusGepubliceerd - feb. 2016
Extern gepubliceerdJa


Duik in de onderzoeksthema's van 'Markov beslissingsprocessen met onbegrensde sprongintensiteit: structuureigenschappen via parametrisatie'. Samen vormen ze een unieke vingerafdruk.

Citeer dit