The numerical exploitation of periodicity in Markov decision processes

L.M.M. Veugen, J. Wal, van der, J. Wessels

Research output: Book/ReportReportAcademic

105 Downloads (Pure)


Periodicity is a simple form of nonstationarity in Markov decision processes. In this paper successive approximations are considered for discounted and undiscounted periodic Markov decision processes. For this type of process iteration steps can be performed without any loss of efficiency, provided the type of procedure is sensibly chosen. Moreover, and most important, it is possible to derive sharp bounds for the value function. Numerical evidence is provided.
Original languageEnglish
Place of PublicationEindhoven
PublisherTechnische Hogeschool Eindhoven
Number of pages24
Publication statusPublished - 1982

Publication series

NameMemorandum COSOR
ISSN (Print)0926-4493


Dive into the research topics of 'The numerical exploitation of periodicity in Markov decision processes'. Together they form a unique fingerprint.

Cite this