Markov decision processes and strongly excessive functions

K.M. Hee, van, J. Wessels

Research output: Book/ReportReportAcademic

54 Downloads (Pure)


This paper is a revised and extended version of Memorandum COSOR 75-22 Strongly excessive functions play an important role in the theory of Markov decision processes and Markov games. In this paper the following question is investigated: What are the probabilistic properties of Markov decision processes which posses a strongly excessive function? A characterization is presented 1n the form of a random drift through a partitioned state space. For strongly excessive functions which have a positive lower bound a characterization is given in terms of the lifetime distribution of the process. Finally we give a characterization in terms of the spectral radius. Key words: Markov decision process, excessive function, transient behaviour, exponentially bounded stopping time, spectral radius.
Original languageEnglish
Place of PublicationEindhoven
PublisherTechnische Hogeschool Eindhoven
Number of pages22
Publication statusPublished - 1977

Publication series

NameMemorandum COSOR
ISSN (Print)0926-4493

Fingerprint Dive into the research topics of 'Markov decision processes and strongly excessive functions'. Together they form a unique fingerprint.

Cite this