This paper is a revised and extended version of Memorandum COSOR 75-22
Strongly excessive functions play an important role in the theory of Markov decision processes and Markov games. In this paper the following question is investigated: What are the probabilistic properties of Markov decision processes which posses a strongly excessive function? A characterization is presented 1n the form of a random drift through a partitioned state space. For strongly excessive functions which have a positive lower bound a characterization is given in terms of the lifetime distribution of the process. Finally we give a characterization in terms of the spectral radius.
Key words: Markov decision process, excessive function, transient behaviour, exponentially bounded stopping time, spectral radius.
|ISSN van geprinte versie||0926-4493|