Note on the optimal strategies for the finite-stage Markov game

In this no te we consider the finite-stage Markov game with finitely many states and actions as described by Zachrisson [5]. Zachrisson proves that this game has a value and shows that value and optimal strategies may be determined with a dynamic programming approach. However, he silently assumed that both players would use only Markov strategies. Here we will give a simple proof which shows this restriction to be irrelevant.