Dynamic Programming for Partially Observable Stochastic Games

Eric A. Hansen, Daniel S. Bernstein, and Shlomo Zilberstein. Dynamic Programming for Partially Observable Stochastic Games. Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI), 709-715, San Jose, California, 2004.

Abstract

We develop an exact dynamic programming algorithm for partially observable stochastic games (POSGs). The algorithm is a synthesis of dynamic programming for partially observable Markov decision processes (POMDPs) and iterated elimination of dominated strategies in normal form games. We prove that when applied to finite-horizon POSGs, the algorithm iteratively eliminates very weakly dominated strategies without first forming a normal form representation of the game. For the special case in which agents share the same payoffs, the algorithm can be used to find an optimal solution. We present preliminary empirical results and discuss ways to further exploit POMDP theory in solving POSGs.

Bibtex entry:

@inproceedings{HBZaaai04,
  author	= {Eric A. Hansen and Daniel S. Bernstein and Shlomo Zilberstein},
  title		= {Dynamic Programming for Partially Observable Stochastic Games},
  booktitle     = {Proceedings of the Nineteenth National Conference on Artificial
                   Intelligence},
  year		= {2004},
  pages		= {709-715},
  address       = {San Jose, California},
  url		= {http://rbr.cs.umass.edu/shlomo/papers/HBZaaai04.html}
}

shlomo@cs.umass.edu