Optimal Fixed-Size Controllers for Decentralized POMDPs

Christopher Amato, Daniel S. Bernstein, and Shlomo Zilberstein. Optimal Fixed-Size Controllers for Decentralized POMDPs. Proceedings of the AAMAS Workshop on Multi-Agent Sequential Decision Making in Uncertain Domains (MSDM), 61-71, Hakodate, Japan, May, 2006.

Abstract

Solving decentralized partially observable Markov decision processes (DEC-POMDPs) is a difficult task. Exact solutions are intractable in all but the smallest problems and approximate solutions provide limited optimality guarantees. As a more principled alternative, we present a novel formulation of an optimal fixed-size solution of a DEC-POMDP as a nonlinear program. We discuss the benefits of this representation and evaluate several optimization methods. While the methods used in this paper only guarantee locally optimal solutions, a wide range of powerful nonlinear optimization techniques may now be applied to this problem. We show that by using our formulation in various domains, solution quality is higher than a current state-of-the-art approach. These results show that optimization can be used to provide high quality solutions to DEC-POMDPs while maintaining moderate memory and time usage.

Bibtex entry:

@inproceedings{ABZmsdm06,
  author	= {Christopher Amato and Daniel S. Bernstein and Shlomo Zilberstein},
  title		= {Optimal Fixed-Size Controllers for Decentralized {POMDP}s},
  booktitle     = {Proceedings of the {AAMAS} Workshop on Multi-Agent Sequential 
                   Decision Making in Uncertain Domains},
  year		= {2006},
  pages		= {61-71},
  address       = {Hakodate, Japan},
  url		= {http://rbr.cs.umass.edu/shlomo/papers/ABZmsdm06.html}
}

shlomo@cs.umass.edu