Optimizing Memory-Bounded Controllers for Decentralized POMDPs
Christopher Amato
Daniel S. Bernstein
Shlomo Zilberstein
Abstract
We present a memory-bounded optimization
approach for solving infinite-horizon decentralized
POMDPs. Policies for each agent
are represented by stochastic finite state controllers.
We formulate the problem of optimizing
these policies as a nonlinear program,
leveraging powerful existing nonlinear optimization
techniques for solving the problem.
While existing solvers only guarantee locally
optimal solutions, we show that our formulation
produces higher quality controllers than
the state-of-the-art approach. We also incorporate
a shared source of randomness in
the form of a correlation device to further
increase solution quality with only a limited
increase in space and time. Our experimental
results show that nonlinear optimization
can be used to provide high quality, concise
solutions to decentralized decision problems
under uncertainty.
Download
[pdf]