Linear Dynamic Programs for Resource Management

Marek Petrik and Shlomo Zilberstein. Linear Dynamic Programs for Resource Management. Proceedings of the Twenty-Fifth Conference on Artificial Intelligence (AAAI), 1377-1383, San Francisco, California, 2011.

Abstract

Sustainable resource management in many domains presents large continuous stochastic optimization prob- lems, which can often be modeled as Markov decision processes (MDPs). To solve such large MDPs, we identify and leverage linearity in state and action sets that is common in resource management. In particular, we introduce linear dynamic programs(LDPs) that generalize resource management problems and partially observable MDPs (POMDPs). We show that the LDP framework makes it possible to adapt point-based methods -- the state of the art in solving POMDPs -- to solving LDPs. The experimental results demonstrate the efficiency of this approach in managing the water level of a river reservoir. Finally, we discuss the relationship with dual dynamic programming, a method used to optimize hydroelectric systems.

Bibtex entry:

@inproceedings{PZaaai11,
  author	= {Marek Petrik and Shlomo Zilberstein},
  title		= {Linear Dynamic Programs for Resource Management},
  booktitle     = {Proceedings of the Twenty-Fifth Conference on
		   Artificial Intelligence},
  year		= {2011},
  pages		= {1377-1383},
  address       = {San Francisco, California},
  url		= {http://rbr.cs.umass.edu/shlomo/papers/PZaaai11.html}
}

shlomo@cs.umass.edu