Multi-Objective MDPs with Conditional Lexicographic Reward Preferences

Kyle Hollins Wray, Shlomo Zilberstein, and Abdel-Illah Mouaddib. Multi-Objective MDPs with Conditional Lexicographic Reward Preferences. Proceedings of the Twenty-Ninth Conference on Artificial Intelligence (AAAI), 3418-3424, Austin, Texas, 2015.

Abstract

Sequential decision problems that involve multiple objectives are prevalent. Consider for example a driver of a semi-autonomous car who may want to optimize competing objectives such as travel time and the effort associated with manual driving. We introduce a rich model called Lexicographic MDP (LMDP) and a corresponding planning algorithm called LVI that generalize previous work by allowing for conditional lexicographic preferences with slack. We analyze the convergence characteristics of LVI and establish its game theoretic properties. The performance of LVI in practice is tested within a realistic benchmark problem in the domain of semi-autonomous driving. Finally, we demonstrate how GPU-based optimization can improve the scalability of LVI and other value iteration algorithms for MDPs.

Bibtex entry:

@inproceedings{WZMaaai15,
  author	= {Kyle Hollins Wray and Shlomo Zilberstein and Abdel-Illah Mouaddib},
  title		= {Multi-Objective MDPs with Conditional Lexicographic
                   Reward Preferences},
  booktitle     = {Proceedings of the Twenty-Ninth Conference on Artificial
                   Intelligence},
  year		= {2015},
  pages		= {3418-3424},
  address       = {Austin, Texas},
  url		= {http://rbr.cs.umass.edu/shlomo/papers/WZMaaai15.html}
}

shlomo@cs.umass.edu