Preprint: Maximum Causal Entropy Correlated Equilibria for Markov Games

February 22, 2011

Brian D. Ziebart, J. Andrew Bagnell, Anind K. Dey
Carnegie Mellon University
To appear at International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2011).

Link to Paper

Motivated by a machine learning perspective|that game theoretic
equilibria constraints should serve as guidelines for
predicting agents’ strategies, we introduce maximum causal
entropy correlated equilibria (MCECE), a novel solution
concept for general-sum Markov games. In line with this
perspective, a MCECE strategy pro le is a uniquely-de ned
joint probability distribution over actions for each game
state that minimizes the worst-case prediction of agents’ actions
under log-loss. Equivalently, it maximizes the worstcase
growth rate for gambling on the sequences of agents’
joint actions under uniform odds. We present a convex optimization
technique for obtaining MCECE strategy pro les
that resembles value iteration in nite-horizon games. We
assess the predictive bene ts of our approach by predicting
the strategies generated by previously proposed correlated
equilibria solution concepts, and compare against those previous
approaches on that same prediction task.

Previous post:

Next post: