Online Implicit Agent Modelling
Nolan Bard, Michael Johanson, Neil Burch, and Michael Bowling. Online Implicit Agent Modelling. In Proceedings of the Twelfth International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2013.Download
Abstract
The traditional view of agent modelling is to infer the explicit parameters of another agent’s strategy (i.e., their probability of taking each action in each situation). Unfortunately, in complex domains with high dimensional strategy spaces, modelling every parameter often requires a prohibitive number of observations. Furthermore, given a model of such a strategy, computing a response strategy that is robust to modelling error may be impractical to compute online. Instead, we propose an implicit modelling framework where agents aim to estimate the utility of a fixed portfolio of pre-computed strategies. Using the domain of heads-up limit Texas hold’em poker, this work describes an end-to-end approach for building an implicit modelling agent. We compute robust response strategies, show how to select strategies for the portfolio, and apply existing variance reduction and online learning techniques to dynamically adapt the agent’s strategy to its opponent. We validate the approach by showing that our implicit modelling agent would have won the heads-up limit opponent exploitation event in the 2011 Annual Computer Poker Competition.
Links
- ACPC Forum thread
- The Imaginary Observations with Importance Sampling paper, which forms the core of the online adaptation aspect of this work.
- The The Data Biased Response paper, which describes the opponent modelling and counter-strategy generation step of our approach.
BibTeX
@InProceedings{ 2013aamas-implicit-modelling,
Title = "Online Implicit Agent Modelling",
Author = "Nolan Bard and Michael Johanson and Neil Burch and Michael Bowling",
Booktitle = "Proceedings of the Twelfth International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS)",
Year = "2013",
}