UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
A Neuroevolution Approach to General Atari Game Playing (2013)
Matthew Hausknecht
,
Joel Lehman
,
Risto Miikkulainen
, and
Peter Stone
This article addresses the challenge of learning to play many different video games with little domain-specific knowledge. Specifically, it introduces a neuro-evolution approach to general Atari 2600 game playing. Four neuro-evolution algorithms were paired with three different state representations and evaluated on a set of 61 Atari games. The neuro-evolution agents represent different points along the spectrum of algorithmic sophistication - including weight evolution on topologically fixed neural networks (Conventional Neuro-evolution), Covariance Matrix Adaptation Evolution Strategy (CMA-ES), evolution of network topology and weights (NEAT), and indirect network encoding (HyperNEAT). State representations include an object representation of the game screen, the raw pixels of the game screen, and seeded noise (a comparative baseline). Results indicate that direct-encoding methods work best on compact state representations while indirect-encoding methods (i.e. HyperNEAT) allow scaling to higher-dimensional representations (i.e. the raw game screen). Previous approaches based on temporal-difference learning had trouble dealing with the large state spaces and sparse reward gradients often found in Atari games. Neuro-evolution ameliorates these problems and evolved policies achieve state-of-the-art results, even surpassing human high scores on three games. These results suggest that neuro-evolution is a promising approach to general video game playing.
View:
PDF
,
PS
,
HTML
Citation:
IEEE Transactions on Computational Intelligence and AI in Games
(2013).
Bibtex:
@article{hausknecht:tciaig13, title={A Neuroevolution Approach to General Atari Game Playing}, author={Matthew Hausknecht and Joel Lehman and Risto Miikkulainen and Peter Stone}, journal={IEEE Transactions on Computational Intelligence and AI in Games}, url="http://www.cs.utexas.edu/users/ai-lab?hausknecht:tciaig14", year={2013} }
People
Matthew Hausknecht
Formerly affiliated Collaborator
mhauskn [at] cs utexas edu
Joel Lehman
Postdoctoral Alumni
joel [at] cs utexas edu
Risto Miikkulainen
Faculty
risto [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Projects
A Neuroevolution Approach to General Atari Game Playing
2013 - Present
Learning Strategic Behavior in Sequential Decision Tasks
2009 - 2014
Areas of Interest
General Game Playing
Neuroevolution
Demos
A Neuroevolution Approach to General Atari Game Playing
Matthew Hausknecht
2013
Labs
Neural Networks
Learning Agents