UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Protecting Against Evaluation Overfitting in Empirical Reinforcement Learning (2011)
Shimon Whiteson, Brian Tanner,
Matthew E. Taylor
, and
Peter Stone
Empirical evaluations play an important role in machine learning. However, the usefulness of any evaluation depends on the
empirical methodology
employed. Designing good empirical methodologies is difficult in part because agents can
overfit
test evaluations and thereby obtain misleadingly high scores. We argue that reinforcement learning is particularly vulnerable to
environment overfitting
and propose as a remedy
generalized methodologies
, in which evaluations are based on multiple environments sampled from a distribution. In addition, we consider how to summarize performance when scores from different environments may not have commensurate values. Finally, we present proof-of-concept results demonstrating how these methodologies can validate an intuitively useful range-adaptive tile coding method.
View:
PDF
,
PS
,
HTML
Citation:
In
{IEEE} Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)
, April 2011.
Bibtex:
@InProceedings{ADPRL11-shimon, title={Protecting Against Evaluation Overfitting in Empirical Reinforcement Learning}, author={Shimon Whiteson and Brian Tanner and Matthew E. Taylor and Peter Stone}, booktitle={{IEEE} Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)}, month={April}, url="http://www.cs.utexas.edu/users/ai-lab?ADPRL11-shimon", year={2011} }
People
Peter Stone
Faculty
pstone [at] cs utexas edu
Matthew Taylor
Ph.D. Alumni
taylorm [at] eecs wsu edu
Areas of Interest
Reinforcement Learning
Labs
Learning Agents