• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Data-Efficient Policy Evaluation Through Behavior Policy Search.
Josiah
Hanna, Philip Thomas, Peter Stone, and Scott
Niekum.
In Proceedings of the 34th International Conference on Machine Learning (ICML), August 2017.
We consider the task of evaluating a policy for a Markov decision process (MDP). The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance. We show that the data collected from deploying a different policy, commonly called the behavior policy, can be used to produce unbiased estimates with lower mean squared error than this standard technique. We derive an analytic expression for the optimal behavior policy---the behavior policy that minimizes the mean squared error of the resulting estimates. Because this expression depends on terms that are unknown in practice, we propose a novel policy evaluation sub-problem, behavior policy search: searching for a behavior policy that reduces mean squared error. We present a behavior policy search algorithm and empirically demonstrate its effectiveness in lowering the mean squared error of policy performance estimates.
@InProceedings{ICML17-Hanna,
author = {Josiah Hanna and Philip Thomas and Peter Stone and Scott Niekum},
title = {Data-Efficient Policy Evaluation Through Behavior Policy Search},
booktitle = {Proceedings of the 34th International Conference on Machine Learning (ICML)},
location = {Sydney, Australia},
month = {August},
year = {2017},
abstract = {
We consider the task of evaluating a policy for a Markov decision process
(MDP). The standard unbiased technique for evaluating a policy is to deploy
the policy and observe its performance. We show that the data collected from
deploying a different policy, commonly called the behavior policy, can be
used to produce unbiased estimates with lower mean squared error than this
standard technique. We derive an analytic expression for the optimal behavior
policy---the behavior policy that minimizes the mean squared error of the
resulting estimates. Because this expression depends on terms that are
unknown in practice, we propose a novel policy evaluation sub-problem,
behavior policy search: searching for a behavior policy that reduces mean
squared error. We present a behavior policy search algorithm and empirically
demonstrate its effectiveness in lowering the mean squared error of policy
performance estimates.
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Apr 17, 2024 18:42:52