Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Data-Efficient Policy Evaluation Through Behavior Policy Search

Josiah Hanna, Philip Thomas, Peter Stone, and Scott Niekum. Data-Efficient Policy Evaluation Through Behavior Policy Search. In Proceedings of the 34th International Conference on Machine Learning (ICML), August 2017.

Download

[PDF]1.2MB  [slides.pdf]1.1MB  

Abstract

We consider the task of evaluating a policy for a Markov decision process (MDP). The standard unbiased technique for evaluating a policy is to deploy the policy and observe its performance. We show that the data collected from deploying a different policy, commonly called the behavior policy, can be used to produce unbiased estimates with lower mean squared error than this standard technique. We derive an analytic expression for the optimal behavior policy---the behavior policy that minimizes the mean squared error of the resulting estimates. Because this expression depends on terms that are unknown in practice, we propose a novel policy evaluation sub-problem, behavior policy search: searching for a behavior policy that reduces mean squared error. We present a behavior policy search algorithm and empirically demonstrate its effectiveness in lowering the mean squared error of policy performance estimates.

BibTeX Entry

@InProceedings{ICML17-Hanna,
  author = {Josiah Hanna and Philip Thomas and Peter Stone and Scott Niekum},
  title = {Data-Efficient Policy Evaluation Through Behavior Policy Search},
  booktitle = {Proceedings of the 34th International Conference on Machine Learning (ICML)},
  location = {Sydney, Australia},
  month = {August},
  year = {2017},
  abstract = {
  We consider the task of evaluating a policy for a Markov decision process
  (MDP). The standard unbiased technique for evaluating a policy is to deploy
  the policy and observe its performance. We show that the data collected from
  deploying a different policy, commonly called the behavior policy, can be
  used to produce unbiased estimates with lower mean squared error than this
  standard technique. We derive an analytic expression for the optimal behavior
  policy---the behavior policy that minimizes the mean squared error of the
  resulting estimates. Because this expression depends on terms that are
  unknown in practice, we propose a novel policy evaluation sub-problem,
  behavior policy search: searching for a behavior policy that reduces mean
  squared error. We present a behavior policy search algorithm and empirically
  demonstrate its effectiveness in lowering the mean squared error of policy
  performance estimates.
  },
}

Generated by bib2html.pl (written by Patrick Riley ) on Sun Sep 22, 2019 17:28:14