Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Generalized Model Learning for Reinforcement Learning in Factored Domains

Generalized Model Learning for Reinforcement Learning in Factored Domains.
Todd Hester and Peter Stone.
In The Eighth International Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2009.
AAMAS 2009

Download

[PDF]181.5kB  [postscript]425.5kB  

Abstract

Improving the sample efficiency of reinforcement learning algorithms to scale up to larger and more realistic domains is a current research challenge in machine learning. Model-based methods use experiential data more efficiently than model-free approaches but often require exhaustive exploration to learn an accurate model of the domain. We present an algorithm, Reinforcement Learning with Decision Trees (\textscrl-dt), that uses supervised learning techniques to learn the model by generalizing the relative effect of actions across states. Specifically, \textscrl-dt uses decision trees to model the relative effects of actions in the domain. The agent explores the environment exhaustively in early episodes when its model is inaccurate. Once it believes it has developed an accurate model, it exploits its model, taking the optimal action at each step. The combination of the learning approach with the targeted exploration policy enables fast learning of the model. The sample efficiency of the algorithm is evaluated empirically in comparison to five other algorithms across three domains. \textscrl-dt consistently accrues high cumulative rewards in comparison with the other algorithms tested.

BibTeX Entry

@InProceedings{AAMAS09-hester,
  author="Todd Hester and Peter Stone",
  title="Generalized Model Learning for Reinforcement Learning in Factored Domains",
  booktitle = "The Eighth International Conference on Autonomous Agents and Multiagent Systems (AAMAS)",
  location = "Budapest, Hungary",
  month = "May",
  year = "2009",
  abstract = "Improving the sample efficiency of reinforcement learning 
    algorithms to scale up to
    larger and more realistic domains is a current research challenge
    in machine learning.
    Model-based methods use experiential data more efficiently than model-free
    approaches but often require exhaustive exploration
    to learn an accurate model of the domain. 
    We present an algorithm, Reinforcement Learning with 
    Decision Trees (\textsc{rl-dt}), that
    uses supervised learning techniques to learn the model 
    by generalizing the relative effect of actions across states.
    Specifically, \textsc{rl-dt} uses decision trees to model the
    relative effects of actions in the domain.
    The agent explores the environment exhaustively in early episodes
    when its model is inaccurate.
    Once it believes it has developed an accurate model, it exploits
    its model, taking the optimal action at each step. 
    The combination of the learning
    approach with the targeted exploration policy enables fast learning
    of the model. 
    The sample efficiency of the algorithm is evaluated empirically 
    in comparison to five other algorithms across three domains.
    \textsc{rl-dt} consistently accrues high cumulative rewards
    in comparison with the other algorithms tested.",
  wwwnote={<a href="http://www.conferences.hu/AAMAS2009/">AAMAS 2009</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Mar 25, 2024 00:05:16