Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


An Empirical Comparison of Abstraction in Models of Markov Decision Processes

Todd Hester and Peter Stone. An Empirical Comparison of Abstraction in Models of Markov Decision Processes. In Proceedings of the ICML/UAI/COLT Workshop on Abstraction in Reinforcement Learning, June 2009.
ICML ARL 2009

Download

[PDF]127.0kB  [postscript]363.0kB  

Abstract

Reinforcement learning studies the problem of solvingsequential decision making problems. Model-based methods learn an effective policy in few actionsby learning a model of the domain and simulating experiencein their models. Typical model-based methods must visiteach state at least once, which can be infeasiblein large domains. To overcome this problem, the model learning algorithm needs to generalizeknowledge to unseen states and provide information aboutthe states in which it needs more experience. In this paper,we use existing supervised learning techniques to learn the model of the domain. We empirically comparetheir effectiveness at generalizing knowledge acrossstates on three different domains. Our results indicatethat tree-based models perform the best after trainingon a small number of transitions, while support vectormachines perform the best after a large number of transitions.

BibTeX Entry

@InProceedings{ICMLARL09-hester,
  author="Todd Hester and Peter Stone",
  title="An Empirical Comparison of Abstraction in Models of Markov Decision Processes",
  booktitle = "Proceedings of the ICML/UAI/COLT Workshop on Abstraction in Reinforcement Learning",
  location = "Montreal, Canada",
  month = "June",
  year = "2009",
  abstract = "Reinforcement learning studies the problem of solving
sequential decision making problems. 
Model-based methods learn an effective policy in few actions
by learning a model of the domain and simulating experience
in their models. Typical model-based methods must visit
each state at least once, which can be infeasible
in large domains. To overcome this problem, 
the model learning algorithm needs to generalize
knowledge to unseen states and provide information about
the states in which it needs more experience. In this paper,
we use existing supervised learning techniques to 
learn the model of the domain. We empirically compare
their effectiveness at generalizing knowledge across
states on three different domains. Our results indicate
that tree-based models perform the best after training
on a small number of transitions, while support vector
machines perform the best after a large number of 
transitions.",
  wwwnote={<a href="http://www-all.cs.umass.edu/~gdk/arl/">ICML ARL 2009</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Fri Nov 21, 2014 15:59:49