UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Hierarchical Model-Based Reinforcement Learning: Rmax + MAXQ (2008)
Nicholas K. Jong
and
Peter Stone
Hierarchical decomposition promises to help scale reinforcement learning algorithms naturally to real-world problems by exploiting their underlying structure. Model-based algorithms, which provided the first finite-time convergence guarantees for reinforcement learning, may also play an important role in coping with the relative scarcity of data in large environments. In this paper, we introduce an algorithm that fully integrates modern hierarchical and model-learning methods in the standard reinforcement learning setting. Our algorithm, extscR-maxq, inherits the efficient model-based exploration of the extscR-max algorithm and the opportunities for abstraction provided by the MAXQ framework. We analyze the sample complexity of our algorithm, and our experiments in a standard simulation environment illustrate the advantages of combining hierarchies and models.
View:
PDF
,
PS
,
HTML
Citation:
In
Proceedings of the Twenty-Fifth International Conference on Machine Learning
, July 2008.
Bibtex:
@InProceedings{ICML08-jong, title={Hierarchical Model-Based Reinforcement Learning: Rmax + MAXQ}, author={Nicholas K. Jong and Peter Stone}, booktitle={Proceedings of the Twenty-Fifth International Conference on Machine Learning}, month={July}, url="http://www.cs.utexas.edu/users/ai-lab?ICML08-jong", year={2008} }
People
Nicholas Jong
Ph.D. Alumni
nickjong [at] me com
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Machine Learning
Planning
Labs
Learning Agents