UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Monte Carlo Hierarchical Model Learning (2015)
Jacob Menashe
and
Peter Stone
Reinforcement learning (RL) is a well-established paradigm for enabling autonomous agents to learn from experience. To enable RL to scale to any but the smallest domains, it is necessary to make use of abstraction and generalization of the state-action space, for example with a factored representation. However, to make effective use of such a representation, it is necessary to determine which state variables are relevant in which situations. In this work, we introduce T-UCT, a novel model-based RL approach for learning and exploiting the dynamics of structured hierarchical environments. When learning the dynamics while acting, a partial or inaccurate model may do more harm than good. T-UCT uses graph-based planning and Monte Carlo simulations to exploit models that may be incomplete or inaccurate, allowing it to both maximize cumulative rewards and ignore trajectories that are unlikely to succeed. T-UCT incorporates new experiences in the form of more accurate plans that span a greater area of the state space. T-UCT is fully implemented and compared empirically against B-VISA, the best known prior approach to the same problem. We show that T-UCT learns hierarchical models with fewer samples than B-VISA and that this effect is magnified at deeper levels of hierarchical complexity.
View:
PDF
,
PS
,
HTML
Citation:
In
Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
, Istanbul, Turkey, May 2015.
Bibtex:
@inproceedings{AAMAS15-Menashe, title={Monte Carlo Hierarchical Model Learning}, author={Jacob Menashe and Peter Stone}, booktitle={Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)}, month={May}, address={Istanbul, Turkey}, url="http://www.cs.utexas.edu/users/ai-lab?menashe:aamas15", year={2015} }
People
Jacob Menashe
Ph.D. Student
jmenashe [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Reinforcement Learning
Labs
Learning Agents