UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
A Real-Time Model-Based Reinforcement Learning Architecture for Robot Control (2011)
Todd Hester
,
Michael Quinlan
, and
Peter Stone
Reinforcement Learning (RL) is a method for learning decision-making tasks that could enable robots to learn and adapt to their situation on-line. For an RL algorithm to be practical for robotic control tasks, it must learn in very few actions, while continually taking those actions in real-time. Existing model-based RL methods learn in relatively few actions, but typically take too much time between each action for practical on-line learning. In this paper, we present a novel parallel architecture for model-based RL that runs in real-time by 1) taking advantage of sample-based approximate planning methods and 2) parallelizing the acting, model learning, and planning processes such that the acting process is sufficiently fast for typical robot control cycles. We demonstrate that algorithms using this architecture perform nearly as well as methods using the typical sequential architecture when both are given unlimited time, and greatly out-perform these methods on tasks that require real-time actions such as controlling an autonomous vehicle.
View:
PDF
,
PS
,
HTML
Bibtex:
@techreport{ARXIV11-hester, title={A Real-Time Model-Based Reinforcement Learning Architecture for Robot Control}, author={Todd Hester and Michael Quinlan and Peter Stone}, month={May}, url="http://www.cs.utexas.edu/users/ai-lab?ARXIV11-hester", year={2011} }
People
Todd Hester
Postdoctoral Alumni
todd [at] cs utexas edu
Michael Quinlan
Formerly affiliated Research Scientist
mquinlan [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Projects
TEXPLORE: Real-Time Sample Efficient Reinforcement Learning
2009 - Present
Areas of Interest
Machine Learning
Reinforcement Learning
Demos
TEXPLORE: Real-Time Sample Efficient Reinforcement Learning
Todd Hester
2012
Labs
Learning Agents