UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Reinforcement Learning from Human Reward: Discounting in Episodic Tasks (2012)
W. Bradley Knox
and
Peter Stone
Several studies have demonstrated that teaching agents by human-generated reward can be a powerful technique. However, the algorithmic space for learning from human reward has hitherto not been explored systematically. Using model-based reinforcement learning from human reward in goal-based, episodic tasks, we investigate how anticipated future rewards should be discounted to create behavior that performs well on the task that the human trainer intends to teach. We identify a “positive circuits” problem with low discounting (i.e., high discount factors) that arises from an observed bias among humans towards giving positive reward. Empirical analyses indicate that high discounting (i.e., low discount factors) of human reward is necessary in goal-based, episodic tasks and lend credence to the existence of the positive circuits problem.
View:
PDF
,
HTML
Citation:
In
In Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man)
, September 2012.
Bibtex:
@inproceedings{ROMAN12-knox, title={Reinforcement Learning from Human Reward: Discounting in Episodic Tasks}, author={W. Bradley Knox and Peter Stone}, booktitle={In Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man)}, month={September}, url="http://www.cs.utexas.edu/users/ai-lab?ROMAN12-knox", year={2012} }
People
W. Bradley Knox
Ph.D. Alumni
bradknox [at] mit edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Projects
Teaching an Agent Manually via Evaluative Reinforcement (TAMER)
2008 - Present
Areas of Interest
Planning
Reinforcement Learning
Social Agents
Demos
Teaching an Agent Manually via Evaluative Reinforcement (TAMER)
W. Bradley Knox and Peter Stone
2009
Labs
Learning Agents