UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Autonomous Task Sequencing for Customized Curriculum Design in Reinforcement Learning (2017)
Sanmit Narvekar
,
Jivko Sinapov
, and
Peter Stone
Transfer learning is a method where an agent reuses knowledge learned in a source task to improve learning on a target task. Recent work has shown that transfer learning can be extended to the idea of curriculum learning, where the agent incrementally accumulates knowledge over a sequence of tasks (i.e. a curriculum). In most existing work, such curricula have been constructed manually. Furthermore, they are fixed ahead of time, and do not adapt to the progress or abilities of the agent. In this paper, we formulate the design of a curriculum as a Markov Decision Process, which directly models the accumulation of knowledge as an agent interacts with tasks, and propose a method that approximates an execution of an optimal policy in this MDP to produce an agent-specific curriculum. We use our approach to automatically sequence tasks for 3 agents with varying sensing and action capabilities in an experimental domain, and show that our method produces curricula customized for each agent that improve performance relative to learning from scratch or using a different agent's curriculum.
View:
PDF
,
HTML
Citation:
In
Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI)
, Melbourne, Australia, August 2017.
Bibtex:
@inproceedings{IJCAI17-Narvekar, title={Autonomous Task Sequencing for Customized Curriculum Design in Reinforcement Learning}, author={Sanmit Narvekar and Jivko Sinapov and Peter Stone}, booktitle={Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI)}, month={August}, address={Melbourne, Australia}, url="http://www.cs.utexas.edu/users/ai-lab?narvekar:ijcai17", year={2017} }
Presentation:
Slides (PDF)
People
Sanmit Narvekar
Ph.D. Student
sanmit [at] cs utexas edu
Jivko Sinapov
Postdoctoral Alumni
jsinapov [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Reinforcement Learning
Transfer Learning
Labs
Learning Agents