Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Autonomous Task Sequencing for Customized Curriculum Design in Reinforcement Learning

Sanmit Narvekar, Jivko Sinapov, and Peter Stone. Autonomous Task Sequencing for Customized Curriculum Design in Reinforcement Learning. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI), August 2017.

Download

[PDF]826.2kB  [slides.pdf]5.8MB  

Abstract

Transfer learning is a method where an agent reuses knowledge learned in a source task to improve learning on a target task. Recent work has shown that transfer learning can be extended to the idea of curriculum learning, where the agent incrementally accumulates knowledge over a sequence of tasks (i.e. a curriculum). In most existing work, such curricula have been constructed manually. Furthermore, they are fixed ahead of time, and do not adapt to the progress or abilities of the agent. In this paper, we formulate the design of a curriculum as a Markov Decision Process, which directly models the accumulation of knowledge as an agent interacts with tasks, and propose a method that approximates an execution of an optimal policy in this MDP to produce an agent-specific curriculum. We use our approach to automatically sequence tasks for 3 agents with varying sensing and action capabilities in an experimental domain, and show that our method produces curricula customized for each agent that improve performance relative to learning from scratch or using a different agent's curriculum.

BibTeX Entry

@InProceedings{IJCAI17-Narvekar,
  author = {Sanmit Narvekar and Jivko Sinapov and Peter Stone},
  title = {Autonomous Task Sequencing for Customized Curriculum Design in Reinforcement Learning},
  booktitle = {Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI)},
  location = {Melbourne, Australia},
  month = {August},
  year = {2017},
  abstract = {
  Transfer learning is a method where an agent reuses knowledge learned in a
  source task to improve learning on a target task. Recent work has shown that
  transfer learning can be extended to the idea of curriculum learning,
  where the agent incrementally accumulates knowledge over a sequence of
  tasks (i.e. a curriculum). In most existing work, such curricula have been
  constructed manually. Furthermore, they are fixed ahead of time, and do not
  adapt to the progress or abilities of the agent. In this paper, we formulate
  the design of a curriculum as a Markov Decision Process, which directly
  models the accumulation of knowledge as an agent interacts with tasks, and
  propose a method that approximates an execution of an optimal policy in this
  MDP to produce an agent-specific curriculum. We use our approach to
  automatically sequence tasks for 3 agents with varying sensing and action
  capabilities in an experimental domain, and show that our method produces
  curricula customized for each agent that improve performance relative to
  learning from scratch or using a different agent's curriculum.  
  },
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Jul 06, 2020 17:00:34