UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Learning Inter-Task Transferability in the Absence of Target Task Samples (2015)
Jivko Sinapov
,
Sanmit Narvekar
,
Matteo Leonetti
, and
Peter Stone
In a reinforcement learning setting, the goal of transfer learning is to improve performance on a target task by re-using knowledge from one or more source tasks. A key problem in transfer learning is how to choose appropriate source tasks for a given target task. Current approaches typically require that the agent has some experience in the target domain, or that the target task is specified by a model (e.g., a Markov Decision Process) with known parameters. To address these limitations, this paper proposes a framework for selecting source tasks in the absence of a known model or target task samples. Instead, our approach uses meta-data (e.g., attribute-value pairs) associated with each task to learn the expected benefit of transfer given a source-target task pair. To test the method, we conducted a large-scale experiment in the Ms. Pac-Man domain in which an agent played over 170 million games spanning 192 variations of the task. The agent used vast amounts of experience about transfer learning in the domain to model the benefit (or detriment) of transferring knowledge from one task to another. Subsequently, the agent successfully selected appropriate source tasks for previously unseen target tasks.
View:
PDF
,
HTML
Citation:
In
Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
, Istanbul, Turkey, May 2015.
Bibtex:
@inproceedings{AAMAS15-sinapov, title={Learning Inter-Task Transferability in the Absence of Target Task Samples}, author={Jivko Sinapov and Sanmit Narvekar and Matteo Leonetti and Peter Stone}, booktitle={Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS)}, month={May}, address={Istanbul, Turkey}, url="http://www.cs.utexas.edu/users/ai-lab?sinapov:aamas15", year={2015} }
People
Matteo Leonetti
Postdoctoral Alumni
matteo [at] cs utexas edu
Sanmit Narvekar
Ph.D. Student
sanmit [at] cs utexas edu
Jivko Sinapov
Postdoctoral Alumni
jsinapov [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Machine Learning
Transfer Learning
Labs
Learning Agents