• Sorted by Date • Classified by Publication Type • Sorted by First Author Last Name • Classified by Research Category •
Mazda Ahmadi, Matthew E. Taylor,
and Peter Stone. IFSA: Incremental Feature-Set Augmentation for Reinforcement
Learning Tasks. In The Sixth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1120–1127,
May 2007.
Best Student Paper Nomination at AAMAS-2007.
[PDF]261.6kB [postscript]1.0MB
Reinforcement learning is a popular and successful framework for many agent-related problems because only limited environmental feedback is necessary for learning. While many algorithms exist to learn effective policies in such problems, learning is often used to solve real world problems, which typically have large state spaces, and therefore suffer from the ``curse of dimensionality.'' One effective method for speeding-up reinforcement learning algorithms is to leverage expert knowledge. In this paper, we propose a method for dynamically augmenting the agent's feature set in order to speed up value-function-based reinforcement learning. The domain expert divides the feature set into a series of subsets such that a novel problem concept can be learned from each successive subset. Domain knowledge is also used to order the feature subsets in order of their importance for learning. Our algorithm uses the ordered feature subsets to learn tasks significantly faster than if the entire feature set is used from the start. Incremental Feature-Set Augmentation (IFSA) is fully implemented and tested in three different domains: Gridworld, Blackjack and RoboCup Soccer Keepaway. All experiments show that IFSA can significantly speed up learning and motivates the applicability of this novel RL method.
@InProceedings{AAMAS07-ahmadi,
author="Mazda Ahmadi and Matthew E.\ Taylor and Peter Stone",
title="IFSA: Incremental Feature-Set Augmentation for Reinforcement Learning Tasks",
booktitle="The Sixth International Joint Conference on Autonomous Agents and Multiagent Systems",
pages="1120--1127",
month="May",year="2007",
abstract={
Reinforcement learning is a popular and successful framework for
many agent-related problems because only limited environmental
feedback is necessary for learning. While many algorithms exist to
learn effective policies in such problems, learning is often
used to solve real world problems, which typically have large state
spaces, and therefore suffer from the ``curse of dimensionality.''
One effective method for speeding-up reinforcement learning algorithms
is to leverage expert knowledge. In this paper, we propose a method
for dynamically augmenting the agent's feature set in order to
speed up value-function-based reinforcement learning. The domain
expert divides the feature set into a series of subsets such that a
novel problem concept can be learned from each successive
subset. Domain knowledge is also used to order the feature subsets in
order of their importance for learning. Our algorithm uses the
ordered feature subsets to learn tasks significantly faster than if
the entire feature set is used from the start. Incremental
Feature-Set Augmentation (IFSA) is fully implemented and tested in
three different domains: Gridworld, Blackjack and RoboCup Soccer
Keepaway. All experiments show that IFSA can significantly speed up
learning and motivates the applicability of this novel RL method.},
wwwnote={<span align="left" style="color: red; font-weight: bold">Best Student Paper Nomination</span> at <a href="http://www.aamas2007.nl/">AAMAS-2007</a>.},
}
Generated by bib2html.pl (written by Patrick Riley ) on Mon Nov 10, 2008 14:24:53