Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Real Time Targeted Exploration in Large Domains

Todd Hester and Peter Stone. Real Time Targeted Exploration in Large Domains. In The Ninth International Conference on Development and Learning (ICDL), August 2010.
ICDL 2010

Download

[PDF]253.9kB  [postscript]554.1kB  

Abstract

A developing agent needs to explore to learn about the world and learn good behaviors. In many real world tasks, this exploration can take far too long, and the agent must make decisions about which states to explore, and which states not to explore. Bayesian methods attempt to address this problem, but take too much computation time to run in reasonably sized domains. In this paper, we present TEXPLORE, the first algorithm to perform targeted exploration in real time in large domains. The algorithm learns multiple possible models of the domain that generalize action effects across states. We experiment with possible ways of adding intrinsic motivation to the agent to drive exploration. TEXPLORE is fully implemented and tested in a novel domain called Fuel World that is designed to reflect the type of targeted exploration needed in the real world. We show that our algorithm significantly outperforms representative examples of both model-free and model-based RL algorithms from the literature and is able to quickly learn to perform well in a large world in real-time.

BibTeX Entry

@InProceedings{ICDL10-hester,
	author="Todd Hester and Peter Stone",
	title="Real Time Targeted Exploration in Large Domains",
	booktitle = "The Ninth International Conference on Development and Learning (ICDL)",
	location = "Ann Arbor, Michigan",
	month = "August",
	year = "2010",
	abstract = "A developing agent needs to explore to learn about
		the world and learn good behaviors. In many real world tasks,
		this exploration can take far too long, and the agent must make
		decisions about which states to explore, and which states not
		to explore. Bayesian methods attempt to address this problem,
		but take too much computation time to run in reasonably sized
		domains. In this paper, we present TEXPLORE, the first algorithm
		to perform targeted exploration in real time in large domains.
		The algorithm learns multiple possible models of the domain
		that generalize action effects across states. We experiment with
		possible ways of adding intrinsic motivation to the agent to drive
		exploration. TEXPLORE is fully implemented and tested in a novel
		domain called Fuel World that is designed to reflect the type of
		targeted exploration needed in the real world. We show that
		our algorithm significantly outperforms representative examples
		of both model-free and model-based RL algorithms from the
		literature and is able to quickly learn to perform well in a large
		world in real-time.",
	wwwnote={<a href="http://www.eecs.umich.edu/icdl-2010/">ICDL 2010</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Wed Sep 24, 2014 22:15:11