Type of Talk: GRACS Speaker Series

Speaker/Affiliation:
Todd Hester/University of Texas at Austin

Date/Time: Tuesday, April
27, 2010, 2:00 p.m.

Location: TAY 3.128

Host: GRACS

Talk Title: Generalized Model Learning for Reinforcement Learning on a Huma

noid Robot

Talk Abstract:

This is a practice talk for my upcomi

ng presentation at ICRA 2010 of work
with Michael Quinlan and Peter St

one.

Reinforcement learning (RL) algorithms have long been promi

sing methods for
enabling an autonomous robot to improve its behavior

on sequential
decision-making tasks. The obvious enticement is that th

e robot should be
able to improve its own behavior without the need fo

r detailed step-by-step
programming. However, for RL to reach its ful

l potential, the algorithms
must be sample efficient: they must learn
competent behavior from very few
real-world trials. From this perspec

tive, model-based methods, which use
experiential data more efficien

tly than model-free approaches, are
appealing. But they often require
exhaustive exploration to learn an
accurate model of the domain. In t

his paper, we present an algorithm,
Reinforcement Learning with Deci

sion Trees (RL-DT), that uses decision trees
to learn the model by ge

neralizing the relative effect of actions across
states. The agent exp

lores the environment until it believes it has a
reasonable policy. Th

e combination of the learning approach with the
targeted exploration p

olicy enables fast learning of the model. We compare
RL-DT against sta

ndard model-free and model-based learning methods, and
demonstrate it

s effectiveness on an Aldebaran Nao humanoid robot scoring
goals in a

penalty kick scenario.

Speaker Bio:

 Todd Hester is a Ph.

D. student in computer science at The University of
Texas at Austin.&n

bsp; His research interests include reinforcement learning and
roboti

cs.  Todd received his BS in computer engineering from NortheasternUniversity in 2005 and his advisor is Peter Stone.