This web page provides supplementary material to the following paper:
In several realistic domains an agent's behavior
is composed of multiple interdependent skills. For example, consider a
humanoid robot that must play soccer, as is the focus of this
paper. In order to succeed, it is clear that the robot needs to walk
quickly, turn sharply, and kick the ball far. However, these
individual skills are ineffective if the robot falls down when
switching from walking to turning, or if it cannot position itself
behind the ball for a kick.
This paper presents a learning architecture for a humanoid robot
soccer agent that has been fully deployed and tested within the
RoboCup 3D simulation environment. First, we demonstrate that
individual skills such as walking and turning can be parameterized and
optimized to match the best performance statistics reported in the
literature. These results are achieved through effective use of the
CMA-ES optimization algorithm. Next, we describe a framework for
optimizing skills in conjunction with one another, a
little-understood problem with substantial practical
significance. Over several phases of learning, a total of roughly 100--150
parameters are optimized. Detailed experiments show that an agent thus
optimized performs comparably with the top teams from the RoboCup 2010
competitions, while taking relatively few man-hours for development.