Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study

Shivaram Kalyanakrishnan, Yaxin Liu, and Peter Stone. Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study. In Gerhard Lakemeyer, Elizabeth Sklar, Domenico Sorenti, and Tomoichi Takahashi, editors, RoboCup-2006: Robot Soccer World Cup X, Lecture Notes in Artificial Intelligence, pp. 72–85, Springer Verlag, Berlin, 2007.
BEST STUDENT PAPER AWARD WINNER at RoboCup International Symposium.
Some simulations referenced in the paper.

Download

[PDF]992.2kB  [postscript]1.6MB  

Abstract

We present half field offense, a novel subtask of RoboCup simulated soccer, and pose it as a problem for reinforcement learning. In this task, an offense team attempts to outplay a defense team in order to shoot goals. Half field offense extends keepaway\citeStone+SK:2005, a simpler subtask of RoboCup soccer in which one team must try to keep possession of the ball within a small rectangular region, and away from the opposing team. Both keepaway and half field offense have to cope with the usual problems of RoboCup soccer, such as a continuous state space, noisy actions, and multiple agents, but the latter is a significantly harder multiagent reinforcement learning problem because of sparse rewards, a larger state space, a richer action set, and the sheer complexity of the policy to be learned. We demonstrate that the algorithm that has been successful for keepaway is inadequate to scale to the more complex half field offense task, and present a new algorithm to address the aforementioned problems in multiagent reinforcement learning. The main feature of our algorithm is the use of inter-agent communication, which allows for more frequent and reliable learning updates. We show empirical results verifying that our algorithm registers significantly higher performance and faster learning than the earlier approach. We also assess the contribution of inter-agent communication by considering several variations of the basic learning method. This work is a step further in the ongoing challenge to learn complete team behavior for the RoboCup simulated soccer task.

BibTeX Entry

@incollection(LNAI2006-shivaram,
        author="Shivaram Kalyanakrishnan and Yaxin Liu and Peter Stone",
        title="Half Field Offense in {R}obo{C}up Soccer: A Multiagent Reinforcement Learning Case Study",
        booktitle= "{R}obo{C}up-2006: {R}obot {S}occer {W}orld {C}up {X}",
        Editor="Gerhard Lakemeyer and Elizabeth Sklar and Domenico Sorenti and Tomoichi Takahashi",
        Publisher="Springer Verlag",address="Berlin",year="2007",
        issn="0302-9743",
        isbn="978-3-540-74023-0",
        series="Lecture Notes in Artificial Intelligence",      
	volume="4434",
        pages="72--85",
        abstract={
                  We present half field offense, a novel subtask of
                  RoboCup simulated soccer, and pose it as a problem
                  for reinforcement learning.  In this task, an
                  offense team attempts to outplay a defense team in
                  order to shoot goals.  Half field offense extends
                  keepaway\cite{Stone+SK:2005}, a simpler subtask
                  of RoboCup soccer in which one team must try to keep
                  possession of the ball within a small rectangular
                  region, and away from the opposing team.  Both
                  keepaway and half field offense have to cope with
                  the usual problems of RoboCup soccer, such as a
                  continuous state space, noisy actions, and multiple
                  agents, but the latter is a significantly harder
                  multiagent reinforcement learning problem because of
                  sparse rewards, a larger state space, a richer
                  action set, and the sheer complexity of the policy
                  to be learned.  We demonstrate that the algorithm
                  that has been successful for keepaway is inadequate
                  to scale to the more complex half field offense
                  task, and present a new algorithm to address the
                  aforementioned problems in multiagent reinforcement
                  learning.  The main feature of our algorithm is the
                  use of inter-agent communication, which allows for
                  more frequent and reliable learning updates.  We
                  show empirical results verifying that our algorithm
                  registers significantly higher performance and
                  faster learning than the earlier approach.  We also
                  assess the contribution of inter-agent communication
                  by considering several variations of the basic
                  learning method.  This work is a step further in the
                  ongoing challenge to learn complete team behavior
                  for the RoboCup simulated soccer task.
        },
        wwwnote={<b>BEST STUDENT PAPER AWARD WINNER</b> at RoboCup International Symposium.<br> Some <a href="http://www.cs.utexas.edu/~AustinVilla/sim/halffieldoffense/">simulations</a> referenced in the paper.},
)

Generated by bib2html.pl (written by Patrick Riley ) on Fri Aug 15, 2014 16:26:03