UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study (2007)
Shivaram Kalyanakrishnan
,
Yaxin Liu
, and
Peter Stone
We present half field offense, a novel subtask of RoboCup simulated soccer, and pose it as a problem for reinforcement learning. In this task, an offense team attempts to outplay a defense team in order to shoot goals. Half field offense extends keepawayciteStone+SK:2005, a simpler subtask of RoboCup soccer in which one team must try to keep possession of the ball within a small rectangular region, and away from the opposing team. Both keepaway and half field offense have to cope with the usual problems of RoboCup soccer, such as a continuous state space, noisy actions, and multiple agents, but the latter is a significantly harder multiagent reinforcement learning problem because of sparse rewards, a larger state space, a richer action set, and the sheer complexity of the policy to be learned. We demonstrate that the algorithm that has been successful for keepaway is inadequate to scale to the more complex half field offense task, and present a new algorithm to address the aforementioned problems in multiagent reinforcement learning. The main feature of our algorithm is the use of inter-agent communication, which allows for more frequent and reliable learning updates. We show empirical results verifying that our algorithm registers significantly higher performance and faster learning than the earlier approach. We also assess the contribution of inter-agent communication by considering several variations of the basic learning method. This work is a step further in the ongoing challenge to learn complete team behavior for the RoboCup simulated soccer task.
View:
PDF
,
PS
,
HTML
Citation:
In
RoboCup-2006: Robot Soccer World Cup X
, Gerhard Lakemeyer and Elizabeth Sklar and Domenico Sorenti and Tomoichi Takahashi (Eds.), Vol. 4434, pp. 72-85, Berlin 2007. Springer Verlag.
Bibtex:
@incollection{LNAI2006-shivaram, title={Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study}, author={Shivaram Kalyanakrishnan and Yaxin Liu and Peter Stone}, booktitle={RoboCup-2006: Robot Soccer World Cup X}, volume={4434}, editor={Gerhard Lakemeyer and Elizabeth Sklar and Domenico Sorenti and Tomoichi Takahashi}, series={Lecture Notes in Artificial Intelligence}, address={Berlin}, publisher={Springer Verlag}, pages={72-85}, url="http://www.cs.utexas.edu/users/ai-lab?LNAI2006-shivaram", year={2007} }
People
Shivaram Kalyanakrishnan
Ph.D. Alumni
shivaram [at] cs utexas edu
Yaxin Liu
Postdoctoral Alumni
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Other Areas
Reinforcement Learning
Simulated Robot Soccer
Demos
Simulated RoboCup Soccer
2004
Labs
Learning Agents