by Peter Stone
MIT Press, 2000.
Available from MIT Press and amazon.com
This book is based upon my
(Computer Science Department, Carnegie Mellon University, 1998)
First, it describes an architecture within which a flexible team structure allows member agents to decompose a task into flexible roles and to switch roles while acting. Second, it presents layered learning, a general-purpose machine-learning method for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable with existing machine-learning methods. Third, the book introduces a new multiagent reinforcement learning algorithm--team-partitioned, opaque-transition reinforcement learning (TPOT-RL)--designed for domains in which agents cannot necessarily observe the state-changes caused by other agents' actions. The final contribution is a fully functioning multiagent system that incorporates learning in a real-time, noisy domain with teammates and adversaries--a computer-simulated robotic soccer team.
Peter Stone's work is the basis for the CMUnited Robotic Soccer Team, which has dominated recent RoboCup competitions. RoboCup not only helps roboticists to prove their theories in a realistic situation, but has drawn considerable public and professional attention to the field of intelligent robotics. The CMUnited team won the 1999 Stockholm simulator competition, outscoring its opponents by the rather impressive cumulative score of 110-0.
Chapter 1: Introduction 1.1 Motivation 1.2 Objectives and Approach 1.3 Contributions 1.4 Reader's Guide to the Book Chapter 2: Substrate Systems 2.1 Overview 2.2 The RoboCup Soccer Server 2.3 The CMUnited-97 Real Robots 2.4 Network Routing Chapter 3: Team Member Agent Architecture 3.1 Periodic Team Synchronization (PTS) Domains 3.2 Architecture Overview 3.3 Teamwork Structure 3.4 Communication Paradigm 3.5 Implementation in Robotic Soccer 3.6 Results 3.7 Transfer to Real Robots 3.8 Discussion and Related Work Chapter 4: Layered Learning 4.1 Principles 4.2 Formalism 4.3 Instantiation in Simulated Robotic Soccer 4.4 Discussion 4.5 Related Work Chapter 5: Learning an Individual Skill 5.1 Ball Interception in the Soccer Server 5.2 Training 5.3 Results 5.4 Discussion 5.5 Related Work Chapter 6: Learning a Multi-Agent Behavior 6.1 Decision Tree Learning for Pass Evaluation 6.2 Using the Learned Behaviors 6.3 Scaling up to Full Games 6.4 Discussion 6.5 Related Work Chapter 7: Learning a Team Behavior 7.1 Motivation 7.2 TPOT-RL 7.3 TPOT-RL Applied to Simulated Robotic Soccer 7.4 TPOT-RL Applied to Network Routing 7.5 Discussion 7.6 Related Work Chapter 8: Competition Results 8.1 Pre-RoboCup-96 8.2 MiroSot-96 8.3 RoboCup-97 8.4 RoboCup-98 8.4 RoboCup-99 8.6 Lessons Learned from Competitions Chapter 9: Related Work 9.1 MAS from an ML Perspective 9.2 Robotic Soccer Chapter 10: Conclusion 10.1 Contributions 10.2 Future Directions 10.3 Concluding Remarks Appendices A List of Acronyms B Robotic Soccer Agent Skills B.1 CMUnited-98 Simulator Agent Skills B.2 CMUnited-97 Small-Robot Skills C CMUnited-98 Simulator Team Behavior Modes C.1 Conditions C.2 Effects D CMUnited Simulator Team Source Code
Information, source, and executables pertaining to the CMUnited-99 simulator team are also available from the CMUnited-99 team page.