**Actual stories demonstrating how UTCS talent is making this a better world**

Is it Professor Stone or Coach Stone? Peter Stone, Assistant Professor at the University of Texas at Austin, Department of Computer Sciences (UTCS), has been involved with soccer most of his life. He played on his college soccer team for four years, played semi-professional soccer, and in graduate school at Carnegie Mellon, co-founded the school's four-time world champion robot soccer team. Today Stone plays the real game with Austin Villa of the Austin premier amateur league, and serves as both coach and teacher to a cadre of enthusiastic undergraduate and graduate students. The students compete in both simulated robot soccer and real robot soccer played with four-legged Sony AIBO robots, which look a lot like puppies. When Stone's undergraduates presented their semester project, a digital simulation team, he sounded more like a coach than a computer science professor as he explained to the audience the games' strategies. Students competing in both simulated and real robot soccer use the programming language C++ to train their teams to play, but they do not control the individual players. During the games, the players work with no human interaction. The players are programmed to cooperate with one another as a team, though each is still able to make its own individual decisions. Stone and his students write computer programs that use strategies that are, in part, motivated by those used by real players when they play. That is why all those years on the playing field may help make Stone's robot teams so competitive. The players are also known as agents in the field of Artificial Intelligence (AI). Agents (robot soccer players in this instance) are programmed to sense their environments, make decisions about actions to execute, and then actually execute them in their environments. A main research focus in Stone's lab is enabling the agents to change their behavior in reaction to stimuli, so they learn to improve their performance from past experience. How the Robot Soccer Players Perceive and Learn Machine learning is a large subfield of AI that focuses on enabling computers to improve their performance on some task based on experience. One form of machine learning Stone uses in the robot soccer domain is known as reinforcement learning , in which the learner learns by trying different behaviors through trial and error. The learner (robot soccer player) is not told which actions to take, but must figure out which actions lead to "desirable" outcomes (as indicated by a numerical reward signal), and continue to use them. The robot soccer players learn several different skills including recognizing the ball in its camera image, walking as fast as possible, and manipulating the ball once it has reached it. Learning is particularly necessary because the robots have to learn to walk and see slightly differently when they travel to competitions and play on different surfaces under different lights. They cannot adjust to a new playing field without modification to their gaits and their vision settings. Robot soccer players must know where the other players are on the playing field. Effective behaviors will be different depending on the robot's proximity to the ball, teammates, and opponents. Interaction among independent agents (other autonomous robots) is the study of multiagent systems. Since the robots do not play the game alone, part of the challenge is working as a team and learning to deal with their opponents. Stone's research within the " keepaway" scenario explores machine learning in multiagent systems.	Austin Villa Robot Soccer 2005 team members, left to right: Mohan Sridharan, Nate Kohl, Peter Stone, Greg Kuhlmann and Peggy Fidelman. Not pictured: Kurt Dresner, Selim Erdogan, Nick Jong and Dan Stronger.
Keepaway Soccer Remember playing Keepaway as a child? It was the game where you had a ball (or something you stole from your sister) that one team passed among themselves, and tried to keep the other team from stealing. That is also the idea behind Keepaway (Robot) Soccer. The problem for machine learning researchers with little or no RoboCup experience was that they found the domain inaccessible. Researchers had to program their own agents that could produce low-level behaviors such as passing a ball. This proved time-consuming and difficult. Stone, graduate students Greg Kukhlann, Matthew Taylor, and postdoc Yaxin Liu have developed a set of programs, tools, tutorials and resources that make the domain easier to use for other researchers doing experimentation. One of the goals of this research is to provide benchmarks for comparison of the relative performance of different learning methods. For more information, read their paper (pdf file) “Keepaway Soccer: From Machine Learning Testbed to Benchmark.” Above. Students, Ani Papova and Joseph Knaack, presenting their simulated soccer project in Stone's class, spring 2005. Sensory Perception in Robots In competition, all teams use the exact same type of robots so the only difference is in the way a team is programmed. Unlike auto racing or downhill skiing, there are no advantages due to better equipment.	Above. Close-up of Sony Aibo robot's head with embedded video camera. The dogs have sensors for sound and touch, but rely mainly on a color video camera located in their heads for vision. They can move about 20 joints in their bodies. They are not equipped with a GPS (global positioning system), but Stone said it wouldn't be accurate enough anyway. Everything the dog can learn about its environment comes from the transmission of data from the video camera and sensors. One of Professor Stone's graduate students, Mohan Sridharan, is currently working with him on the challenges of perfecting vision and understanding of the environment by the robots. Sridharan and Stone are trying to achieve a system where the robots can see and act as humans do. The problem of color and object recognition as well as the ability to track a moving object by an agent with limited memory resources is extremely difficult. They want the robots to autonomously acquire knowledge of their environment, and then be able to remember it and quickly adapt to variations in the environment, such as lighting and variable arena surfaces. It takes a lot of extra time when they go to competition for the robots to relearn how to walk and analyze colors on different surfaces and in slightly different lighting. They need to eliminate or shorten that part of the process. For detailed information on this work, see their research paper, (pdf file) “Toward Eliminating Manual Color Calibration at RoboCup.” Resolution of these issues will prove critical when search and rescue robots replace humans in real life situations in the future. In real world situations, there won't be time for robots to constantly relearn their environments. Programming Robots Stone and his students program their robot soccer players in C++. Programming the robots is particularly challenging due to several factors. First, it takes a good deal of time to boot up the robots and test out any change to the code. Second, it can be very difficult to debug the code without the sophisticated debugging software con't next page-->
For more information contact Dr. Peter Stone, pstone@cs.utexas.edu \| Peter Stone's web site \| UT Austin Villa Robot Soccer \| Simulated Soccer League, UT Austin Villa Team \| UTCS External Affairs site \| Main Computer Sciences site \| ©The University of Texas at Austin, Department of Computer Sciences, article by Carol Grosvenor, November, 2005

Actual stories demonstrating how UTCS talent is making this a better world

Is it Professor Stone or Coach Stone?

Peter Stone, Assistant Professor at the University of Texas at Austin, Department of Computer Sciences (UTCS), has been involved with soccer most of his life. He played on his college soccer team for four years, played semi-professional soccer, and in graduate school at Carnegie Mellon, co-founded the school's four-time world champion robot soccer team. Today Stone plays the real game with Austin Villa of the Austin premier amateur league, and serves as both coach and teacher to a cadre of enthusiastic undergraduate and graduate students. The students compete in both simulated robot soccer and real robot soccer played with four-legged Sony AIBO robots, which look a lot like puppies.

When Stone's undergraduates presented their semester project, a digital simulation team, he sounded more like a coach than a computer science professor as he explained to the audience the games' strategies. Students competing in both simulated and real robot soccer use the programming language C++ to train their teams to play, but they do not control the individual players. During the games, the players work with no human interaction. The players are programmed to cooperate with one another as a team, though each is still able to make its own individual decisions.

Stone and his students write computer programs that use strategies that are, in part, motivated by those used by real players when they play. That is why all those years on the playing field may help make Stone's robot teams so competitive. The players are also known as agents in the field of Artificial Intelligence (AI). Agents (robot soccer players in this instance) are programmed to sense their environments, make decisions about actions to execute, and then actually execute them in their environments. A main research focus in Stone's lab is enabling the agents to change their behavior in reaction to stimuli, so they learn to improve their performance from past experience.

How the Robot Soccer Players Perceive and Learn

Machine learning is a large subfield of AI that focuses on enabling computers to improve their performance on some task based on experience. One form of machine learning Stone uses in the robot soccer domain is known as reinforcement learning , in which the learner learns by trying different behaviors through trial and error. The learner (robot soccer player) is not told which actions to take, but must figure out which actions lead to "desirable" outcomes (as indicated by a numerical reward signal), and continue to use them.

The robot soccer players learn several different skills including recognizing the ball in its camera image, walking as fast as possible, and manipulating the ball once it has reached it. Learning is particularly necessary because the robots have to learn to walk and see slightly differently when they travel to competitions and play on different surfaces under different lights. They cannot adjust to a new playing field without modification to their gaits and their vision settings.

Robot soccer players must know where the other players are on the playing field. Effective behaviors will be different depending on the robot's proximity to the ball, teammates, and opponents.

Interaction among independent agents (other autonomous robots) is the study of multiagent systems. Since the robots do not play the game alone, part of the challenge is working as a team and learning to deal with their opponents. Stone's research within the " keepaway" scenario explores machine learning in multiagent systems.

photo of Austin Villa Robot
Soccer Team, 2005

Austin Villa Robot Soccer 2005 team members, left to right: Mohan Sridharan, Nate Kohl, Peter Stone, Greg Kuhlmann and Peggy Fidelman. Not pictured: Kurt Dresner, Selim Erdogan, Nick Jong and Dan Stronger.

Keepaway Soccer

Remember playing Keepaway as a child? It was the game where you had a ball (or something you stole from your sister) that one team passed among themselves, and tried to keep the other team from stealing. That is also the idea behind Keepaway (Robot) Soccer. The problem for machine learning researchers with little or no RoboCup experience was that they found the domain inaccessible. Researchers had to program their own agents that could produce low-level behaviors such as passing a ball. This proved time-consuming and difficult. Stone, graduate students Greg Kukhlann, Matthew Taylor, and postdoc Yaxin Liu have developed a set of programs, tools, tutorials and resources that make the domain easier to use for other researchers doing experimentation.

One of the goals of this research is to provide benchmarks for comparison of the relative performance of different learning methods. For more information, read their paper (pdf file) “Keepaway Soccer: From Machine Learning Testbed to Benchmark.”

simulated soccer screen

Above. Students, Ani Papova and Joseph Knaack, presenting their simulated soccer project in Stone's class, spring 2005.

Sensory Perception in Robots

In competition, all teams use the exact same type of robots so the only difference is in the way a team is programmed. Unlike auto racing or downhill skiing, there are no advantages due to better equipment.

closeup of dog's head

Above. Close-up of Sony Aibo robot's head with embedded video camera.

The dogs have sensors for sound and touch, but rely mainly on a color video camera located in their heads for vision. They can move about 20 joints in their bodies. They are not equipped with a GPS (global positioning system), but Stone said it wouldn't be accurate enough anyway. Everything the dog can learn about its environment comes from the transmission of data from the video camera and sensors.

One of Professor Stone's graduate students, Mohan Sridharan, is currently working with him on the challenges of perfecting vision and understanding of the environment by the robots. Sridharan and Stone are trying to achieve a system where the robots can see and act as humans do. The problem of color and object recognition as well as the ability to track a moving object by an agent with limited memory resources is extremely difficult. They want the robots to autonomously acquire knowledge of their environment, and then be able to remember it and quickly adapt to variations in the environment, such as lighting and variable arena surfaces. It takes a lot of extra time when they go to competition for the robots to relearn how to walk and analyze colors on different surfaces and in slightly different lighting. They need to eliminate or shorten that part of the process. For detailed information on this work, see their research paper, (pdf file) “Toward Eliminating Manual Color Calibration at RoboCup.”

Resolution of these issues will prove critical when search and rescue robots replace humans in real life situations in the future. In real world situations, there won't be time for robots to constantly relearn their environments.

Programming Robots

Stone and his students program their robot soccer players in C++. Programming the robots is particularly challenging due to several factors. First, it takes a good deal of time to boot up the robots and test out any change to the code. Second, it can be very difficult to debug the code without the sophisticated debugging software con't next page-->

For more information contact Dr. Peter Stone, pstone@cs.utexas.edu | Peter Stone's web site | UT Austin Villa Robot Soccer | Simulated Soccer League, UT Austin Villa Team | UTCS External Affairs site | Main Computer Sciences site |
©The University of Texas at Austin, Department of Computer Sciences, article by Carol Grosvenor, November, 2005