Research
My long-term research goal is to create complete, robust, autonomous agents that can learn to interact with other intelligent agents in a wide range of complex, dynamic environments.
Learning Agents Research Group
I am director of LARG, the Learning Agents Research Group in the UTCS AI Lab. All of the current research projects and many of the past projects listed on this page were done in this group. All of my current and past students are (or were) a part of this group.[ LARG Publications ]
Reinforcement Learning
A large part of the lab's research focus is on developing new reinforcement learning algorithms with a particular focus on scaling up to large-scale applications.Representative Publication:
Hierarchical Model-Based Reinforcement Learning: Rmax + MAXQ (ICML, 2008).
[ Full List of Relevant Publications ]
Multiagent Systems
One main theme of the lab is the study of interactions among independent autonomous agents (including robots), be they teammates, adversaries, or neither. Some of our research on this topic contributes to and makes use of game theory.Representative Publication:
Multiagent Systems: A survey from a machine learning perspective (Autonomous Robots, 2000).
[ Full List of Relevant Publications ]
Robot Soccer
One of the main application domains used throughout the lab is robot soccer, both in simulation and on real robots. We have won multiple RoboCup championships.Representative Publication:
Reinforcement Learning for RoboCup-Soccer Keepaway (Adaptive Behavior, 2005).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]
Ad Hoc Teamwork
We study algorithms that enable an individual agent to collaborate with previously unknown team members.Representative Publication:
Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination (AAAI, 2010).
[ Full List of Relevant Publications ]
Autonomous Traffic Management
We introduced a novel, efficient multiagent mechanism for future autonomous vehicles to navigate intersections.Representative Publication:
A Multiagent Approach to Autonomous Intersection Management (JAIR, 2008).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]
Autonomous Driving
We have a full-size autonomous vehicle that we use to study autonomous driving in the real world.Representative Publication:
Multiagent Interactions in Urban Driving (JoPhA, 2008).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]
Multiagent Learning (MAL)
We study learning algorithms that perform well against other learning agents.Representative Publication:
Convergence, Targeted Optimality and Safety in Multiagent Learning (ICML, 2010).
Teaching an Agent Manually via Evaluative Reinforcement (TAMER)
The TAMER project seeks to create agents which can be effectively taught behaviors by lay people using positive and negative feedback signals (akin to "shaping" by reward and punishment in animal training).Representative Publication:
Combining Manual Feedback with Subsequent MDP Reward Signals for Reinforcement Learning (AAMAS, 2010).
[ Project Page ]
[ Watch Video ]
Trading Agents
Another main application domain has been autonomous trading agents, including supply chain management, ad auctions, and mechanism design. We have won multiple Trading Agent Competitions.Representative Publication:
TacTex-2005: A Champion Supply Chain Management Agent (AAAI, 2006).
[ Full List of Relevant Publications ]
[ Project Page ]
Transfer Learning
We have developed algorithms from transfering knowledge from a previously learned task to a similar, but different, new learning task. We focus particularly on reinforcement learning tasks.Representative Publication:
Transfer Learning via Inter-Task Mappings for Temporal Difference Learning (JMLR, 2007).
[ Full List of Relevant Publications ]
Robot Vision
We developed algorithms suitable for real-time visual sensing of the physical world on mobile robots.Representative Publication:
Color Learning on a Mobile Robot: Towards Full Autonomy under Changing Illumination (IJCAI, 2007).
[ Full List of Relevant Publications ]
[ Project Page ]
Learned Quadruped Robot Walking
We enabled an Aibo robot to learn to walk faster than was previously possible.Representative Publication:
Machine Learning for Fast Quadrupedal Locomotion (AAAI, 2004).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]
General Game Playing
We participated successfully in the first few general game playing competitions.Representative Publication:
Automatic Heuristic Construction in a Complete General Game Player (AAAI, 2006).
[ Full List of Relevant Publications ]
Autonomic Computing
We developed machine learning approaches for computer systems applications.Representative Publication:
Machine Learning for On-Line Hardware Reconfiguration (IJCAI, 2007).
[ Full List of Relevant Publications ]
Developmental Robotics
We have developed methods for robots to autonomously discover models of their own sensor and actuators.Representative Publication:
Towards Autonomous Sensor and Actuator Model Induction on a Mobile Robot (Connection Science, 2006).
[ Full List of Relevant Publications ]
[ Project Page ]
[ Watch Video ]
Predictive State Representations
We have contributed to the literature on representating an agent's state entirely via predictions of its future sensations as a function of its possible actions. Thus it does not need to reason explicitly about objects.Representative Publication:
Learning Predictive State Representations (ICML, 2003).
[ Full List of Relevant Publications ]
Layered Learning
My Ph.D. thesis introduced a general hierarchical machine learning paradigm by which complex tasks can be learned via several interacting learned layers.Representative Publication:
Concurrent Layered Learning (AAMAS, 2003).
[ Full List of Relevant Publications ]
Planning
My first research as a Ph.D. student was within the area of classical AI planning. Some of our current research falls in the area of modern planning and schedulingRepresentative Publication:
FLECS: Planning with a Flexible Commitment Strategy (JAIR, 1995).
[ Full List of Relevant Publications ]