Todd Hester's Research - Department of Computer Science

Research Interests

Todd I am a PhD student in the Department of Computer Science at the University of Texas at Austin studying artificial intelligence. My advisor is Peter Stone and I am a member of the Learning Agents Research Group at UT. I am also a part of the UT Austin Villa robot soccer team, which won the 2009 US Open in the Standard Platform League. I am researching reinforcement learning and robotics, specifically looking to make reinforcement learning apply to very large domains and applying it to make curious robots.

Before coming to UT, I worked in the Motion Analysis Laboratory at Spaulding Rehabiliation Hospital in Boston. There I worked on methods to evaluate the mobility of stroke, arthritis, and Parkinson's patients using wearable sensors. We used machine learning techniques to analyze wearable sensor data and quantify the quality of the patients' movements. I've also worked on a number of projects on my own, including building my own robot from scratch, writing a program to predict the scores of NFL football games based on machine learning techniques, and writing a 3D Connect Four game.

Robot Soccer

I have been a member of the UT Austin Villa robot soccer team since 2006 and participated in the Legged League and Standard Platform League at RoboCup. In 2009, we won the US Open Championship and came in 4th place in the internation competition. My research focus on robot soccer started with localization (See ICRA paper), but has moved on to encompass all parts of the robot soccer code, including, vision, locomotion, behaviors, coordination, and debug (See team paper).

Reinforcement Learning

Reinforcement Learning is a learning method where an agent can learn to act optimally by interacting with its environment. The agent is in some state and chooses from a set of available actions. Its action leads it to a new state and gives it some reward, which it tries to optimize over time. I am specifically focused on model-based reinforcement learning, where an agent learns a model of its environment and can learn a policy by simulating actions in its model. My research attempts to extend these model-based approaches to larger domains by incorporating generalization into the learning of the model (See AAMAS paper, ICRA paper, and video). In addition, I am examining the problem of exploration versus exploitation, looking at when the agent should exploit what it thinks it knows compared to when it should explore more of the environment.

Links:

Personal Home Page
Curriculum Vitae
University of Texas Dept of Computer Science

Contact Info:

Office: ENS32NE (Robotics Lab)
E-mail: todd AT cs DOT utexas DOT edu

Teaching

In the Fall 2009 semester, I was the TA for CS393R: Autonomous Robotics. I won the department's Outstanding TA Award.

In Spring 2009, I was a TA for CS307 Foundations of Computing..

Publications

Journal Articles

S. Patel, T. Hester, R. Hughes, N. Huggins, A. Flaherty, D. Standaert, J. Growdon, and P. Bonato. Processing Wearable Sensor Data to Optimize Deep-Brain Stimulation. In IEEE Pervasive Computing, Jan 2008.

Book Chapters

P. Stone, M. Quinlan, and T. Hester. The Essence of Soccer: Can Robots Play Too? In Soccer and Philosophy, Open Court Publishing, 2010.

Refereed Conferences

T. Hester, M. Quinlan, and P. Stone. Generalized Model Learning for Reinforcement Learning on a Humanoid Robot. In IEEE International Conference on Robotics and Automation (ICRA), May 2010. Video.

S. Kalyanakrishnan, T. Hester, M. Quinlan, Y. Bentor, and P. Stone. Three Humanoid Soccer Platforms: Comparison and Synthesis In Proceedings of the RoboCup International Symposium, July 2009.

T. Hester and P. Stone. Generalized Model Learning for Reinforcement Learning in Factored Domains In The Eighth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2009.

T. Hester and P. Stone. Negative Information and Line Observations for Monte Carlo Localization. In IEEE International Conference on Robotics and Automation (ICRA), May 2008.

N. Jong, T. Hester, and P. Stone. The Utility of Temporal Abstraction in Reinforcement Learning. In The Seventh International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS), May 2008.

P. Boissy, T. Hester, D. M. Sherrill, H. Corriveau, and P. Bonato. Monitoring Mobility Assistive Device Use in Post-Stroke Patients. In Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), August 2007.

S. Patel, T. Hester, R. Hughes, N. Huggins, D. Standaert, A. Flaherty, and P.Bonato. Using Wearable Sensors to Enhance DBS Parameter Adjustment for Parkinson's Disease Patients Through Measures of Motor Response. In Proceedings of the 3rd IEEE EMBS International Summer School and Symposium on Medical Devices and Biosensors, September 2006.

T. Hester, D. M. Sherrill, M. Hamel, K. Perreault, P. Boissy, and P. Bonato. Identification of Tasks Performed by Stroke Patients Using a Mobility Assistive Device. In Proceedings of the 28th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), Aug-Sept 2006.

P. Boissy, T. Hester, D. M. Sherrill, H. Corriveau, and P. Bonato. Monitoring Mobility Assistive Device Use in Patients After Stroke. In Proceedings of the 16th Congress of the International Society of Electrophysiology and Kinesiology (ISEK), June-July 2006.

T. Hester, R. Hughes, D. M. Sherrill, S. Patel, N. Huggins, A. Flaherty, D. Standaert, and P. Bonato. Adjusting DBS Settings to Optimize Parkinson’s Control Therapy. In Proceedings of the 16th Congress of the International Society of Electrophysiology and Kinesiology (ISEK), June-July 2006.

T. Hester, D. M. Sherrill, M. Hamel, K. Perreault, P. Boissy, and P. Bonato. Using Wearable Sensors to Analyze the Quality of Use of Mobility Assistive Devices. In Proceedings of the Third Annual International Workshop on Wearable and Implantable Body Sensor Networks (BSN), April 2006.

T. Hester, R. Hughes, D. M. Sherrill, B. Knorr, M. Akay, J. Stein, and P. Bonato. Using Wearable Sensors to Measure Motor Abilities following Stroke. In Proceedings of the Third Annual International Workshop on Wearable and Implantable Body Sensor Networks (BSN), April 2006.

S. Patel, D. Sherrill, R. Hughes, T. Hester, N. Huggins, T. Lie-Nemeth, D. Standaert, and P. Bonato. Analysis of the Severity of Dyskinesia in Patients with Parkinson’s Disease via Wearable Sensors. In Proceedings of the Third Annual International Workshop on Wearable and Implantable Body Sensor Networks (BSN), April 2006.

Refereed Workshop Papers

T. Hester and P. Stone. An Empirical Comparison of Abstraction in Models of Markov Decision Processes In Proceedings of the ICML/UAI/COLT Workshop on Abstraction in Reinforcement Learning, June 2009.

Technical Reports

T. Hester, M. Quinlan, P. Stone, and M. Sridharan. UT Austin Villa 2009: Naos Across Texas. Technical Report UT-AI-TR-09-08, The University of Texas at Austin, Department of Computer Science, AI Laboratory, 2009.

T. Hester, M. Quinlan, and P. Stone. UT Austin Villa 2008: Standing on Two Legs. Technical Report UT-AI-TR-08-8, The University of Texas at Austin, Department of Computer Science, AI Laboratory, 2008.

Videos

Learning to Score Penalty Kicks via Reinforcement Learning

The accompanying video for our ICRA 2010 paper, where we learn to score penalty kicks via a novel model-based reinforcement learning method.

2009 RoboCup Highlights

Highlights of TT-UT Austin Villa at the 2009 RoboCup Standard Platform League. TT-UT Austin Villa finished in 4th place, losing to only two teams during the tournament.

2009 US Open Highlights

Highlights of TT-UT Austin Villa at the 2009 US Open. TT-UT Austin Villa won the 2009 US Open with a finals win over UPenn (1-1 tie, 3-2 in penalty kicks).

Aibo Highlights

This video shows highlights (both shots and saves) from demonstrations held during Explore UT on March 7, 2009.