Josiah Hanna

Picture of Josiah
jphanna 'at' cs 'dot' utexas 'dot' edu
Department of Computer Science
The University of Texas at Austin

I am a third year PhD student in the Learning Agents Research Group in the Computer Science Department at the University of Texas at Austin. My advisor is Peter Stone. I study machine learning and specifically a branch of machine learning called reinforcement learning (RL). The goal of RL is that autonomous agents can acquire useful skills from interaction with their environment. A main goal of my research is applying RL algorithms to robot learning. I am also interested in using machine learning to create smarter, more adaptive traffic systems for autonomous vehicles.

I graduated from the University of Kentucky with a B.S. in computer science and mathematics. As an undergraduate I worked with Judy Goldsmith on algorithms for multi-objective and factored Markov Decision Processes. I also had a research internship with Patrice Perny and Paul Weng at LIP6 in Paris, France working on approximating the set of Pareto optimal solutions for multi-objective MDPs.


Many RL algorithms are able to solve complex control tasks in simulated environments. However, the amount of experience required to find a good solution often prevents their application to physical robotic systems. Towards removing this barrier, I'm investigating methods for transferring behaviors learned in simulation to physical robots. A second goal of my research is to evaluate new robot behaviors before they are deployed on the physical robot.

I am also part of a project investigating adaptive tolling in traffic networks. Assuming that travelers desire to minimize travel time and tolls paid, adjusting tolls can improve the efficiency of the system as traffic patterns change over a day. While existing system might adapt tolls on a few links in a network, we consider potentially tolling every link to maximize performance gains.

I am a part of UT Austin Villa - UT's robot soccer team. I've contributed to both the standard platform (SPL) and 3D simulation league. In SPL I work on motion control (e.g. walking) and high level behavior (e.g. keeper behavior, kick strategy). For the 3D Sim team I've optimized variable distance kicks for passing.


Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation.
Josiah Hanna, Peter Stone, and Scott Niekum.
In Proceedings of the 16th Conference on Autonomous Agents and Multi-Agent Sytems (AAMAS 2017), May 2017.
    BibTeX     Download: [pdf] (714.7kB )

Real-Time Adaptive Tolling Scheme for Optimized Social Welfare in Traffic Networks.
Guni Sharon, Josiah Hanna, Tarun Rambha, Michael Levin, Michael Albert, Peter Stone, and Steve Boyles.
In Proceedings of the 16th Conference on Autonomous Agents and Multi-Agent Sytems (AAMAS 2017), May 2017. Intelligence (AAMAS-17), February 2017.
    BibTeX     Download: [pdf] (714.7kB )

Grounded Action Transformation for Robot Learning in Simulation.
Josiah Hanna and Peter Stone.
In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17), February 2017.
    BibTeX     Download: [pdf] (714.7kB )


I am from Lexington, Kentucky, USA. I moved from there to Austin in Fall of 2014 along with my wife, Aimee. We have two dogs: a Jack Russell terrier named Baxter and a chihuahua named Doby. I enjoy just about any type of athletic activity but particularly like running and basketball. Other interests include traveling, hiking, reading, and learning about languages.