Josiah Hanna

Picture of Josiah
jphanna 'at' cs 'dot' utexas 'dot' edu
Department of Computer Science
The University of Texas at Austin

I am a third year PhD student in the Learning Agents Research Group in the Computer Science Department at the University of Texas at Austin. My advisor is Peter Stone. I study machine learning and specifically a branch of machine learning called reinforcement learning (RL). The goal of RL is that autonomous agents can acquire useful skills from interaction with their environment. A main goal of my research is applying RL algorithms to robot learning. I am also interested in using machine learning to create smarter, more adaptive traffic systems for autonomous vehicles.

I graduated from the University of Kentucky with a B.S. in computer science and mathematics. As an undergraduate I worked with Judy Goldsmith on algorithms for multi-objective and factored Markov Decision Processes. I also had a research internship with Patrice Perny and Paul Weng at LIP6 in Paris, France working on approximating the set of Pareto optimal solutions for multi-objective MDPs.


Many RL algorithms are able to solve complex control tasks in simulated environments. However, the amount of experience required to find a good solution often prevents their application to physical robotic systems. Towards removing this barrier, I'm investigating methods for transferring behaviors learned in simulation to physical robots. A second goal of my research is to evaluate new robot behaviors before they are deployed on the physical robot.

I am also part of a project investigating adaptive tolling in traffic networks. Assuming that travelers desire to minimize travel time and tolls paid, adjusting tolls can improve the efficiency of the system as traffic patterns change over a day. While existing system might adapt tolls on a few links in a network, we consider potentially tolling every link to maximize performance gains.

I am a part of UT Austin Villa - UT's robot soccer team. I've contributed to both the standard platform (SPL) and 3D simulation league. In SPL I work on motion control (e.g. walking) and high level behavior (e.g. keeper behavior, kick strategy). For the 3D Sim team I've optimized variable distance kicks for passing.


Data-efficient Policy Evaluation through Behavior Policy Search.
Josiah Hanna, Phil Thomas, Peter Stone, and Scott Niekum.
In Proceedings of the 34th International Conference on Machine Learning (ICML 2017), August 2017.
Paper coming soon!

Bootstrapping with Models: Confidence Intervals for Off-Policy Evaluation.
Josiah Hanna, Peter Stone, and Scott Niekum.
In Proceedings of the 16th Conference on Autonomous Agents and Multi-Agent Sytems (AAMAS 2017), May 2017.
    BibTeX     Download: [pdf] (714.7kB )

Real-Time Adaptive Tolling Scheme for Optimized Social Welfare in Traffic Networks.
Guni Sharon, Josiah Hanna, Tarun Rambha, Michael Levin, Michael Albert, Peter Stone, and Steve Boyles.
In Proceedings of the 16th Conference on Autonomous Agents and Multi-Agent Sytems (AAMAS 2017), May 2017. Intelligence (AAMAS-17), February 2017.
    BibTeX     Download: [pdf] (714.7kB )

Grounded Action Transformation for Robot Learning in Simulation.
Josiah Hanna and Peter Stone.
In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI-17), February 2017.
    BibTeX     Download: [pdf] (714.7kB )

Minimum Cost Matching for Autonomous Carsharing.
Josiah Hanna, Michael Albert, Donna Chen, and Peter Stone
In Proceedings of the 9th IFAC Symposium on Intelligent and Autonomous Vehicles 2016 (IAV 2016), June 2016.

Operations of a shared, autonomous, electric vehicle fleet: Implications of vehicle & charging infrastructure decisions.
Donna Chen, Kara M Kockelman, Josiah Hanna.
Transportation Research Part A: Policy and Practice 94, 243-254

UT Austin Villa: RoboCup 2015 3D Simulation League Competition and Technical Challenges Champions.
Patrick MacAlpine, Josiah Hanna, Jason Liang, and Peter Stone.
In Luis Almeida, Jianmin Ji, Gerald Steinbauer, and Sean Luke, editors, RoboCup-2015: Robot Soccer World Cup XIX, Lecture Notes in Artificial Intelligence, Springer Verlag, Berlin, 2016.
Accompanying videos at
Details     BibTeX     Download: [pdf] (790.2kB )  [ps] (7.2MB )  

Approximation of lorenz-optimal solutions in multiobjective markov decision processes.
P Perny, P Weng, J Goldsmith, and Josiah Hanna
In Proceedings of the International Conference on Uncertainty in Artificial Intelligence (UAI), July 2013.

The academic advising planning domain.
JT Guerin, Josiah Hanna, L Ferland, N Mattei, J Goldsmith.
In Proceedings of the 3rd Workshop on the International Planning Competition at the International Conference on Automated Planning and Scheduling, July 2012.


I am from Lexington, Kentucky, USA. I moved from there to Austin in Fall of 2014 along with my wife, Aimee. We have two dogs: a Jack Russell terrier named Baxter and a chihuahua named Doby. I enjoy just about any type of athletic activity but particularly like running and basketball. Other interests include traveling, hiking, reading, and learning about languages.