UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration (2019)
Brahma S. Pavse,
Faraz Torabi
,
Josiah Hanna
,
Garrett Warnell
, and
Peter Stone
Imitation learning has long been an approach to alleviate the tractability issues that arise in reinforcement learning. However, most literature makes several assumptions such as access to the expert's actions, availability of many expert demonstrations, and injection of task-specific domain knowledge into the learning process. We propose reinforced inverse dynamics modeling (RIDM), a method of combining reinforcement learning and imitation from observation (IfO) to perform imitation using a single expert demonstration, with no access to the expert's actions, and with little task-specific domain knowledge. Given only a single set of the expert's raw states, such as joint angles in a robot control task, at each time-step, we learn an inverse dynamics model to produce the necessary low-level actions, such as torques, to transition from one state to the next such that the reward from the environment is maximized. We demonstrate that RIDM outperforms other techniques when we apply the same constraints on the other methods on six domains of the MuJoCo simulator and for two different robot soccer tasks for two experts from the RoboCup 3D simulation league on the SimSpark simulator.
View:
PDF
,
HTML
Bibtex:
@inproceedings{ICML19-pavse, title={RIDM: Reinforced Inverse Dynamics Modeling for Learning from a Single Observed Demonstration}, author={Brahma S. Pavse and Faraz Torabi and Josiah Hanna and Garrett Warnell and Peter Stone}, month={June}, address={Long Beach, California, USA}, url="http://www.cs.utexas.edu/users/ai-lab?ICML19-pavse", year={2019} }
People
Josiah Hanna
Ph.D. Student
jphanna [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Faraz Torabi
Ph.D. Student
faraztrb [at] cs utexas edu
Garrett Warnell
Research Scientist
warnellg [at] cs utexas edu
Areas of Interest
Imitation Learning
Machine Learning
Labs
Learning Agents