Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


The EMPATHIC Framework for Task Learning from Implicit Human Feedback

Yuchen Cui, Qiping Zhang, Alessandro Allievi, Peter Stone, Scott Niekum, and W. Bradley Knox. The EMPATHIC Framework for Task Learning from Implicit Human Feedback. In Proceedings of the 4th Conference on Robot Learning (CoRL 2020), November 2020.
5-minute video presentation; 47-minute in-depth talk.
Project website.
Raw data from experiments.

Download

[PDF]6.4MB  [slides.pptx]59.4MB  

Abstract

Reactions such as gestures, facial expressions, and vocalizations are an abundant, naturally occurring channel of information that humans provide during interactions. A robot or other agent could leverage an understanding of such implicit human feedback to improve its task performance at no cost to the human. This approach contrasts with common agent teaching methods based on demonstrations, critiques, or other guidance that need to be attentively and intentionally provided. In this paper, we first define the general problem of learning from implicit human feedback and then propose to address this problem through a novel data-driven framework, EMPATHIC. This two-stage method consists of (1) mapping implicit human feedback to relevant task statistics such as reward, optimality, and advantage; and (2) using such a mapping to learn a task. We instantiate the first stage and three second-stage evaluations of the learned mapping. To do so, we collect a dataset of human facial reactions while participants observe an agent execute a sub-optimal policy for a prescribed training task. We train a deep neural network on this data and demonstrate its ability to (1) infer relative reward ranking of events in the training task from prerecorded human facial reactions; (2) improve the policy of an agent in the training task using live human facial reactions; and (3) transfer to a novel domain in which it evaluates robot manipulation trajectories.

BibTeX Entry

@InProceedings{CORL20-Cui,
  author = {Yuchen Cui and Qiping Zhang and Alessandro Allievi and Peter Stone and Scott Niekum and W. Bradley Knox},
  title = {The EMPATHIC Framework for Task Learning from Implicit Human Feedback},
  booktitle = {Proceedings of the 4th Conference on Robot Learning (CoRL 2020)},
  location = {Cambridge MA, USA},
  month = {November},
  year = {2020},
  abstract = {
  Reactions such as gestures, facial expressions, and vocalizations are an 
  abundant, naturally occurring channel of information that humans provide 
  during interactions. A robot or other agent could leverage an understanding 
  of such implicit human feedback to improve its task performance at no cost 
  to the human. This approach contrasts with common agent teaching methods 
  based on demonstrations, critiques, or other guidance that need to be 
  attentively and intentionally provided. In this paper, we first define the 
  general problem of learning from implicit human feedback and then propose to 
  address this problem through a novel data-driven framework, EMPATHIC. This 
  two-stage method consists of (1) mapping implicit human feedback to relevant 
  task statistics such as reward, optimality, and advantage; and (2) using such 
  a mapping to learn a task. We instantiate the first stage and three second-stage 
  evaluations of the learned mapping. To do so, we collect a dataset of human 
  facial reactions while participants observe an agent execute a sub-optimal 
  policy for a prescribed training task. We train a deep neural network on this 
  data and demonstrate its ability to (1) infer relative reward ranking of events 
  in the training task from prerecorded human facial reactions; (2) improve the 
  policy of an agent in the training task using live human facial reactions; and 
  (3) transfer to a novel domain in which it evaluates robot manipulation 
  trajectories.
  },
  wwwnote={<a href="https://www.youtube.com/watch?v=sTvNUpsf4P8">5-minute video presentation</a>; 
           <a href="https://www.youtube.com/watch?v=7FCttQyl9ag&feature=emb_logo">47-minute in-depth talk</a>.<br>
           <a href="https://sites.google.com/utexas.edu/empathic">Project website</a>.<br>
           <a href="https://zenodo.org/record/4290896#.X76zmcJMGV6">Raw data from experiments</a>.},
}

Generated by bib2html.pl (written by Patrick Riley ) on Fri Jun 18, 2021 14:00:31