Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog

Jesse Thomason, Aishwarya Padmakumar, Jivko Sinapov, Nick Walker, Yuqian Jiang, Harel Yedidsion, Justin Hart, Peter Stone, and Raymond J. Mooney. Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog. The Journal of Artificial Intelligence Research (JAIR), 67, February 2020.

Download

[PDF]4.0MB  

Abstract

In this work, we present methods for using human-robot dialog to improve language understanding for a mobile robot agent. The agent parses natural language to underlying semantic meanings and uses robotic sensors to create multi-modal models of perceptual concepts like red and heavy. The agent can be used for showing navigation routes, delivering objects to people, and relocating objects from one location to another. We use dialog clarification questions both to understand commands and to generate additional parsing training data. The agent employs opportunistic active learning to select questions about how words relate to objects, improving its understanding of perceptual concepts. We evaluated this agent on Amazon Mechanical Turk. After training on data induced from conversations, the agent reduced the number of dialog questions it asked while receiving higher usability ratings. Additionally, we demonstrated the agent on a robotic platform, where it learned new perceptual concepts on the fly while completing a real-world task.

BibTeX Entry

@article{JAIR20-thomason,
  title={Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog},
  author={Jesse Thomason and Aishwarya Padmakumar and Jivko Sinapov and Nick Walker and Yuqian Jiang and Harel Yedidsion and Justin Hart and Peter Stone and Raymond J. Mooney},
  journal={The Journal of Artificial Intelligence Research (JAIR)},
  volume={67},
  year={2020},
  month={February},
  abstract={In this work, we present methods for using human-robot dialog to
  improve language understanding for a mobile robot agent. The agent parses 
  natural language to underlying semantic meanings and uses robotic sensors to 
  create multi-modal models of perceptual concepts like \emph{red} and 
  \emph{heavy}. The agent can be used for showing navigation routes, 
  delivering objects to people, and relocating objects from one location to 
  another. We use dialog clarification questions both to understand commands 
  and to generate additional parsing training data. The agent employs 
  opportunistic active learning to select questions about how words relate to 
  objects, improving its understanding of perceptual concepts. We evaluated 
  this agent on Amazon Mechanical Turk. After training on data induced from 
  conversations, the agent reduced the number of dialog questions it asked 
  while receiving higher usability ratings. Additionally, we demonstrated the 
  agent on a robotic platform, where it learned new perceptual concepts on the 
  fly while completing a real-world task.}
}

Generated by bib2html.pl (written by Patrick Riley ) on Wed Nov 25, 2020 10:07:49