UTCS AI Colloquia - Joyce Chai, "Bridging the Gaps of Perception and Action towards Situated Human-Robot Dialogue"

Contact Name: 
Karl Pichotta
GDC 6.302
Mar 21, 2014 2:00pm - 3:00pm

Signup Schedule: http://apps.cs.utexas.edu/talkschedules/cgi/list_events.cgi

Talk Audience: UTCS Faculty, Grads, Undergrads, Other Interested Parties

Host:  Ray Mooney

Talk Abstract: A new generation of robots have emerged in recent years which serve as humans’ assistants and companions. However, due to significantly mismatched capabilities in perception and action between humans and robots, natural language based communication becomes difficult. First, the robot’s representation of its perceived world and its action space are often continuous and numerical in nature. But human language is discrete and symbolic. For the robot to understand human language and thus take corresponding actions, it needs to first ground the meanings of human language to its own sensorimotor representation. Second, the robot may not have complete knowledge about the shared environment and the joint task. It may not be able to connect human language to its own representations. It is important for the robot to continuously acquire new knowledge through interaction with humans and the environment.

To address these challenges, we have been working towards bridging the gaps of perception and action between humans and robots. To mediate perceptual differences, we have developed graph-based approaches to support collaborative dialogue for referential grounding. We have also developed collaborative models for referring expression generation that takes into account of perceptual differences. To bridge the gap of action, we are exploring approaches for the robot (a robotic arm) to learn new high-level actions through natural language dialogue. In this talk, I will give an introduction to this line of research and discuss our approaches and empirical results.

Speaker Bio: Joyce Chai is a Professor in the Department of Computer Science and Engineering at Michigan State University. She received a Ph.D. in Computer Science from Duke University in 1998. Prior to joining MSU in 2003, she was a Research Staff Member at IBM T. J. Watson Research Center. Her research interests include natural language processing, situated dialogue agents, information extraction and retrieval, and intelligent user interfaces. She has served as a Program Co-chair for the Annual Meeting of Special Interest Group on Dialogue and Discourse (SIGDIAL) in 2011 and a Program Co-chair for the ACM International Conference on Intelligent User Interfaces in 2014. She is a recipient of the National Science Foundation Career Award in 2004 and a co-recipient of the Best Long Paper Award from the Annual Meeting of Association of Computational Linguistics (ACL) in 2010.