Learning to Interpret Natural Language Navigation Instructions from Observations (2011)
The ability to understand natural-language instructions is critical to building intelligent agents that interact with humans. We present a system that learns to transform natural-language navigation instructions into executable formal plans. Given no prior linguistic knowledge, the system learns by simply observing how humans follow navigation instructions. The system is evaluated in three complex virtual indoor environments with numerous objects and landmarks. A previously collected realistic corpus of complex English navigation instructions for these environments is used for training and testing data. By using a learned lexicon to refine inferred plans and a supervised learner to induce a semantic parser, the system is able to automatically learn to correctly interpret a reasonable fraction of the complex instructions in this corpus.
Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI-2011) (2011), pp. 859-865.

Slides (PPT)
David Chen Ph.D. Alumni cooldc [at] hotmail com
Raymond J. Mooney Faculty mooney [at] cs utexas edu