Grounded Natural Language Processing
Raymond J. Mooney
Time and Place
Spring 2021, Mon & Wed, 2-3:30pm, Zoom link on Canvas
- Instructor: Mon & Wed, 11am-noon, Zoom link on Canvas
- TA: Tue & Thu, 1pm-2pm, Zoom link on Canvas
General Course Information
This course will be a graduate research seminar in grounded natural language
processing (GNLP), a subarea
of AI that studies the connection between natural language and
perception and action in the world. It makes connections between natural
language processing (NLP) and
computer vision, robotics, and computer graphics. Almost all work in the
area uses machine learning to learn the connection between language and
perception and/or action from some form of multi-modal training data.
The class will mostly revolve around reading both seminal and recent research papers in the area.
Each student will be required to prepare organized oral presentations
of papers and participate in class discussion about these papers.
Many of the papers will be selected from the ones that have been covered in
the local CLAMP reading
group over the last 12 years.
Finally, it will involve conducting an original research project in the area, writing
a paper on this project suitable for submitting to a research conference,
and presenting an oral presentation on the project.
- Watch my AAAI 2013 Keynote talk "Grounded Language Learning" on video.
- Visually Grounded Language: Past,
Present, and Future, NIPS-2017 Workshop on Visually-Grounded Interaction
and Language (ViGIL), Long Beach, CA, Dec. 8, 2017.
- Using Natural Language Processing to Aid
Computer Vision, University of Washington
and Microsoft Research Workshop on Understanding Situated Language in Everyday Life, Union, WA,
July 22-25, 2013
- A Review of Work on Natural
Language Navigation Instructions, Combined Workshop on Spatial
Language Understanding (SpLU) & Grounded Communication for Robotics
(RoboNLP), NAACL 2019.