Grounded Natural Language Processing
Raymond J. Mooney
Time and Place
Spring 2019, Mon & Wed, 9:30-11am, GDC 2.410, (unique# 51240)
- Instructor: Mon & Wed, 11am-noon, GDC 3.512
- TA: TBA, GDC 6.802A
General Course Information
This course will be a graduate research seminar in grounded natural language
processing (GNLP), a subarea
of AI that studies the connection between natural language and
perception and action in the world. It makes connections between natural
language processing (NLP) and
computer vision, robotics, and computer graphics. Almost all work in the
area uses machine learning to learn the connection between language and
perception and/or action from some form of multi-modal training data.
The class will mostly revolve around reading both seminal and recent research papers in the area.
Each student will be required to prepare organized oral presentations
of papers and participate in class discussion about these papers.
Many of the papers will be selected from the ones that have been covered in
the local CLAMP reading
group over the last 11 years.
Finally, it will involve conducting an original research project in the area, writing
a paper on this project suitable for submitting to a research conference,
and presenting an oral presentation on the project.
- Visually Grounded Language: Past,
Present, and Future, NIPS-2017 Workshop on Visually-Grounded Interaction
and Language (ViGIL), Long Beach, CA, Dec. 8, 2017.
- Using Natural Language Processing to Aid
Computer Vision, University of Washington
and Microsoft Research Workshop on Understanding Situated Language in Everyday Life, Union, WA,
July 22-25, 2013
- Explainable AI for VQA
- Joint VQA and Captioning