CS 395T:
Grounded Natural Language Processing


Raymond J. Mooney

Time and Place

Spring 2019, Mon & Wed, 9:30-11am, GDC 2.410, (unique# 51240)

Office Hours

Course Information

This course will be a graduate research seminar in grounded NLP, a subarea of AI that studies the connection between natural language and perception and action in the world. It makes connections between NLP and computer vision, robotics, and computer graphics. Almost all work in the area uses machine learning to learn the connection between language and perception and/or action from some form of multi-modal training data.

The class will mostly revolve around reading both seminal and recent research papers in the area. Each student will be required to prepare an organized oral presentation on one or more papers and participate in class discussion about these papers. Many of the papers will be selected from the ones that have been covered in the local CLAMP reading group over the last 11 years.

Finally, it will involve conducting an original research project in the area, writing a paper on this project suitable for submitting to a research conference, and possibly presenting an oral presentation on the project.

The details of the specific class requirements (e.g. how many paper presentations and whether projects are individual or group) will partly depend on the number of students who enroll in the class.


This class is intended for graduate students actively working in NLP, computer vision, robotics, machine learning, and/or computer graphics. Students should have had a graduate class in at least one of these areas, ideally more than one. A graduate NLP class such as CS 388 will be particularly valuable.