CS388: Natural Language Processing

Instructor: Greg Durrett, gdurrett@cs.utexas.edu
Lecture: Tuesday and Thursday 12:30pm - 2:00pm, GDC 4.302
Instructor Office Hours: Wednesday 4pm-5pm, Thursday 2pm-3pm, GDC 3.812
TA: Uday Kusupati, kusupatiuday@gmail.com
TA Office Hours: Monday 12pm-1pm, Tuesday 11am-12pm, GDC 1.302

Piazza Piazza

Description

This class is a graduate-level introduction to Natural Language Processing (NLP), the study of computing systems that can process, understand, or communicate in human language. The course covers fundamental approaches, largely machine learning and deep learning, used across the field of NLP as well as a comprehensive set of NLP tasks both historical and contemporary. Techniques studied include Bayes nets, conditional random fields, structured max-margin, tree-structured models, recurrent and convolutional neural networks, and attention mechanisms. Problems range from syntax (part-of-speech tagging, parsing) to semantics (lexical semantics, question answering, grounding) and include various applications such as summarization, machine translation, information extraction, and dialogue systems. Programming assignments throughout the semester involve building scalable machine learning systems for various NLP tasks.

Requirements

Syllabus

Detailed syllabus with course policies

Assignments: There are four programming assignments that require implementing models discussed in class. Framework code in Python and datasets will be provided. In addition, there is an open-ended final project to be done in teams of 2 (preferred) or individually. This project should constitute novel exploration beyond directly implementing concepts from lecture and should result in a report that roughly reads like an NLP/ML conference submission in terms of presentation and scope.

Mini 1: Classification for Person Name Detection [code and dataset download]

Project 1: CRF tagging for NER [code and dataset download] [great project writeup] [good project writeup]

Mini 2: Neural Networks for Sentiment Analysis

Project 2: Encoder-Decoder Models for Question Answering

Final Project

Readings: Readings are purely optional and intended to supplement lecture and give you another view of the material. Two main sources will be used:

Readings for future lectures are tentative and subject to change.

Date Topics Readings Assignments
Aug 29 Introduction [4pp] Mini1 out
Sept 3 Binary classification [4pp] Eisenstein 2.0-2.5, 4.2-4.4.1, JM 4, JM 5.0-5.5
Sept 5 Multiclass classification [4pp] Eisenstein 4.2, JM 5.6, Structured SVM secs 1-2
Sept 10 Sequence Models 1: HMMs (Guest Lecture: Ray Mooney) [4pp] Eisenstein 7.0-7.4, 8.1, JM 8, Manning POS, Viterbi algorithm lecture note Mini1 due / Proj1 out
Sept 12 Sequence Models 2: CRFs [4pp] Eisenstein 7.5, 8.3, Sutton CRFs 2.3, 2.6.1, Wallach CRFs tutorial, Illinois NER
Sept 17 NN1: Feedforward [4pp] Eisenstein 3.0-3.3, Goldberg 1-4, 6, NLP with FFNNs, DANs, ffnn_example.py
Sept 19 NN2: Word embeddings [4pp] Eisenstein 3.3.4, 14.5-14.6, JM 6, Goldberg 5, word2vec, Levy, GloVe, fastText, Bolukbasi, Gonen Debiasing
Sept 24 NN3: RNNs JM 9.1-9.4, Goldberg 10-11, Karpathy Proj1 due
Sept 26 NN4: Language Modeling and Pretraining Eisenstein 6, JM 9.2.1, ELMo Mini2 out
Oct 1 NN5: Interpretability/CNNs/Neural CRFs/etc.
Oct 3 Trees 1: Constituency, PCFGs Eisenstein 10.0-10.5, JM 12.1-12.6, 12.8, Structural, Lexicalized, State-split
Oct 8 Trees 2: Constituency Parsers + Dependency Eisenstein 11.1-11.2, JM 13.1-13.3, 13.5, Dozat Mini2 due / FP out
Oct 10 Trees 3: Dependency Parsers Eisenstein 11.3, JM 13.4, Parsey, Huang 2
Oct 15 Semantics 1 Eisenstein 12, Zettlemoyer, Berant FP proposal due
Oct 17 Semantics 2 / Seq2seq 1 Seq2seq, Jia Proj2 out
Oct 22 Seq2seq 2: Attention and Pointers Attention, Luong Attention, Transformer
Oct 24 Machine Translation 1
Oct 29 Machine Translation 2 / Transformers
Oct 31 Pretrained Transformers / BERT BERT, RoBERTa
Nov 5 Information Extraction / SRL Proj2 due
Nov 7 Question Answering 1
Nov 12 Question Answering 2
Nov 14 Dialogue RNN chatbots, Diversity, Goal-oriented, Latent Intention, QA-as-dialogue
Nov 19 Summarization Eisenstein 19, MMR, Gillick, Sentence compression, SummaRuNNER, Pointer
Nov 21 Multilinguality and morphology Xlingual POS, Xlingual parsing, Xlingual embeddings
Nov 26 Wrapup + Ethics
Nov 28 No class (Thanksgiving)
Dec 3 Project presentations 1
Dec 5 Project presentations 2
Dec 13 FP due