CS388: Natural Language Processing

Instructor: Greg Durrett, gdurrett@cs.utexas.edu
Lecture: Tuesday and Thursday 9:30am - 11:00am, GDC 4.304
Instructor Office Hours: Wednesday 10am-12pm, GDC 3.420
TA: Jifan Chen
TA Office Hours: Monday 2pm-3pm, Tuesday 1pm-2pm, GDC 1.304 TA Desk 1

Piazza Piazza

Description

This class is a graduate-level introduction to Natural Language Processing (NLP), the study of computing systems that can process, understand, or communicate in human language. The course covers fundamental approaches, largely machine learning and deep learning, used across the field of NLP as well as a comprehensive set of NLP tasks both historical and contemporary. Techniques studied include Bayes nets, conditional random fields, structured max-margin, tree-structured models, recurrent and convolutional neural networks, and attention mechanisms. Problems range from syntax (part-of-speech tagging, parsing) to semantics (lexical semantics, question answering, grounding) and include various applications such as summarization, machine translation, information extraction, and dialogue systems. Programming assignments throughout the semester involve building scalable machine learning systems for various NLP tasks.

Requirements

Syllabus

Detailed syllabus with course policies

Assignments: There are four programming assignments that require implementing models discussed in class. Framework code in Python and datasets will be provided. In addition, there is an open-ended final project to be done in teams of 2 (preferred) or individually. This project should constitute novel exploration beyond directly implementing concepts from lecture and should result in a report that roughly reads like an NLP/ML conference submission in terms of presentation and scope.

Mini 1: Classification for Person Name Detection [code and dataset download]

Project 1: CRF tagging for NER [code and dataset download]

Mini 2: Neural Networks for Sentiment Analysis [code and dataset download]

Project 2: Encoder-Decoder Models for Question Answering [code and dataset download]

Final Project

Readings: Readings are purely optional and intended to supplement lecture and give you another view of the material. Two main sources will be used:

Readings for future lectures are tentative and subject to change.

Date Topics Readings Assignments
Aug 30 Introduction [4pp] Mini1 out
Sept 4 Binary classification [4pp] Eisenstein 2.0-2.5, 4.2-4.4.1, JM 4, JM 5.0-5.5
Sept 6 Multiclass classification [4pp] Eisenstein 4.2, JM 5.6, Structured SVM secs 1-2
Sept 11 Sequence models I: HMMs [4pp] Eisenstein 7.0-7.4, 8.1, JM 8, Manning POS Mini1 due / Proj1 out
Sept 13 Sequence models II: CRFs [4pp] Eisenstein 7.5, 8.3, Sutton CRFs 2.3, 2.6.1, Wallach CRFs tutorial, Illinois NER
Sept 18 Neural Nets I: FFNNs [4pp] Eisenstein 3.0-3.3, Goldberg 1-4, 6, NLP with FFNNs, DANs
Sept 20 Neural Nets II: Word embeddings [4pp] Eisenstein 3.3.4, 14.5-14.6, JM 6, Goldberg 5, word2vec, Levy, GloVe, Dropout
Sept 25 Neural Nets III: RNN encoders [4pp] Goldberg 10-11, Karpathy
Sept 27 Neural Nets IV: CNNs, Neural CRFs [4pp] Eisenstein 3.4, 7.6, Goldberg 9, Kim, Collobert and Weston, Neural NER Proj1 due / Mini2 out
Oct 2 Trees I: Constituency, PCFGs [4pp] Eisenstein 10.0-10.5, JM 12.1-12.6, 12.8, Structural, Lexicalized, State-split
Oct 4 Trees II: Dependency I [4pp] Eisenstein 11.1-11.2, JM 13.1-13.3, 13.5, Dozat
Oct 9 Trees III: Dependency II [4pp] Eisenstein 11.3, JM 13.4, Parsey, Huang 2
Oct 11 Semantics I [4pp] Eisenstein 12, Zettlemoyer, Berant Mini2 due
Oct 16 Semantics II / Seq2seq I [4pp] Seq2seq, Jia
Oct 18 Seq2seq II: Attention and Pointers [4pp] Attention, Luong Attention, Transformer Proj2 out
Oct 23 Information Extraction / SRL [4pp] Eisenstein 13, 17, SRL, Distant supervision, TextRunner, ReVerb
Oct 25 Machine Translation I: Phrase-based [4pp] Eisenstein 18.1-18.2, HMM alignment, Pharaoh
Oct 30 Machine Translation II: Neural [4pp] Eisenstein 18.3
Nov 1 Applications I: Reading comprehension / MemNets [4pp] E2E Memory Networks, CBT, SQuAD, BiDAF Proj2 due *Nov 2*
Nov 6 Applications II: Summarization [4pp] Eisenstein 19, MMR, Gillick, Sentence compression, SummaRuNNER, Pointer
Nov 8 Applications III: Dialogue [4pp] RNN chatbots, Diversity, Goal-oriented, Latent Intention, QA-as-dialogue FP Proposals due
Nov 13 Applications IV: Language grounding [4pp] Spatial, Captioning, VQA 2.0, Negotiation
Nov 15 Unsupervised Learning [4pp] Painless unsup, Bowman VAE, ELMo, BERT
Nov 20 Discourse (Guest lecture: Jessy Li)
Nov 22 NO CLASS (Thanksgiving)
Nov 27 Multilinguality and morphology [4pp] Xlingual POS, Xlingual parsing, Xlingual embeddings
Nov 29 Wrapup + Ethics [4pp]
Dec 4 Project presentations I
Dec 6 Project presentations II
Dec 14 FP due