CS388: Natural Language Processing (Fall 2018)

NOTE: This page is for an old semester of this class

Instructor: Greg Durrett, gdurrett@cs.utexas.edu
Lecture: Tuesday and Thursday 9:30am - 11:00am, GDC 4.304
Instructor Office Hours: Wednesday 10am-12pm, GDC 3.420
TA: Jifan Chen
TA Office Hours: Monday 2pm-3pm, Tuesday 1pm-2pm, GDC 1.304 TA Desk 1

Piazza

Description

This class is a graduate-level introduction to Natural Language Processing (NLP), the study of computing systems that can process, understand, or communicate in human language. The course covers fundamental approaches, largely machine learning and deep learning, used across the field of NLP as well as a comprehensive set of NLP tasks both historical and contemporary. Techniques studied include Bayes nets, conditional random fields, structured max-margin, tree-structured models, recurrent and convolutional neural networks, and attention mechanisms. Problems range from syntax (part-of-speech tagging, parsing) to semantics (lexical semantics, question answering, grounding) and include various applications such as summarization, machine translation, information extraction, and dialogue systems. Programming assignments throughout the semester involve building scalable machine learning systems for various NLP tasks.

Requirements

• 391L - Machine Learning, 343 - Artificial Intelligence, or equivalent AI/ML course experience
• Familiarity with Python (for programming assignments)
• Additional prior exposure to discrete math, probability, linear algebra, optimization, linguistics, and NLP useful but not required

Syllabus

Detailed syllabus with course policies

Assignments: There are four programming assignments that require implementing models discussed in class. Framework code in Python and datasets will be provided. In addition, there is an open-ended final project to be done in teams of 2 (preferred) or individually. This project should constitute novel exploration beyond directly implementing concepts from lecture and should result in a report that roughly reads like an NLP/ML conference submission in terms of presentation and scope.

Mini 1: Classification for Person Name Detection [code and dataset download]

Project 1: CRF tagging for NER [code and dataset download]

Mini 2: Neural Networks for Sentiment Analysis [code and dataset download]

Project 2: Encoder-Decoder Models for Question Answering [code and dataset download]

Final Project

Readings: Readings are purely optional and intended to supplement lecture and give you another view of the material. Two main sources will be used:

• Eisenstein: Natural Language Processing. I recommend this book highly: I think the presentation is overall a bit closer to mine than the presentation in Jurafsky and Martin. However, I will suggest readings from both books and you can decide for yourself which one you prefer.
• Jurafsky and Martin: Speech and Language Processing (3rd ed. draft). This is a draft textbook so chapters are available online in PDF. This is the "classic" NLP book.
• Goldberg: A Primer on Neural Network Models for Natural Language Processing

Readings for future lectures are tentative and subject to change.

Date	Topics	Readings	Assignments
Aug 30	Introduction [4pp]		Mini1 out
Sept 4	Binary classification [4pp]	Eisenstein 2.0-2.5, 4.2-4.4.1, JM 4, JM 5.0-5.5
Sept 6	Multiclass classification [4pp]	Eisenstein 4.2, JM 5.6, Structured SVM secs 1-2
Sept 11	Sequence models I: HMMs [4pp]	Eisenstein 7.0-7.4, 8.1, JM 8, Manning POS	Mini1 due / Proj1 out
Sept 13	Sequence models II: CRFs [4pp]	Eisenstein 7.5, 8.3, Sutton CRFs 2.3, 2.6.1, Wallach CRFs tutorial, Illinois NER
Sept 18	Neural Nets I: FFNNs [4pp]	Eisenstein 3.0-3.3, Goldberg 1-4, 6, NLP with FFNNs, DANs
Sept 20	Neural Nets II: Word embeddings [4pp]	Eisenstein 3.3.4, 14.5-14.6, JM 6, Goldberg 5, word2vec, Levy, GloVe, Dropout
Sept 25	Neural Nets III: RNN encoders [4pp]	Goldberg 10-11, Karpathy
Sept 27	Neural Nets IV: CNNs, Neural CRFs [4pp]	Eisenstein 3.4, 7.6, Goldberg 9, Kim, Collobert and Weston, Neural NER	Proj1 due / Mini2 out
Oct 2	Trees I: Constituency, PCFGs [4pp]	Eisenstein 10.0-10.5, JM 12.1-12.6, 12.8, Structural, Lexicalized, State-split
Oct 4	Trees II: Dependency I [4pp]	Eisenstein 11.1-11.2, JM 13.1-13.3, 13.5, Dozat
Oct 9	Trees III: Dependency II [4pp]	Eisenstein 11.3, JM 13.4, Parsey, Huang 2
Oct 11	Semantics I [4pp]	Eisenstein 12, Zettlemoyer, Berant	Mini2 due
Oct 16	Semantics II / Seq2seq I [4pp]	Seq2seq, Jia
Oct 18	Seq2seq II: Attention and Pointers [4pp]	Attention, Luong Attention, Transformer	Proj2 out
Oct 23	Information Extraction / SRL [4pp]	Eisenstein 13, 17, SRL, Distant supervision, TextRunner, ReVerb
Oct 25	Machine Translation I: Phrase-based [4pp]	Eisenstein 18.1-18.2, HMM alignment, Pharaoh
Oct 30	Machine Translation II: Neural [4pp]	Eisenstein 18.3
Nov 1	Applications I: Reading comprehension / MemNets [4pp]	E2E Memory Networks, CBT, SQuAD, BiDAF	Proj2 due *Nov 2*
Nov 6	Applications II: Summarization [4pp]	Eisenstein 19, MMR, Gillick, Sentence compression, SummaRuNNER, Pointer
Nov 8	Applications III: Dialogue [4pp]	RNN chatbots, Diversity, Goal-oriented, Latent Intention, QA-as-dialogue	FP Proposals due
Nov 13	Applications IV: Language grounding [4pp]	Spatial, Captioning, VQA 2.0, Negotiation
Nov 15	Unsupervised Learning [4pp]	Painless unsup, Bowman VAE, ELMo, BERT
Nov 20	Discourse (Guest lecture: Jessy Li)
Nov 22	NO CLASS (Thanksgiving)
Nov 27	Multilinguality and morphology [4pp]	Xlingual POS, Xlingual parsing, Xlingual embeddings
Nov 29	Wrapup + Ethics [4pp]
Dec 4	Project presentations I
Dec 6	Project presentations II
Dec 14			FP due