Data Mining: A Mathematical Perspective

CS 391D

Unique No. 50761

Spring 2020
Fri 10am-1pm
GDC 3.516

Instructor: Prof. Inderjit Dhillon (send email)
Office: GDC 4.704
Office Hours: Fri 9-10am and by appointment
Guest Lecturer: Ali Jalali (send email)

TA: Xingchao Liu (send email)
Office: GDC 4.802C
Office Hours: TTh 3-4:30pm

Course Description

Data mining is the automated discovery of interesting patterns and relationships in massive data sets. This graduate course will focus on various mathematical and statistical aspects of data mining and machine learning. Topics covered include supervised methods (regression, classification, support vector machines) and unsupervised methods (clustering, principal components analysis, non-linear dimensionality reduction). The technical tools used in the course will draw from linear algebra, multivariate statistics and optimization. The main tools from these areas will be covered in class, but undergraduate level linear algebra is a pre-requisite (see below). A substantial portion of the course will focus on student presentations and projects; projects can vary in their theoretical/mathematical content, and in the implementation/programming involved. Projects will be conducted by teams of 2-4 students.

Pre-requisites: Basics (undergraduate level) of linear algebra (M341 or equivalent) and some mathematical sophistication.

Reference Books

  • "Deep Learning" by Ian Goodfellow, Yoshua Bengio and Aaron Courville, MIT Press, 2016.
  • "Pattern Recognition and Machine Learning" by C. Bishop, Springer, 2006.
  • "Elements of Statistical Learning: Data Mining, Inference, and Prediction" by T. Hastie, R. Tibshirani, J. Friedman, Springer-Verlag, 2001.
  • "Pattern Classification" by R. Duda, P. Hart and D. Stork, John Wiley and Sons, 2000.

    • Class Presentations

  • Schedule

    • Class Projects
  • Project Suggestions

    • Syllabus

  • Regression
  • Classification
  • Deep Learning
  • Unsupervised Learning

    • Lecture Notes

  • Lecture 1 - Linear Regression and Linear Algebra Background.
  • Lecture 2 - More on Linear Regression (Probabilistic Modeling, Overfitting, Regularization).
  • Lecture 3 - Probability Theory Background, SVD, Ridge Regression.
  • Lecture 4 - Classification: Linear Methods, Logistic Regression.
  • Lecture 5 - Convexity, Gradient Descent.
  • Lecture 6 - Classification: Perceptron, Support Vector Machine.
  • Lecture 7 - Proximal Gradient Descent, Stochastic Gradient Descent (Notes).
  • Lecture 8 - Non-linearly Separable SVMs, Kernel SVMs and Introduction to Deep Learning.
  • Lecture 9 - Unsupervised Learning (Clustering).

    • Grading

  • 50% Class Project
  • 30% Homeworks
  • 20% Class Presentation

    • Handouts

  • Class Survey

    • Code of Conduct