Instructors 
Adam Klivans and Pradeep Ravikumar

Office Hours 
ACES 2.434, Mondays 3:305:00 pm (by appointment)

Overview 
A central problem in machine learning is to develop algorithms that have provable guarantees in terms of both running time and number of "training" observations required. Computational Learning Theory has traditionally focused on the first issue (the computational complexity of learning algorithms) while Statistical Learning Theory has focused on the second (their statistical efficiency). In this course we will cover both these aspects, and try to understand how learning is constrained given limited computation and limited data.

Grading 
Four problem sets (3/4 of final grade), and a final paper presentation (1/4 of final grade).

(Optional) Textbooks 
An Introduction to Computational Learning Theory. Michael Kearns, Umesh Vazirani
A Probabilistic Theory of Pattern Recognition.
Luc Devroye, Laszlo Györfi, Gabor Lugosi.
The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Trevor Hastie, Robert Tibshirani, Jerome Friedman.

Homeworks 

(Tentative) Schedule 
Module 
Date 
Topic 
Notes 
Faculty 
1,2 

... 
... 
AK 


... 
... 
AK 


Fourier Learning 
Papers: [1]
[2]
[3]

Guest Lect: Homin Lee 
3: Statistical Analysis Foundations
(2 weeks). Consistency, Convergence Rates, Generalization Error.


Useful Inequalities 
DGL; Chapter 8 
PR 

GlivenkoCantelli Theorem 
DGL; Chapter 12 
PR 

VC Theorem 
DGL; Chapter 12 
PR 
4: Complexity of Learning
(2 weeks). VC Theory, Metric Entropy, Rademacher Complexity, Margin Bounds.


Shatter Coefficients, VC Dimension 
DGL; Chapter 13 
PR 

Metric Entropy 
DGL; Chapter 28 
PR 

Uniform Deviations of Averages from Expectations 
DGL; Chapter 29 
PR 

Rademacher Complexity 

Guest Lect: Ambuj Tewari 
5: Misc


Surrogate Losses for Classification 
Paper: [1] 
PR 

Reproducing Kernel Hilbert Spaces 
Book: [1] 
PR 
