The University of Texas at Austin

Computer Science 395T


This course will cover a wide range of topics in numerical optimization. The major goal is to learn a set of tools that will be useful for research in Artificial Intelligence and Computer Graphics. The course is a graduatelevel course that combines instruction of basic material, written homeworks , and a final project. The course material integrates the theory of optimization and concrete real applications. Grading is based on homeworks (50%) and the final project (50%).
Prereqs: The course assumes a good knowledge of linear algebra and probability. Please talk to me or email me if you are unsure if the course is a good match for your background.
Textbook: Numerical Optimization.
Date  Topics  Reading  Notes 
August 31th  Introduction  
September 5th  Linear Algebra  Introduction to linear algebra.  Homework 1. 
September 7th  Probability  Basic concentration bounds.  
September 12th  Fundamentals of Unconstrained Optimization  
September 14th  Line Search  
September 19th  Line Search Applications  
September 21th  Trust Region Methods I (Subproblem)  Homework 1 due. Homework 2 out.  
September 26th  Trust Region Methods II (Global Convergence)  
September 28th  Trust Region Methods (Applications)  
October 3th  Conjugate Gradient Methods (Linear)  
October 5th  Conjugate Gradient Methods (Nonlinear)  Homework 2 due. Homework 3 out.  
October 10th  Proximal Gradient Methods  
October 12th  Theory of Constrained Optimization  
October 17th  Optimality Condition  Final project proposal Due.  
October 19th  Linear Programming (Simplex Method)  Homework 3 due. Homework 4 out.  
October 24th  Linear Programming (Simplex Method II and Interior Point Method I)  
October 26th  Linear Programming (Interior Point Method II)  
October 31th  Quadratic Programming (Algorithms and Applications)  
November 2nd  Guest Lecture  
November 7th  Quadratic Programming II (Algorithms and Applications)  
November 9th  Penalty, Augmented Lagrangian and SDP  
November 14th  Spectral Methods I  Homework 4 due. Homework 5 out.  
November 16th  Spectral Methods II  
Novmeber 21th  Topics in Convex Optimization I (Compressive Sensing)  
November 28th  Topics in Convex Optimization II (Lowrank Matrix Recovery)  
November 30th  Topics in NonConvex Optimization I (Lowrank Matrix Recovery)  
December 5th  Topics in NonConvex Optimization II (Deep Neural Networks)  Homework 5 due.  
December 7th  Topics in NonConvex Optimization III (Reweighted Least Squares)  Final project report due. 