All notes (PDFs and annotated PDFs) and lecture recordings (including audio transcripts) can be found on the course canvas page.
Lec # | Date | Topic | Readings | Announcements |
1 | 1/19 | Introduction | Assignment 0 released. | |
2 | 1/21 | Compute Infrastructure | Datacenter as a computer, Chapters 1, 2 | |
3 | 1/24 | Storage | HDFS (sections I, II.A-C, III.A-E) | Deadline: Assignment 0, Assignment 1 released |
4 | 1/26 | MapReduce | MapReduce (Section 1--3) | |
5 | 1/28 | MapReduce contd. | MapReduce | |
6 | 1/31 | Spark | Spark | |
7 | 2/2 | Schedulers | Mesos | |
8 | 2/4 | Resource Allocation Policies | DRF | |
9 | 2/9 | Batch Analytics | SparkSQL | Deadline: Assignment 1, Assignment 2 released |
10 | 2/11 | SparkSQL Code Jam | No assigned reading | |
11 | 2/14 | Data formats and operators | Hive | |
12 | 2/16 | Data Warehouses | Snowflake | |
13 | 2/18 | Assignment 2 Q&A and streaming/ML demo | No assigned reading | |
14 | 2/21 | Data-driven execution | Whiz | Extended Deadline: Assignment 2 till 2/27 |
15 | 2/28 | Streaming | SparkStreaming | |
16 | 3/4 | Semantics of streaming systems and Flink | Flink | |
17 | 3/7 | Flink continued and Spark Streaming demo | No assigned reading | |
18 | 3/9 | Storm and Heron | Heron | |
19 | 3/11 | Guest lecture on conversational data systems | No assigned reading | Deadline: Assignment 3 till 3/20 |
20 | 3/21 | GraphX | GraphX | Assignment 4 released with deadline till 4/4 |
21 | 3/23 | GraphFrames tutorial | No assigned reading | |
22 | 3/25 | GraphX continued | No assigned reading | |
23 | 3/28 | Custom Graph Processing Systems - Pregel and Powergraph | Pregel and Powergraph (optional) | |
24 | 3/30 | Guest lecture by Matei Zaharia, founder DataBricks | No assigned reading | |
25 | 4/1 | Distributed Machine Learning Introduction | Pytorch and Parameter Server (optional) | |
26 | 4/4 | Pytorch and training architectures | Pytorch and Parameter Server (optional) | Assignment 5 released with deadline till 4/24 |
27 | 4/6 | Demo of distributed training and inference | No assigned reading | |
28 | 4/6 | Guest lecture by Raajay Viswanathan, (Uber AI Platforms) | No assigned reading | |
29 | 4/12 | Deep learning compilers (TVM and Onyx) | No assigned reading | |
30 | 4/13 | Inference systems and Clipper | Clipper | |
31 | 4/15 | Ray and ML Platforms | Ray | |
32 | 4/18 | GPU schedulers | Gandiva | |
33 | 4/20 | |||
34 | 4/22 | |||
35 | 4/25 | |||
36 | 4/27 | |||
37 | 4/29 | |||
38 | 5/2 | |||
39 | 5/4 | |||
40 | 5/6 |