All notes (PDFs and annotated PDFs) and lecture recordings (including audio transcripts) can be found on the course canvas page.

Lec # Date Topic Readings Announcements
1 1/19 Introduction Assignment 0 released.
2 1/21 Compute Infrastructure Datacenter as a computer, Chapters 1, 2
3 1/24 Storage HDFS (sections I, II.A-C, III.A-E) Deadline: Assignment 0, Assignment 1 released
4 1/26 MapReduce MapReduce (Section 1--3)
5 1/28 MapReduce contd. MapReduce
6 1/31 Spark Spark
7 2/2 Schedulers Mesos
8 2/4 Resource Allocation Policies DRF
9 2/9 Batch Analytics SparkSQL Deadline: Assignment 1, Assignment 2 released
10 2/11 SparkSQL Code Jam No assigned reading
11 2/14 Data formats and operators Hive
12 2/16 Data Warehouses Snowflake
13 2/18 Assignment 2 Q&A and streaming/ML demo No assigned reading
14 2/21 Data-driven execution Whiz Extended Deadline: Assignment 2 till 2/27
15 2/28 Streaming SparkStreaming
16 3/4 Semantics of streaming systems and Flink Flink
17 3/7 Flink continued and Spark Streaming demo No assigned reading
18 3/9 Storm and Heron Heron
19 3/11 Guest lecture on conversational data systems No assigned reading Deadline: Assignment 3 till 3/20
20 3/21 GraphX GraphX Assignment 4 released with deadline till 4/4
21 3/23 GraphFrames tutorial No assigned reading
22 3/25 GraphX continued No assigned reading
23 3/28 Custom Graph Processing Systems - Pregel and Powergraph Pregel and Powergraph (optional)
24 3/30 Guest lecture by Matei Zaharia, founder DataBricks No assigned reading
25 4/1 Distributed Machine Learning Introduction Pytorch and Parameter Server (optional)
26 4/4 Pytorch and training architectures Pytorch and Parameter Server (optional) Assignment 5 released with deadline till 4/24
27 4/6 Demo of distributed training and inference No assigned reading
28 4/6 Guest lecture by Raajay Viswanathan, (Uber AI Platforms) No assigned reading
29 4/12 Deep learning compilers (TVM and Onyx) No assigned reading
30 4/13 Inference systems and Clipper Clipper
31 4/15 Ray and ML Platforms Ray
32 4/18 GPU schedulers Gandiva
33 4/20
34 4/22
35 4/25
36 4/27
37 4/29
38 5/2
39 5/4
40 5/6