CS395T - Fall 2026: Schedule

Lecture Schedule

Below is the tentative schedule for the course. Note that dates and topics may change as the semester progresses.

Topics marked as lectures will be presented by Professor Pingali. All other presentations are by students in the course.

Paper presentation and review sign-ups can be found here.

Meeting	Date	Topic	Materials/Readings	Deadlines
1	8/25 (T)	Lecture: Introduction (Slides)	AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery (Novikov et al., 2025) DeepSeek-V3 Technical Report (DeepSeek-AI et al., 2024) AlphaGo The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition (Lange et al., 2025) Website with latest results on kernel generation: https://kernelbench.com/
2	8/27 (Th)	Lecture: Abstract NN & gradient computation (Slides)	Haykin Textbook - Introduction; Chapter 1: Perceptron; Chapter 4: Multi-layer Perceptrons (MLPs) Who invented backpropagation? (Schmidhuber 2014) Tutorial on Autograd Deep Residual Learning for Image Recognition (ResNet) (Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, 2015)	Introduction post/survey due
5	9/1 (T)	Lecture: Memory hierarchy optimization (Slides)
3	9/3 (Th)	Lecture: GPU architectures (1) (Slides)		Presentation sign-up due (paper presentation and paper review)
4	9/8 (T)	Lecture: GPU architectures (2)
6	9/10 (Th)	Case studies of high-performance implementations (I)	Matrix multiplication: Modular FlashAttention FlashAttention 3 FlashAttention 4
7	9/15 (T)	Case studies of high-performance implementations (II)	PagedAttention DeepSeek-V3 Technical Report
8	9/17 (Th)	Benchmark suites for ML-generated kernels	KernelBench SOL-ExecBench
9	9/22 (T)	Lecture: Attention, Transformers, LLMs (I) (Slides)	Neural Machine Translation by Jointly Learning to Align & Translate (Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, 2014) Attention Is All You Need (Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, 2017) Understanding and Coding Self-Attention (Sebastian Raschka)
10	9/24 (Th)	Lecture: Attention, Transformers, LLMs (II)	How to Scale Your Model (Google DeepMind) Meta Large Language Model Compiler: Foundation Models of Compiler Optimization (Cummins et al., 2024)
11	9/29 (T)	Languages: in addition to CUDA (I)	Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations Gluon
12	10/1 (Th)	Languages: in addition to CUDA (II)	ThunderKittens: Simple, Fast, and Adorable AI Kernels CAKE: Compiler-Agent Co-Design for Frontier Kernel Evolution
13	10/6 (T)	Lecture: Monte Carlo methods & variance reduction (Slides)	Classic tutorial by Hammersley & Handscombe Barto & Sutton – Ch 5 Monte Carlo Methods for MDPS
14	10/8 (Th)	Lecture: Markov Decision Processes (MDPs) (Slides)	Barto & Sutton – Ch 3&4 MDPs
15	10/13 (T)	Lecture: Tabular methods (TD(0), TD(n), Q-learning, MC) (Slides)	Barto & Sutton – Ch 6 Temporal Difference Learning
16	10/15 (Th)	Lecture: Policy gradient methods (I) (Slides)	Barto & Sutton – Ch 13: Policy Gradients OpenAI DRL – Intro to Policy Optimization Definitive Guide to Policy Gradients §1–2 (Matthias Lehmann, 2024) Deep RL: Pong from Pixels (Andrej Karpathy) Karpathy - Overview of Policy Gradients Video
17	10/20 (T)	Lecture: Policy gradient methods (II): (Baseline slides) (TRPO/PPO slides)	Trust Region Policy Optimization (TRPO) (John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz, 2015) Proximal Policy Optimization (PPO) (John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov, 2017)
18	10/22 (Th)	RL-based systems (I)	RLTF: Reinforcement Learning from Unit Test Feedback Training language models to follow instructions with human feedback CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
19	10/27 (T)	RL-based systems (II)	Kevin Dr. Kernel SparseRL: Mastering Sparse CUDA Generation through Pretrained Models and Deep Reinforcement Learning
20	10/29 (Th)	Evolutionary computing	KernelEvolve KernelFoundry
21	11/3 (T)	Verification of ML-generated kernels	Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimization KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
22	11/5 (Th)	Agentic systems (I)	GEAK: Introducing Triton Kernel AI Agent & Evaluation Benchmarks AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation
23	11/10 (T)	Lecture: Evolutionary Computation (Slides)	Neuroevolution (Risto Miikkulainen, 2022) Evolving neural networks through augmenting topologies (Kenneth O. Stanley, Risto Miikkulainen, 2002)	Project check-in (meeting required (11/13, 11/15)
24	11/12 (Th)	Agentic systems (II)	CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation STARK: Strategic Team of Agents for Refining Kernels
25	11/17 (T)	Domain-specific knowledge	AutoComp: A Powerful and Portable Code Optimizer for Tensor Accelerators Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization
26	11/19 (Th)	Making models smaller	From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph MOPD
		Thanksgiving break
27	12/1 (T)	Project presentations
28	12/3 (Th)	Project presentations		Final project paper due (no extensions will be allowed, no exceptions)