CS395T - Fall 2026: Schedule

Lecture Schedule

Below is the tentative schedule for the course. Note that dates and topics may change as the semester progresses.

Topics marked as lectures will be presented by Professor Pingali. All other presentations are by students in the course.

Paper presentation and review sign-ups can be found here.

Meeting	Date	Topic	Materials/Readings	Deadlines
1	8/25 (T)	Lecture: Introduction (Slides)	AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery (Novikov et al., 2025) DeepSeek-V3 Technical Report (DeepSeek-AI et al., 2024) AlphaGo The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition (Lange et al., 2025) Website with latest results on kernel generation: https://kernelbench.com/
2	8/27 (Th)	Lecture: Abstract NN & gradient computation (Slides)	Haykin Textbook - Introduction; Chapter 1: Perceptron; Chapter 4: Multi-layer Perceptrons (MLPs) Who invented backpropagation? (Schmidhuber 2014) Tutorial on Autograd Deep Residual Learning for Image Recognition (ResNet) (Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, 2015)	Introduction post/survey due
3	9/3 (Th)	Lecture: GPU architectures (1) (Slides)		Presentation sign-up due (paper presentation and paper review)
4	9/8 (T)	Lecture: GPU architectures (2)
5	9/10 (Th)	Lecture: Memory hierarchy optimization (Slides)
6	9/15 (T)	Students: Kernel programming languages [Triton, Thunderkittens]	Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations (Tillet, Kung, Cox, 2019) ThunderKittens: Simple, Fast, and Adorable AI Kernels (Spector et al., 2024)
7	9/17 (Th)	Students: Kernel generation benchmarks [KernelBench, SOL-ExecBench]	KernelBench: Can LLMs Write Efficient GPU Kernels? (Ouyang et al., 2025) NVIDIA/SOL-ExecBench github repository
8	9/22 (T)	Students: High-performance MMM	Modular's blog posts on MMM for Blackwell
9	9/24 (Th)	Lecture: Attention, Transformers, LLMs (I) (Slides)	Neural Machine Translation by Jointly Learning to Align & Translate (Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio, 2014) Attention Is All You Need (Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, 2017) Understanding and Coding Self-Attention (Sebastian Raschka)
10	9/29 (T)	Lecture: Attention, Transformers, LLMs (II)	How to Scale Your Model (Google DeepMind) Meta Large Language Model Compiler: Foundation Models of Compiler Optimization (Cummins et al., 2024)
11	10/1 (Th)	Students: Optimizing Attention (I)	FlashAttention (Ma et al., 2022) FlashAttention-3 (Shah et al., 2024) FlashAttention-4 (Zadouri et al. 2026)
12	10/6 (T)	Students: Optimizing Attention (II)	PagedAttention (Kwon et al., 2023) (Slides) DeepSeek-V3 Technical Report (DeepSeek-AI et al., 2024) (Slides)
13	10/8 (Th)	Lecture: Monte Carlo methods & variance reduction (Slides)	Classic tutorial by Hammersley & Handscombe Barto & Sutton – Ch 5 Monte Carlo Methods for MDPS
14	10/13 (T)	Lecture: Markov Decision Processes (MDPs) (Slides)	Barto & Sutton – Ch 3&4 MDPs
15	10/15 (Th)	Lecture: Tabular methods (TD(0), TD(n), Q-learning, MC) (Slides)	Barto & Sutton – Ch 6 Temporal Difference Learning
16	10/20 (T)	Lecture: Policy gradient methods (I) (Slides)	Barto & Sutton – Ch 13: Policy Gradients OpenAI DRL – Intro to Policy Optimization Definitive Guide to Policy Gradients §1–2 (Matthias Lehmann, 2024) Deep RL: Pong from Pixels (Andrej Karpathy) Karpathy - Overview of Policy Gradients Video
17	10/22 (Th)	Lecture: Policy gradient methods (II): (Baseline slides) (TRPO/PPO slides)	Trust Region Policy Optimization (TRPO) (John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, Philipp Moritz, 2015) Proximal Policy Optimization (PPO) (John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov, 2017)
18	10/27 (T)	Students: Reinforcement Learning from Human Feedback (RLHF) (Slides)	Reinforcement Learning from Human Feedback: Progress and Challenges (John Schulman, 2023) Training language models to follow instructions with human feedback (Long Ouyang et al., 2022) Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback (Jan Brauner et al., 2023) RLTF: Reinforcement Learning from Unit Test Feedback (Liu et al., 2023)
19	10/29 (Th)	Students: RL for CUDA kernel generation: [CUDA-L1, CUDA-L2]	CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning (Li et al., 2025) CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning (Su et al., 2025)
20	11/3 (T)	Students: Post-training for sparse and Triton kernels: [SparseRL, AutoTriton]	SparseRL: Mastering Sparse CUDA Generation through Pretrained Models and Deep Reinforcement Learning (Wang et al., 2026) AutoTriton: Automatic Triton Programming with Reinforcement Learning in LLMs (Li et al., 2025)
21	11/5 (Th)	Students: Multi-turn RL with execution feedback: [Kevin, Dr. Kernel]	Kevin (Baronio et al., 2025) Dr. Kernel (Liu et al., 2026) Self-distillation Policy Optimization (SDPO) (Hübotter et al., 2026)
22	11/10 (T)	Students: Multi-agent refinement and agentic RL: [STARK, CUDA Agent]	STARK: Strategic Team of Agents for Refining Kernels (Dong et al., 2025) CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation (Dai et al., 2026)
23	11/12 (Th)	Lecture: Evolutionary Computation (Slides)	Neuroevolution (Risto Miikkulainen, 2022) Evolving neural networks through augmenting topologies (Kenneth O. Stanley, Risto Miikkulainen, 2002)	Project check-in (meeting required (11/13, 11/15)
24	11/17 (T)	Students: Evolutionary search, profiler-guided optimization, and optimization-memory reuse: [KernelFoundry, KernelPro, From Large to Small]	KernelFoundry: Hardware-aware Evolutionary GPU Kernel Optimization (Wiedemann et al., 2026) Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization (KernelPro) (Gai et al., 2026) From Large to Small: Transferring CUDA Optimization Expertise via Reasoning Graph (ReGraphT) (Gong et al., 2025)
25	11/19 (Th)	Students: Portable kernel optimization across accelerator architectures: [KernelEvolve, Autocomp]	KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Meta (Liao et al., 2025) Autocomp: A Powerful and Portable Code Optimizer for Tensor Accelerators (Hong et al., 2025)
		Thanksgiving break
26	12/1 (T)	Students: Self-improving kernel generation for emerging accelerators: [AccelOpt, NKI-Agent]	AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization (Zhang et al., 2025) NKI-Agent: Domain-Specific Fine-Tuning and Agentic Tool Use for Neuron Kernel Generation (Tang et al., 2026)
27	12/3 (Th)	Project presentations
28	12/8 (T)	Project presentations		Final project paper due (no extensions will be allowed, no exceptions)