Lecture Introduction: Towards General-Purpose Robot Autonomy

• Building Machines That Learn and Think Like People. Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, Samuel J. Gershman (2016)
• Intelligence without Reason. Rodney Brooks (1991)

Lecture Overview of Robot Perception

• The Limits and Potentials of Deep Learning for Robotics. Niko Sünderhauf, Oliver Brock, Walter Scheirer, Raia Hadsell, Dieter Fox, Jürgen Leitner, Ben Upcroft, Pieter Abbeel, Wolfram Burgard, Michael Milford, Peter Corke (2018)
• A Sensorimotor Account of Vision and Visual Consciousness. Kevin O'Regan and Alva Noë (2001)

2D Object Detection

• Mask R-CNN. Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick (2017)
• You Only Look Once: Unified, Real-Time Object Detection. Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi (2015)

• CornerNet: Detecting Objects as Paired Keypoints. Hei Law, Jia Deng (2018)
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (2015)

3D Data Processing

• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas (2017)
• Dynamic Graph CNN for Learning on Point Clouds. Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, Justin M. Solomon (2018)

• PointCNN: Convolution On X-Transformed Points. Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, Baoquan Chen (2018)
• 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. Christopher Choy, JunYoung Gwak, Silvio Savarese (2019)

Tutorial Tutorial on the robosuite simulation framework for robot learning

Implicit Neural Representations

• Implicit Neural Representations with Periodic Activation Functions. Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein (2020)
• Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representations. Zhenyu Jiang, Yifeng Zhu, Maxwell Svetlik, Kuan Fang, Yuke Zhu (2021)

• DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, Steven Lovegrove (2019)
• Neural Sparse Voxel Fields. Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, Christian Theobalt (2020)

Attention Architectures

• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby (2020)
• End-to-End Object Detection with Transformers. Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko (2020)

• Learning Transferable Visual Models From Natural Language Supervision. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever (2021)
• Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions. Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, Ling Shao (2021)

Self-Supervised Visual Learning: Data

• Self-Supervised Learning of Pretext-Invariant Representations. Ishan Misra, Laurens van der Maaten (20)
• Exploring Simple Siamese Representation Learning. Xinlei Chen, Kaiming He (2020)

• A Simple Framework for Contrastive Learning of Visual Representations. Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton (2020)
• Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning. Supasorn Suwajanakorn, Noah Snavely, Jonathan Tompson, Mohammad Norouzi (2018)

Self-Supervised Visual Learning: Motion

• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation. Peter Florence, Lucas Manuelli, Russ Tedrake (2018)
• Unsupervised Learning of Object Keypoints for Perception and Control. Tejas Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, Volodymyr Mnih (2019)

• Contrastive Learning of Structured World Models. Thomas Kipf, Elise van der Pol, Max Welling (2019)
• Time-Contrastive Networks: Self-Supervised Learning from Video. Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine (2017)

Multimodal Perception

• Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks. Michelle A. Lee*, Yuke Zhu*, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg (2018)
• Audio-Visual Embodied Navigation. Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman (2019)

• Swoosh! Rattle! Thump! - Actions that Sound. Dhiraj Gandhi, Abhinav Gupta, Lerrel Pinto (2020)
• More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch. Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, Sergey Levine (2018)

Recursive State Estimation

• Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors. Rico Jonschkowski, Divyam Rastogi, Oliver Brock (2018)
• Particle Filter Networks with Application to Visual Localization. Peter Karkus, David Hsu, Wee Sun Lee (2018)

• Differentiable Algorithm Networks for Composable Robot Learning. Peter Karkus, Xiao Ma, David Hsu, Leslie Pack Kaelbling, Wee Sun Lee, Tomas Lozano-Perez (2019)
• Backprop KF: Learning Discriminative Deterministic State Estimators. Tuomas Haarnoja, Anurag Ajay, Sergey Levine, Pieter Abbeel (2016)

Tutorial Tutorial on the PyTorch deep learning framework

Interactive Perception

• Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks. Dinesh Jayaraman, Kristen Grauman (2017)
• Embodied Amodal Recognition: Learning to Move to Perceive Objects. Jianwei Yang, Zhile Ren, Mingze Xu, Xinlei Chen, David J. Crandall, Devi Parikh, Dhruv Batra (2019)

• Interactive Perception: Leveraging Action in Perception and Perception in Action. Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, Gaurav Sukhatme (2016)
• Learning to Poke by Poking: Experiential Learning of Intuitive Physics. Pulkit Agrawal, Ashvin Nair, Pieter Abbeel, Jitendra Malik, Sergey Levine (2016)

Synthetic Data for Robot Perception

• Meta-Sim: Learning to Generate Synthetic Datasets. Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler (2019)
• Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects. Jonathan Tremblay, Thang To, Balakumar Sundaralingam, Yu Xiang, Dieter Fox, Stan Birchfield (2018)

• Driving Policy Transfer via Modularity and Abstraction. Matthias Müller, Alexey Dosovitskiy, Bernard Ghanem, Vladlen Koltun (2018)
• Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding. Mike Roberts, Nathan Paczan (2020)

Lecture Overview of Robot Decision Making

• Reinforcement Learning in Robotics: A Survey. Jens Kober, J. Andrew Bagnell, Jan Peters (2013)
• Recent Advances in Robot Learning from Demonstration. Harish Ravichandar, Athanasios S. Polydoros, Sonia Chernova, Aude Billard (2020)

Model-free Reinforcement Learning

• Trust Region Policy Optimization. John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel (2015)
• Soft Actor-Critic Algorithms and Applications. Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, Sergey Levine (2018)

• Proximal Policy Optimization Algorithms. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov (2017)
• Continuous Control with Deep Reinforcement Learning. Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra (2015)

Model-based Reinforcement Learning

• Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning. Marc Deisenroth, Carl Edward Rasmussen, Dieter Fox (2012)
• Dream to Control: Learning Behaviors by Latent Imagination. Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi (2019)

• Benchmarking Model-Based Reinforcement Learning. Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, Jimmy Ba (2019)
• Deep Visual Foresight for Planning Robot Motion. Chelsea Finn, Sergey Levine (2016)

Batch (Offline) Reinforcement Learning

• Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skills. Yevgen Chebotar, Karol Hausman, Yao Lu, Ted Xiao, Dmitry Kalashnikov, Jake Varley, Alex Irpan, Benjamin Eysenbach, Ryan Julian, Chelsea Finn, Sergey Levine (2021)
• IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data. Ajay Mandlekar, Fabio Ramos, Byron Boots, Silvio Savarese, Li Fei-Fei, Animesh Garg, Dieter Fox (2019)

• Offline Learning from Demonstrations and Unlabeled Experience. Konrad Zolna, Alexander Novikov, Ksenia Konyushkova, Caglar Gulcehre, Ziyu Wang, Yusuf Aytar, Misha Denil, Nando de Freitas, Scott Reed (2021)
• An Optimistic Perspective on Offline Reinforcement Learning. Rishabh Agarwal, Dale Schuurmans, Mohammad Norouzi (2019)

Imitation as Supervised Learning

• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. Stéphane Ross, Geoffrey Gordon, and Drew Bagnell (2010)
• Agile Autonomous Driving using End-to-End Deep Imitation Learning. Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos A. Theodorou, and Byron Boots (2017)

• Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation. Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, Pieter Abbeel (2017)
• Learning Latent Plans from Play. Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, Pierre Sermanet (2019)

Inverse Reinforcement Learning

• Apprenticeship Learning via Inverse Reinforcement Learning. Pieter Abbeel, Andrew Ng (2004)
• Maximum Entropy Inverse Reinforcement Learning. Brian Ziebart, Andrew Maas, Andrew Bagnell, Anind Dey (2008)

• Active Preference-Based Learning of Reward Functions. Dorsa Sadigh, Anca Dragan, Shankar Sastry, Sanjit Seshia (2017)
• Cooperative Inverse Reinforcement Learning. Dylan Hadfield-Menell, Stuart J. Russell, Pieter Abbeel, Anca Dragan (2016)

Adversarial Imitation Learning

• Generative Adversarial Imitation Learning. Jonathan Ho, Stefano Ermon (2016)
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills. Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool, János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess (2018)

• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations. Yunzhu Li, Jiaming Song, Stefano Ermon (2017)
• A Divergence Minimization Perspective on Imitation Learning Methods. Seyed Kamyar Seyed Ghasemipour, Richard Semel, Shixiang Gu (2019)

Human-in-the-Loop Learning

• Deep TAMER: Interactive Agent Shaping in High-Dimensional State Spaces. Garrett Warnell, Nicholas Waytowich, Vernon Lawhern, Peter Stone (2017)
• HG-DAgger: Interactive Imitation Learning with Human Experts. Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, Mykel J. Kochenderfer (2018)

• Deep Reinforcement Learning from Human Preferences. Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, Dario Amodei (2017)
• Trial without Error: Towards Safe Reinforcement Learning via Human Intervention. William Saunders, Girish Sastry, Andreas Stuhlmueller, Owain Evans (2017)

Hierarchical Policy Learning

• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation. Tejas Kulkarni, Karthik Narasimhan, Ardavan Saeedi, Joshua Tenenbaum (2016)
• Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning. Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, Karol Hausman (2019)

• The Option-Critic Architecture. Pierre-Luc Bacon, Jean Harb, Doina Precup (2016)
• Robot Learning from Demonstration by Constructing Skill Trees. George Konidaris, Scott Kuindersma, Roderic Grupen, Andrew Barto (2011)

Task and Motion Planning

• Hierarchical Task and Motion Planning in the Now. Leslie Pack Kaelbling, Tomás Lozano-Pérez (2011)
• Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning. Marc Toussaint, Kelsey Allen, Kevin Smith, Joshua Tenenbaum (2014)

• Object-Centric Task and Motion Planning in Dynamic Environments. Toki Migimatsu, Jeannette Bohg (2019)
• Integrating Task-Motion Planning with Reinforcement Learning for Robust Decision Making in Mobile Robots. Yuqian Jiang, Fangkai Yang, Shiqi Zhang, Peter Stone (2018)

Neural Program Learning

• Neural Task Programming: Learning to Generalize Across Hierarchical Tasks. Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese (2017)
• Neural Program Synthesis from Diverse Demonstration Videos. Shao-Hua Sun, Hyeonwoo Noh, Sriram Somasundaram, Joseph J. Lim (2018)

• Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration. De-An Huang, Suraj Nair, Danfei Xu, Yuke Zhu, Animesh Garg, Li Fei-Fei, Silvio Savarese, Juan Carlos Niebles (2018)
• DreamCoder: Growing Generalizable, Interpretable Knowledge with Wake-Sleep Bayesian Program Learning. Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sable-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-Lezama, Joshua B. Tenenbaum (2020)

Guest Lecture: Jeff Mahler, Ambi Robotics

Guest Lecture: Adrien Gaidon, Toyota Research Institute

Lecture Conclusion: Open Questions in Robot Learning

• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. Oliver Kroemer, Scott Niekum, George Konidaris (2019)