Lecture Introduction: Towards General-Purpose Robot Autonomy

• Building Machines That Learn and Think Like People. Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, Samuel J. Gershman (2016)
• Intelligence without Reason. Rodney Brooks (1991)

Lecture Overview of Robot Perception

• The Limits and Potentials of Deep Learning for Robotics. Niko Sünderhauf, Oliver Brock, Walter Scheirer, Raia Hadsell, Dieter Fox, Jürgen Leitner, Ben Upcroft, Pieter Abbeel, Wolfram Burgard, Michael Milford, Peter Corke (2018)
• A Sensorimotor Account of Vision and Visual Consciousness. Kevin O'Regan and Alva Noë (2001)

Object Detection

• Mask R-CNN. Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross Girshick (2017)
• You Only Look Once: Unified, Real-Time Object Detection. Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi (2015)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (2015)
• CornerNet: Detecting Objects as Paired Keypoints. Hei Law, Jia Deng (2018)

3D Data Processing

• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas (2017)
• Dynamic Graph CNN for Learning on Point Clouds. Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E. Sarma, Michael M. Bronstein, Justin M. Solomon (2018)

• PointCNN: Convolution On X-Transformed Points. Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, Baoquan Chen (2018)
• 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. Christopher Choy, JunYoung Gwak, Silvio Savarese (2019)

Tutorial Tutorial on physical simulations for robot learning

Representation Learning I: Priors

• Contrastive Learning of Structured World Models. Thomas Kipf, Elise van der Pol, Max Welling (2019)
• Implicit Neural Representations with Periodic Activation Functions. Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, Gordon Wetzstein (2020)

• Momentum Contrast for Unsupervised Visual Representation Learning. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick (2019)
• DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, Steven Lovegrove (2019)

Representation Learning II: Motions

• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation. Peter R. Florence, Lucas Manuelli, Russ Tedrake (2018)
• Unsupervised Learning of Object Keypoints for Perception and Control. Tejas Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, Volodymyr Mnih (2019)

• Learning Features by Watching Objects Move. Deepak Pathak, Ross Girshick, Piotr Dollar, Trevor Darrell, Bharath Hariharan (2016)
• Time-Contrastive Networks: Self-Supervised Learning from Video. Pierre Sermanet, Corey Lynch, Yevgen Chebotar, Jasmine Hsu, Eric Jang, Stefan Schaal, Sergey Levine (2017)

Multimodal Perception

• Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks. Michelle A. Lee, Yuke Zhu, Krishnan Srinivasan, Parth Shah, Silvio Savarese, Li Fei-Fei, Animesh Garg, Jeannette Bohg (2018)
• Audio-Visual Embodied Navigation. Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman (2020)

• More Than a Feeling: Learning to Grasp and Regrasp Using Vision and Touch. Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, Sergey Levine (2018)
• Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectories. Jaeyong Sung, Ian Lenz, Ashutosh Saxena (2015)

Recursive State Estimation

• Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors. Rico Jonschkowski, Divyam Rastogi, Oliver Brock (2018)
• Particle Filter Networks with Application to Visual Localization. Peter Karkus, David Hsu, Wee Sun Lee (2018)

• Differentiable Algorithm Networks for Composable Robot Learning. Peter Karkus, Xiao Ma, David Hsu, Leslie Pack Kaelbling, Wee Sun Lee, Tomas Lozano-Perez (2019)
• Backprop KF: Learning Discriminative Deterministic State Estimators. Tuomas Haarnoja, Anurag Ajay, Sergey Levine, Pieter Abbeel (2016)

Pose Estimation

• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, Dieter Fox (2017)
• Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation. He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, Leonidas J. Guibas (2019)

• Implicit 3D Orientation Learning for 6D Object Detection from RGB Images. Martin Sundermeyer, Zoltan-Csaba Marton, Maximilian Durner, Manuel Brucker, Rudolph Triebel (2019)
• DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion. Chen Wang, Danfei Xu, Yuke Zhu, Roberto Martín-Martín, Cewu Lu, Li Fei-Fei, Silvio Savarese (2019)

Tutorial Tutorial on the PyTorch deep learning framework

Visual Tracking

• DART: Dense Articulated Real-Time Tracking. Tanner Schmidt, Richard Newcombe, Dieter Fox (2015)
• Fully-Convolutional Siamese Networks for Object Tracking. Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, Philip H. S. Torr (2016)

• Fast Online Object Tracking and Segmentation: A Unifying Approach. Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, Philip H.S. Torr (2018)
• Learning to Track at 100 FPS with Deep Regression Networks. David Held, Sebastian Thrun, Silvio Savarese (2016)

Active Perception

• Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Tasks. Dinesh Jayaraman, Kristen Grauman (2017)
• Learning to Poke by Poking: Experiential Learning of Intuitive Physics. Pulkit Agrawal, Ashvin Nair, Pieter Abbeel, Jitendra Malik, Sergey Levine (2016)

• Interactive Perception: Leveraging Action in Perception and Perception in Action. Jeannette Bohg, Karol Hausman, Bharath Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, Gaurav Sukhatme (2016)
• The Curious Robot: Learning Visual Representations via Physical Interactions. Lerrel Pinto, Dhiraj Gandhi, Yuanfeng Han, Yong-Lae Park, Abhinav Gupta (2016)

Lecture Overview of Robot Decision Making

• Reinforcement Learning in Robotics: A Survey. Jens Kober, J. Andrew Bagnell, Jan Peters (2013)
• Recent Advances in Robot Learning from Demonstration. Harish Ravichandar, Athanasios S. Polydoros, Sonia Chernova, Aude Billard (2020)

Model-free Reinforcement Learning

• Trust Region Policy Optimization. John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, Pieter Abbeel (2015)
• Soft Actor-Critic Algorithms and Applications. Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, Sergey Levine (2018)

• Continuous Control with Deep Reinforcement Learning. Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra (2015)
• Proximal Policy Optimization Algorithms. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, Oleg Klimov (2017)

Model-based Reinforcement Learning

• Learning to Control a Low-Cost Manipulator using Data-Efficient Reinforcement Learning. Marc Deisenroth, Carl Edward Rasmussen, Dieter Fox (2011)
• Dream to Control: Learning Behaviors by Latent Imagination. Danijar Hafner, Timothy Lillicrap, Jimmy Ba, Mohammad Norouzi (2019)

• Benchmarking Model-Based Reinforcement Learning. Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, Jimmy Ba (2017)
• Deep Visual Foresight for Planning Robot Motion. Chelsea Finn, Sergey Levine (2016)

Imitation as Supervised Learning

• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. Stéphane Ross, Geoffrey Gordon, and Drew Bagnell (2010)
• Agile Autonomous Driving using End-to-End Deep Imitation Learning. Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos A. Theodorou, and Byron Boots (2017)

• Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation. Tianhao Zhang, Zoe McCarthy, Owen Jow, Dennis Lee, Xi Chen, Ken Goldberg, Pieter Abbeel (2017)
• Learning Latent Plans from Play. Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, Pierre Sermanet (2019)

Inverse Reinforcement Learning

• Apprenticeship Learning via Inverse Reinforcement Learning. Pieter Abbeel, Andrew Ng (2004)
• Maximum Entropy Inverse Reinforcement Learning. Brian Ziebart, Andrew Maas, Andrew Bagnell, Anind Dey (2008)

• Cooperative Inverse Reinforcement Learning. Dylan Hadfield-Menell, Stuart J. Russell, Pieter Abbeel, Anca Dragan (2016)
• Active Preference-Based Learning of Reward Functions. Dorsa Sadigh, Anca Dragan, Shankar Sastry, Sanjit Seshia (2017)

Adversarial Imitation Learning

• Generative Adversarial Imitation Learning. Jonathan Ho, Stefano Ermon (2016)
• A Divergence Minimization Perspective on Imitation Learning Methods. Seyed Kamyar Seyed Ghasemipour, Richard Semel, Shixiang Gu (2019)

• InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations. Yunzhu Li, Jiaming Song, Stefano Ermon (2017)
• Reinforcement and Imitation Learning for Diverse Visuomotor Skills. Yuke Zhu, Ziyu Wang, Josh Merel, Andrei Rusu, Tom Erez, Serkan Cabi, Saran Tunyasuvunakool, János Kramár, Raia Hadsell, Nando de Freitas, Nicolas Heess (2018)

Learning to Learn I: Meta-Learning

• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. Chelsea Finn, Pieter Abbeel, Sergey Levine (2017)
• Meta-Learning with Memory-Augmented Neural Networks. Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, Timothy Lillicrap (2016)

• Optimization as a Model for Few-Shot Learning. Sachin Ravi, Hugo Larochelle (2017)
• Reinforcement Learning, Fast and Slow. Matthew Botvinick, Sam Ritter, Jane X. Wang, Zeb Kurth-Nelson, Charles Blundell, Demis Hassabis (2019)

Learning to Learn II: Lifelong Learning

• RoboBrain: Large-Scale Knowledge Engine for Robots. Ashutosh Saxena, Ashesh Jain, Ozan Sener, Aditya Jami, Dipendra K. Misra, Hema S. Koppula (2014)
• Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions. Rui Wang, Joel Lehman, Jeff Clune, Kenneth Stanley (2019)

• Learning without Forgetting. Zhizhong Li, Derek Hoiem (2016)
• A Deep Hierarchical Approach to Lifelong Learning in Minecraft. Chen Tessler, Shahar Givony, Tom Zahavy, Daniel J. Mankowitz, Shie Mannor (2016)

Compositionality I: Hierarchy

• Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation. Tejas Kulkarni, Karthik Narasimhan, Ardavan Saeedi, Joshua Tenenbaum (2016)
• Neural Task Programming: Learning to Generalize Across Hierarchical Tasks. Danfei Xu, Suraj Nair, Yuke Zhu, Julian Gao, Animesh Garg, Li Fei-Fei, Silvio Savarese (2017)

• Robot Learning from Demonstration by Constructing Skill Trees. George Konidaris, Scott Kuindersma, Roderic Grupen, Andrew Barto (2011)
• The Option-Critic Architecture. Pierre-Luc Bacon, Jean Harb, Doina Precup (2016)

Compositionality II: Task and Motion

• Hierarchical Task and Motion Planning in the Now. Leslie Pack Kaelbling, Tomás Lozano-Pérez (2010)
• Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning. Marc Toussaint, Kelsey Allen, Kevin Smith, Joshua Tenenbaum (2018)

• Object-Centric Task and Motion Planning in Dynamic Environments. Toki Migimatsu, Jeannette Bohg (2019)
• PETLON: Planning Efficiently for Task-Level-Optimal Navigation. Shih-Yun Lo, Shiqi Zhang, Peter Stone (2018)

Causal Reasoning

• Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning. Mark Edmonds, Xiaojian Ma, Siyuan Qi, Yixin Zhu, Hongjing Lu, Song-Chun Zhu (2019)
• Learning Plannable Representations with Causal InfoGAN. Thanard Kurutach, Aviv Tamar, Ge Yang, Stuart Russell, Pieter Abbeel (2018)

• Learning Neural Causal Models from Unknown Interventions. Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Chris Pal, Yoshua Bengio (2019)
• Visual Causal Feature Learning. Krzysztof Chalupka, Pietro Perona, Frederick Eberhardt (2014)

Simulation-Reality Gap

• BayesSim: Adaptive Domain Randomization via Probabilistic Inference for Robotics Simulators. Fabio Ramos, Rafael Carvalhaes Possas, Dieter Fox (2019)
• Driving Policy Transfer via Modularity and Abstraction. Matthias Müller, Alexey Dosovitskiy, Bernard Ghanem, Vladlen Koltun (2018)

• Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks. Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, Konstantinos Bousmalis (2019)
• Meta-Sim: Learning to Generate Synthetic Datasets. Amlan Kar, Aayush Prakash, Ming-Yu Liu, Eric Cameracci, Justin Yuan, Matt Rusiniak, David Acuna, Antonio Torralba, Sanja Fidler (2019)

Data-driven Grasping

• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics. Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, Ken Goldberg (2017)
• Learning a Visuomotor Controller for Real World Robotic Grasping using Simulated Depth Images. Ulrich Viereck, Andreas ten Pas, Kate Saenko, Robert Platt (2017)

• Automatic Grasp Planning Using Shape Primitives. Andrew Miller, Steffen Knoop, Henrik Christensen, Peter Allen (2003)
• Data-Driven Grasp Synthesis - A Survey. Jeannette Bohg, Antonio Morales, Tamim Asfour (2013)

Building Robotic Systems

• QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation. Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, Sergey Levine (2018)
• Learning Agile and Dynamic Motor Skills for Legged Robots. Jemin Hwangbo, Joonho Lee, Alexey Dosovitskiy, Dario Bellicoso, Vassilios Tsounis, Vladlen Koltun, Marco Hutter (2019)

• TossingBot: Learning to Throw Arbitrary Objects with Residual Physics. Andy Zeng, Shuran Song, Johnny Lee, Alberto Rodriguez, Thomas Funkhouser (2019)
• Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems. Clemens Eppner, Sebastian Höfer, Rico Jonschkowski, Roberto Martíın-Martín, Arne Sieverling, Vincent Wall, Oliver Brock (2016)

Lecture Conclusion: Open Questions in Robot Learning

• A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms. Oliver Kroemer, Scott Niekum, George Konidaris (2019)