Dissertation - Data Efficient Reinforcement Learning

Abstract

In recent years, there has been growing interest in building general-purpose personal robots, driven by the promise that the robots can assist people with a large variety of everyday manipulation tasks. Such robots must adapt their skills to a wide range of completely new scenarios. This dissertation considers Open-world Robot Manipulation, a manipulation problem where a robot must generalize or quickly adapt to new objects, scenes, or tasks for which it has not been pre-programmed or pre-trained. This dissertation tackles the problem using a methodology of efficient sensorimotor learning. The key to enabling efficient sensorimotor learning lies in leveraging regular patterns that exist in limited amounts of demonstration data. These patterns, referred to as ``regularity,'' enable the data-efficient learning of generalizable manipulation skills. This dissertation offers a new perspective on formulating manipulation problems through the lens of regularity. Building upon this notion, we introduce three major contributions. First, we introduce methods that endow robots with object-centric priors, allowing them to learn generalizable, closed-loop sensorimotor policies from a small number of teleoperation demonstrations. Second, we introduce methods that constitute robots' spatial understanding, unlocking their ability to imitate manipulation skills from in-the-wild video observations. Last but not least, we introduce methods that enable robots to identify reusable skills from their past experiences, resulting in systems that can continually imitate multiple tasks in a sequential manner. Altogether, the contributions of this dissertation help lay the groundwork for building general-purpose personal robots that can quickly adapt to new situations or tasks with low-cost data collection and interact easily with humans. By enabling robots to learn and generalize from limited data, this dissertation takes a step toward realizing the vision of intelligent robotic assistants that can be seamlessly integrated into everyday scenarios.

This dissertation studies the problem of Open-world Robot Manipulation with a methodology of data-efficient learning. By developing methods to enable efficient sensorimotor learning, we aim to answer the following research question: "How can robots exploit regularities in the physical world to efficiently learn generalizable manipulation policies?"

Defense Presentation

Related Projects

Future Work

Human-Robot Coevolution

This dissertation explores how robots can leverage regularities to enable efficient sensorimotor learning for Open-world Robot Manipulation, working towards autonomous robots that can be deployed in real-world environments. Building on this foundation, the work introduces Human-Robot Coevolution as a new research concept that focuses on how humans and robots will develop together in shared environments, creating a mutually influential relationship. This interdisciplinary field moves beyond the goal of replacing humans with robots, instead emphasizing collaboration where robots serve as intelligent assistants and companions. Inspired by how technologies like writing systems, telecommunications, and computers have transformed human societies, this approach recognizes that robots will fundamentally reshape human behaviors, cognitive processes, and social structures. The concept builds upon open-world robot manipulation research while incorporating broader aspects including robotics, AI, psychology, sociology, ethics, and design to create systems that understand and complement human needs, preferences, and capabilities.

Long-term Robot Autonomy

Long-term robot autonomy research aims to develop robots capable of continuous operation in everyday environments for extended periods—hours, days, or even 24/7—while manipulating diverse objects encountered in daily settings. By developing robots capable of persistent operation and diverse manipulation tasks across building-wide spaces, we can understand better what it takes to build robots that co-exist with us harmoniously. Such a sustained presence of robots allows for meaningful investigation of how robots influence human behavior and society while simultaneously evolving to better understand human needs, preferences, and social dynamics.

Personalized Interactive Robots

Research of building personalized interactive robots focuses on how to build robots that adapt continuously to individual user preferences, representing the robot-to-human influence in human-robot coevolution. By learning throughout their deployment to customize behaviors and safely unlearn undesirable skills, these robots create satisfying interactions that facilitate mutual adaptation between humans and machines—a key dynamic in the coevolutionary relationship where robots shape human experiences while humans simultaneously guide robot development.

Downloads

Full Dissertation (PDF) Download Slides (PPTX)

Contact

For questions or comments about this dissertation, please contact:

Email: yifeng.zhu@utexas.edu