CS 395T: Intelligent Robotics


For robots to be intelligent in the way people are intelligent, they will have to learn about their world, and their own ability to interact with it, much like people do. This research seminar will investigate new research directions in robot learning.

Traditionally, robots have been useful in manufacturing by moving blindly but precisely in totally controlled workcells. Traditionally, symbolic AI systems have given the appearance of intelligence by applying logical inference algorithms to symbol structures whose primitive elements are specified by human programmers. This has left AI systems open to Searle's famous "Chinese Room" critique, arguing that they only mimic intelligence: they are merely "faking it".

To answer this philosophical challenge, and to be useful in a host of real-world application on Earth and in space, AI systems need to be robots, with sensors and effectors embedded in the physical world. Not only that, but these robots must learn the nature of their own sensorimotor interaction with the environment, and must create their own symbols, grounded in their own experience.

Robots are being created with ever more complex and richly structured sensors. The sensorimotor system evolves over time, sometimes deteriorating, but sometimes being augmented with new "plug-and-play" sensors. Humans are astonishingly adaptable to sensorimotor changes, and children do an amazing job of learning to use their sensors and effectors in a few short years after birth. We can learn important things about robots from research on children. And robot models may help us create better theories of child development.

Our main focus is on robot learning of the "foundational domains" that underlie commonsense knowledge: space, time, actions, objects, properties and affordances, causality, and so on. The question is how these higher-level ontologies can be learned from experience with low-level sensor and motor interactions with the world (the "pixel level").

Perhaps surprisingly, this has led us to certain questions about the nature of consciousness, what its properties are, what those properties tell us about the architecture of the mind, and how it is involved in learning these foundational domains. We will read books and papers from a wide range of disciplines, attempting to see the underlying reality that they are describing from different points of view.


This is a research seminar, intended first to bring you to the state of the art, and then to help you do a project and paper of publishable quality. There will be a significant amount of reading and discussion of recent research papers that will be handed out.

The requirements of the course will be:

Class Presentations

Each class member will select a topic and present the material to the class. Each topic will have an associated reading that the entire class will read, but the presenter is responsible for finding and reading additional material, becoming an expert in the area, creating an illuminating example to present, and leading a discussion.

Pick a presentation topic that works well with your term project topic. The papers will be accessible online through the UT Library, or via link here. In some cases, you will need to review several related papers by the authors.

Be prepared to give a 45 minute presentation, followed by specific questions and more general discussion of the value and importance of the material presented.

Here is a thematic outline. You don't need to cover the points in exactly this order, but try to address these needs for your audience.

Prepare PowerPoint slides for your presentation. Send me a copy of your slides two or three days before your presentation, and I will give you feedback as quickly as I can. Make copies of your slides to hand out to the class before your presentation.

Presentation Topics

Each student will pick one of the available sub-bullets, and will be responsible for presenting and discussing that topic. The related paper (or papers) will be provided soon.

Term Projects

Each class member will do a term project. You can apply a method we are learning about to a robot learning problem. Or you can extend an existing method or develop a new method to solve a problem. Ideally, your term project will extend the state of the art, and will be suitable for submission to AAAI, ICRA, IROS or some other major conference.

You are encouraged to select a topic that fits well with your other research interests.

Possible Project Topics

The following is an incomplete list of project topic suggestions. More can be added, and you can propose ideas of your own. In several cases, you will see several projects that are closely related, and might build on each other. People working on those projects should consider coordinating their efforts.


Our own work on these problems starts with [Pierce and Kuipers, AIJ, 1997], which learns the foundations for the Spatial Semantic Hierarchy [Kuipers, AIJ, 2000]. It would be helpful to read these in advance.

The following books are valuable reading for the course.

The following books have valuable insights related to this course, and are well worth reading.

Valuable books for your library

The following are some useful books that you should have in your professional library, and that are related to this course. I will assume that you have immediate access to material in these books. If you do not already have a background in Artificial Intelligence, the following excellent textbook would be another valuable addition to your library, and is undoubtedly available used. Some assignment and project may be best done in a high-level programming environment such as R, MATLAB, or LabVIEW. Make sure you have any documentation you need.

The Computer Science Department has a Code of Conduct that describes the obligations of faculty and students. Read it at http://www.cs.utexas.edu/users/ear/CodeOfConduct.html.