UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors (2022)
Yifeng Zhu, Abhishek Joshi,
Peter Stone
, and Yuke Zhu
We introduce VIOLA, an object-centric imitation learning approach to learning closed-loop visuomotor policies for robot manipulation. Our approach constructs object-centric representations based on general object proposals from a pre-trained vision model. VIOLA uses a transformer-based policy to reason over these representations and attend to the task-relevant visual factors for action prediction. Such object-based structural priors improve deep imitation learning algorithm's robustness against object variations and environmental perturbations. We quantitatively evaluate VIOLA in simulation and on real robots. VIOLA outperforms the state-of-the-art imitation learning methods by 45.8 percents in success rate. It has also been deployed successfully on a physical robot to solve challenging long-horizon tasks, such as dining table arrangement and coffee making.
View:
PDF
Citation:
In
Proceedings of the 6th Conference on Robot Learning (CoRL 2022)
, Auckland, New Zealand, January 2022.
Bibtex:
@inproceedings{corl2022-zhu, title={VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors}, author={Yifeng Zhu and Abhishek Joshi and Peter Stone and Yuke Zhu}, booktitle={Proceedings of the 6th Conference on Robot Learning (CoRL 2022)}, month={January}, address={Auckland, New Zealand}, url="http://www.cs.utexas.edu/users/ai-lab?corl2022-zhu", year={2022} }
People
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Machine Learning
Reinforcement Learning
Robotics
Labs
Learning Agents