• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Learning Generalizable Manipulation Policies with Object-Centric 3D Representations.
Yifeng
Zhu, Zhenyu Jiang, Peter Stone, and Yuke
Zhu.
In Conference on Robot Learning (CoRL), November 2023.
We introduce GROOT, an imitation learning method for learning robust policieswith object-centric and 3D priors. GROOT builds policies that generalize beyondtheir initial training conditions for vision-based manipulation. It constructsobject-centric 3D representations that are robust toward background changes andcamera views and reason over these representations using a transformer-basedpolicy. Furthermore, we introduce a segmentation correspondence model that allowspolicies to generalize to new objects at test time. Through comprehensiveexperiments, we validate the robustness of GROOT policies against perceptualvariations in simulated and real-world environments. GROOT's performance excelsin generalization over background changes, camera viewpoint shifts, and thepresence of new object instances, whereas both state-of-the-art end-to-endlearning methods and object proposal-based approaches fall short. We alsoextensively evaluate GROOT policies on real robots, where we demonstrate theefficacy under very wild changes in setup.
@InProceedings{yifeng_zhu_CORL2023,
author = {Yifeng Zhu and Zhenyu Jiang and Peter Stone and Yuke Zhu},
title = {Learning Generalizable Manipulation Policies with Object-Centric 3D Representations},
booktitle = {Conference on Robot Learning (CoRL)},
year = {2023},
month = {November},
location = {Atlanta, United States},
abstract = {We introduce GROOT, an imitation learning method for learning robust policies
with object-centric and 3D priors. GROOT builds policies that generalize beyond
their initial training conditions for vision-based manipulation. It constructs
object-centric 3D representations that are robust toward background changes and
camera views and reason over these representations using a transformer-based
policy. Furthermore, we introduce a segmentation correspondence model that allows
policies to generalize to new objects at test time. Through comprehensive
experiments, we validate the robustness of GROOT policies against perceptual
variations in simulated and real-world environments. GROOT's performance excels
in generalization over background changes, camera viewpoint shifts, and the
presence of new object instances, whereas both state-of-the-art end-to-end
learning methods and object proposal-based approaches fall short. We also
extensively evaluate GROOT policies on real robots, where we demonstrate the
efficacy under very wild changes in setup.
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Apr 17, 2024 18:42:49