Effective robotic systems for long-horizon human-robot collaboration must adapt to a wide range of human partners, whose physical behavior, willingness to assist, and understanding of the robot's capabilities may change over time. This demands a tightly coupled communication loop that grants both agents the flexibility to propose, accept, or decline requests as they coordinate toward completing the task effectively. We propose MICoBot, a system that enables the human and robot, both using natural language, to take initiative in formulating, accepting, or rejecting proposals on who can best complete different steps of a task. To handle diverse, task-directed dialog, and find successful collaborative strategies that minimize human effort, MICoBot makes decisions at three levels: (1) a meta-planner considers human dialog to formulate and code a high-level collaboration strategy, (2) a planner optimally allocates the remaining steps to either agent based on the robot's capabilities (measured by a simulation-pretrained affordance model) and the estimated human's willingness to help, and (3) an action executor decides the low-level actions to perform or words to say to the human. In physical robot trials with 18 unique human participants, MICoBot significantly improves task success and user experience over a pure LLM baseline and standard agent allocation models. See additional videos and materials at our project site: https://robin-lab.cs.utexas.edu/MicoBot/.

PDF, Arxiv

International Conference on Robotics and Automation (ICRA) (2026).

@article{yu:icra26, title={Mixed-Initiative Dialog for Human-Robot Collaborative Manipulation}, author={Albert Yu and Chengshu Li and Luca Macesanu and Arnav Balaji and Ruchira Ray and Raymond Mooney and Roberto Martín-Martín}, journal={International Conference on Robotics and Automation (ICRA)}, month={June}, url="http://www.cs.utexas.edu/users/ai-labpub-view.php?PubID=128128", year={2026} }