• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Large-Language-Model-Guided State Estimation for Partially Observable Task and Motion Planning.
Yoonwoo Kim, Raghav
Arora, Roberto Martín-Martín, Peter Stone, Ben Abbatematteo, and Yoonchang
Sung.
In International Conference on Robotics and Automation, June 2026.
[PDF]2.9MB [slides.pdf]1.6MB [poster.pdf]1.3MB
Robot planning in partially observable environments, where not all objects are known or visible, is a challenging problem, as it requires reasoning under uncertainty through partially observable Markov decision processes. During the execution of a computed plan, a robot may unexpectedly observe task-irrelevant objects, which are typically ignored by naive planners. In this work, we propose incorporating two types of common-sense knowledge: (1) certain objects are more likely to be found in specific locations; and (2) similar objects are likely to be co-located, while dissimilar objects are less likely to be found together. Manually engineering such knowledge is complex, so we explore leveraging the powerful common-sense reasoning capabilities of large language models (LLMs). Our planning and execution framework, CoCo-TAMP, introduces a hierarchical state estimation that uses LLM-guided information to shape the belief over task-relevant objects, enabling efficient solutions to long-horizon task and motion planning problems. In experiments, CoCo-TAMP achieves an average reduction of 62.7 percent in planning and execution time in simulation, and 72.6 percent in real-world demonstrations, compared to a baseline that does not incorporate either type of common-sense knowledge.
@InProceedings{yoonwoo_kim_ICRA2026,
author = {Yoonwoo Kim and Raghav Arora and Roberto MartÃn-MartÃn and Peter Stone and Ben Abbatematteo and Yoonchang Sung},
title = {Large-Language-Model-Guided State Estimation for Partially Observable Task and Motion Planning},
booktitle = {International Conference on Robotics and Automation},
year = {2026},
month = {June},
location = {Vienna, Austria},
abstract = {Robot planning in partially observable environments, where not all objects are known or visible, is a challenging problem, as it requires reasoning under uncertainty through partially observable Markov decision processes. During the execution of a computed plan, a robot may unexpectedly observe task-irrelevant objects, which are typically ignored by naive planners. In this work, we propose incorporating two types of common-sense knowledge: (1) certain objects are more likely to be found in specific locations; and (2) similar objects are likely to be co-located, while dissimilar objects are less likely to be found together. Manually engineering such knowledge is complex, so we explore leveraging the powerful common-sense reasoning capabilities of large language models (LLMs). Our planning and execution framework, CoCo-TAMP, introduces a hierarchical state estimation that uses LLM-guided information to shape the belief over task-relevant objects, enabling efficient solutions to long-horizon task and motion planning problems. In experiments, CoCo-TAMP achieves an average reduction of 62.7 percent in planning and execution time in simulation, and 72.6 percent in real-world demonstrations, compared to a baseline that does not incorporate either type of common-sense knowledge.},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Jun 10, 2026 15:26:44