• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Reward (Mis)design for Autonomous Driving.
W. Bradley Knox, Alessandro Allievi,
Holger Banzhaf, Felix Schmitt, and Peter Stone.
Artificial Intelligence,
316:103829, 2023.
Paper webpage
[PDF]696.3kB [postscript]5.6MB
This article considers the problem of diagnosing certain common errors in reward design. Its insights are also applicable to the design of cost functions and performance metrics more generally. To diagnose common errors, we develop 8 simple sanity checks for identifying flaws in reward functions. We survey research that is published in top-tier venues and focuses on reinforcement learning (RL) for autonomous driving (AD). Specifically, we closely examine the reported reward function in each publication and present these reward functions in a complete and standardized format in the appendix. Wherever we have sufficient information, we apply the 8 sanity checks to each surveyed reward function, revealing near-universal flaws in reward design for AD that might also exist pervasively across reward design for other tasks. Lastly, we explore promising directions that may aid the design of reward functions for AD in subsequent research, following a process of inquiry that can be adapted to other domains.
@article{AIJ23-KNOX,
title={Reward (Mis)design for Autonomous Driving},
author = {W. Bradley Knox and Alessandro Allievi and Holger Banzhaf and Felix Schmitt and Peter Stone},
journal = {Artificial Intelligence},
volume = {316},
pages = {103829},
year = {2023},
issn = {0004-3702},
doi = {https://doi.org/10.1016/j.artint.2022.103829},
url = {https://www.sciencedirect.com/science/article/pii/S0004370222001692},
abstract = {
This article considers the problem of diagnosing certain common errors in reward design. Its insights are also applicable to the design of cost functions and performance metrics more generally. To diagnose common errors, we develop 8 simple sanity checks for identifying flaws in reward functions. We survey research that is published in top-tier venues and focuses on reinforcement learning (RL) for autonomous driving (AD). Specifically, we closely examine the reported reward function in each publication and present these reward functions in a complete and standardized format in the appendix. Wherever we have sufficient information, we apply the 8 sanity checks to each surveyed reward function, revealing near-universal flaws in reward design for AD that might also exist pervasively across reward design for other tasks. Lastly, we explore promising directions that may aid the design of reward functions for AD in subsequent research, following a process of inquiry that can be adapted to other domains.
},
wwwnote={<a href="https://www.bradknox.net/reward-misdesign/">Paper webpage</a>},
}
Generated by bib2html.pl (written by Patrick Riley ) on Sun Apr 14, 2024 10:20:41