• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Event Tables for Efficient Experience Replay.
Varun Kompella, Thomas Walsh, Samuel
Barrett, Peter Wurman, and Peter Stone.
Transactions
on Machine Learning Research (TMLR), 2023.
Experience replay (ER) is a crucial component of many deep reinforcement learning (RL) systems. However, uniform sampling from an ER buffer can lead to slow convergence and unstable asymptotic behaviors. This paper introduces Stratified Sampling from Event Tables (SSET), which partitions an ER buffer into Event Tables, each capturing important subsequences of optimal behavior. We prove a theoretical advantage over the traditional monolithic buffer approach and combine SSET with an existing prioritized sampling strategy to further improve learning speed and stability. Empirical results in challenging MiniGrid domains, benchmark RL environments, and a high-fidelity car racing simulator demonstrate the advantages and versatility of SSET over existing ER buffer sampling approaches.
@article{TMLR23,
author="Varun Kompella and Thomas Walsh and Samuel Barrett and Peter Wurman and Peter Stone",
title="Event Tables for Efficient Experience Replay",
journal="Transactions on Machine Learning Research (TMLR)",
year="2023",
abstract={
Experience replay (ER) is a crucial component of many deep
reinforcement learning (RL) systems. However, uniform
sampling from an ER buffer can lead to slow convergence
and unstable asymptotic behaviors. This paper introduces
Stratified Sampling from Event Tables (SSET), which
partitions an ER buffer into Event Tables, each capturing
important subsequences of optimal behavior. We prove a
theoretical advantage over the traditional monolithic
buffer approach and combine SSET with an existing
prioritized sampling strategy to further improve learning
speed and stability. Empirical results in challenging
MiniGrid domains, benchmark RL environments, and a
high-fidelity car racing simulator demonstrate the
advantages and versatility of SSET over existing ER buffer
sampling approaches.
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Apr 17, 2024 18:42:45