• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
Deterministic Implementations for Reproducibility in Deep Reinforcement Learning.
Prabhat Nagarajan, Garrett
Warnell, and Peter Stone.
In 2nd Reproducibility in Machine Learning
Workshop at ICML 2018, July 2018.
While deep reinforcement learning (DRL) has led to numerous successes in recentyears, reproducing these successes can be extremely challenging. Onereproducibility challenge particularly relevant to DRL is nondeterminism in thetraining process, which can substantially affect the results. Motivated by thischallenge, we study the positive impacts of deterministic implementations ineliminating nondeterminism in training. To do so, we consider the particularcase of the deep Q-learning algorithm, for which we produce a deterministicimplementation by identifying and controlling all sources of nondeterminism inthe training process. One by one, we then allow individual sources ofnondeterminism to affect our otherwise deterministic implementation, andmeasure the impact of each source on the variance in performance. We find thatindividual sources of nondeterminism can substantially impact the performanceof agent, illustrating the benefits of deterministic implementations. Inaddition, we also discuss the important role of deterministic implementationsin achieving exact replicability of results.
@InProceedings{RML18-nagarajan,
author = {Prabhat Nagarajan and Garrett Warnell and Peter Stone},
title = {Deterministic Implementations for Reproducibility in Deep Reinforcement Learning},
booktitle = {2nd Reproducibility in Machine Learning Workshop at ICML 2018},
location = {Stockholm, Sweden},
month = {July},
year = {2018},
abstract = {
While deep reinforcement learning (DRL) has led to numerous successes in recent
years, reproducing these successes can be extremely challenging. One
reproducibility challenge particularly relevant to DRL is nondeterminism in the
training process, which can substantially affect the results. Motivated by this
challenge, we study the positive impacts of deterministic implementations in
eliminating nondeterminism in training. To do so, we consider the particular
case of the deep Q-learning algorithm, for which we produce a deterministic
implementation by identifying and controlling all sources of nondeterminism in
the training process. One by one, we then allow individual sources of
nondeterminism to affect our otherwise deterministic implementation, and
measure the impact of each source on the variance in performance. We find that
individual sources of nondeterminism can substantially impact the performance
of agent, illustrating the benefits of deterministic implementations. In
addition, we also discuss the important role of deterministic implementations
in achieving exact replicability of results.
},
}
Generated by bib2html.pl (written by Patrick Riley ) on Wed Apr 17, 2024 18:42:58