Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data

Decebal Constantin Mocanu, Maria Torres Vega, Eric Eaton, Peter Stone, and Antonio Liotta. Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data. Technical Report arXiv e-Prints 1610.05555, arXiv, 2016.
Available online

Download

(unavailable)

Abstract

Conceived in the early 1990s, Experience Replay (ER) has been shown to be a successful mechanism to allow online learning algorithms to reuse past experiences. Traditionally, ER can be applied to all machine learning paradigms (i.e., unsupervised, supervised, and reinforcement learning). Recently, ER has contributed to improving the performance of deep reinforcement learning. Yet, its application to many practical settings is still limited by the memory requirements of ER, necessary to explicitly store previous observations. To remedy this issue, we explore a novel approach, Online Contrastive Divergence with Generative Replay (OCD_GR), which uses the generative capability of Restricted Boltzmann Machines (RBMs) instead of recorded past experiences. The RBM is trained online, and does not require the system to store any of the observed data points. We compare OCD_GR to ER on 9 real-world datasets, considering a worst-case scenario (data points arriving in sorted order) as well as a more realistic one (sequential random-order data points). Our results show that in 64.28\% of the cases OCD_GR outperforms ER and in the remaining 35.72\% it has an almost equal performance, while having a considerably reduced space complexity (i.e., memory usage) at a comparable time complexity.

BibTeX Entry

@TechReport{Decebal16,
  author={Decebal Constantin Mocanu and Maria Torres Vega and Eric Eaton and Peter Stone and Antonio Liotta},
  title={Online Contrastive Divergence with Generative Replay: Experience Replay without Storing Data},
  year = "2016", 
  month = "October", 
  institution = "arXiv", 
  number = "arXiv e-Prints 1610.05555",
  abstract="
    Conceived in the early 1990s, Experience Replay (ER) has been
    shown to be a successful mechanism to allow online learning
    algorithms to reuse past experiences. Traditionally, ER can be
    applied to all machine learning paradigms (i.e., unsupervised,
    supervised, and reinforcement learning). Recently, ER has
    contributed to improving the performance of deep reinforcement
    learning. Yet, its application to many practical settings is still
    limited by the memory requirements of ER, necessary to explicitly
    store previous observations. To remedy this issue, we explore a
    novel approach, Online Contrastive Divergence with Generative
    Replay (OCD_GR), which uses the generative capability of
    Restricted Boltzmann Machines (RBMs) instead of recorded past
    experiences. The RBM is trained online, and does not require the
    system to store any of the observed data points. We compare OCD_GR
    to ER on 9 real-world datasets, considering a worst-case scenario
    (data points arriving in sorted order) as well as a more realistic
    one (sequential random-order data points). Our results show that
    in 64.28\% of the cases OCD_GR outperforms ER and in the remaining
    35.72\% it has an almost equal performance, while having a
    considerably reduced space complexity (i.e., memory usage) at a
    comparable time complexity.",
  wwwnote={<a href="https://arxiv.org/abs/1610.05555">Available online</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Fri Sep 07, 2018 20:29:00