next up previous
Next: Storing to Memory Up: Learning Method Previous: Experimental Setup

Memory Model

Storing every individual experience in memory would be inefficient both in terms of amount of memory required and in terms of generalization time. Therefore, we store tex2html_wrap_inline280 and tex2html_wrap_inline282 only at discrete, evenly-spaced values of tex2html_wrap_inline252 . That is, for a memory of size M (with M dividing evenly into 360 for simplicity), we keep values of tex2html_wrap_inline506 and tex2html_wrap_inline508 for tex2html_wrap_inline510 . We store memory as an array ``Mem'' of size M such that Mem[n] has values for both tex2html_wrap_inline300 and tex2html_wrap_inline302 . Using a fixed memory size precludes using memory-based techniques such as K-Nearest-Neighbors (kNN) and kernel regression which require that every experience be stored, choosing the most relevant only at decision time. Most of our experiments were conducted with memories of size 360 (low generalization) or of size 18 (high generalization), i.e. M = 18 or M = 360. As will be seen from our results, the memory size had a large effect on the rate of learning.

Peter Stone
Mon Dec 11 15:42:40 EST 1995