UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
Online Kernel Selection for Bayesian Reinforcement Learning (2008)
Joseph Reisinger
,
Peter Stone
, and
Risto Miikkulainen
Kernel-based Bayesian methods for Reinforcement Learning (RL) such as Gaussian Process Temporal Difference (GPTD) are particularly promising because they rigorously treat uncertainty in the value function and make it easy to specify prior knowledge. However, the choice of prior distribution significantly affects the empirical performance of the learning agent, and little work has been done extending existing methods for prior model selection to the online setting. This paper develops Replacing-Kernel RL, an online model selection method for GPTD using sequential Monte-Carlo methods. Replacing-Kernel RL is compared to standard GPTD and tile-coding on several RL domains, and is shown to yield significantly better asymptotic performance for many different kernel families. Furthermore, the resulting kernels capture an intuitively useful notion of prior state covariance that may nevertheless be difficult to capture manually.
View:
PDF
,
PS
,
HTML
Citation:
In
Proceedings of the Twenty-Fifth International Conference on Machine Learning
, July 2008.
Bibtex:
@InProceedings{ICML08-reisinger, title={Online Kernel Selection for Bayesian Reinforcement Learning}, author={Joseph Reisinger and Peter Stone and Risto Miikkulainen}, booktitle={Proceedings of the Twenty-Fifth International Conference on Machine Learning}, month={July}, url="http://www.cs.utexas.edu/users/ai-lab?reisinger:icml08", year={2008} }
People
Risto Miikkulainen
Faculty
risto [at] cs utexas edu
Joseph Reisinger
Formerly affiliated Ph.D. Student
joeraii [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Machine Learning
Planning
Reinforcement Learning
Labs
Neural Networks
Learning Agents