• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •

Daniel Urieli and Peter Stone.
**Model-Selection for Non-Parametric Function Approximation in Continuous Control Problems: A Case Study in a Smart Energy
System**. In *Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML'13)*,
Sep 2013.

Official publisher version

[PDF]2.3MB [postscript]1.4MB [slides.pdf]3.6MB

This paper investigates the application of value-function-based reinforcement learning to a smart energy control system, specifically the task of controlling an HVAC system to minimize energy while satisfying residents' comfort requirements. In theory, value-function-based reinforcement learning methods can solve control problems such as this one optimally. However, since choosing an appropriate parametric representation of the value function turns out to be difficult, we develop an alternative method, which results in a practical algorithm for value function approximation in continuous state-spaces. To avoid the need to carefully design a parametric representation for the value function, we use a smooth non-parametric function approximator, specifically Locally Weighted Linear Regression (LWR). LWR is used within Fitted Value Iteration (FVI), which has met with several practical successes. However, for efficiency reasons, LWR is used with a limited sample-size, which leads to poor performance without careful tuning of LWR's parameters. We therefore develop an efficient meta-learning procedure that performs online model-selection and tunes LWR's parameters based on the Bellman error. Our algorithm is fully implemented and tested in a realistic simulation of the HVAC control domain, and results in significant energy savings.

@InProceedings{ECML13-urieli, author = {Daniel Urieli and Peter Stone}, title = {Model-Selection for Non-Parametric Function Approximation in Continuous Control Problems: A Case Study in a Smart Energy System}, booktitle = {Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML'13)}, location = {Prague, Czech Republic}, month = {Sep}, year = {2013}, abstract = " This paper investigates the application of value-function-based reinforcement learning to a smart energy control system, specifically the task of controlling an HVAC system to minimize energy while satisfying residents' comfort requirements. In theory, value-function-based reinforcement learning methods can solve control problems such as this one optimally. However, since choosing an appropriate parametric representation of the value function turns out to be difficult, we develop an alternative method, which results in a practical algorithm for value function approximation in continuous state-spaces. To avoid the need to carefully design a parametric representation for the value function, we use a smooth non-parametric function approximator, specifically Locally Weighted Linear Regression (LWR). LWR is used within Fitted Value Iteration (FVI), which has met with several practical successes. However, for efficiency reasons, LWR is used with a limited sample-size, which leads to poor performance without careful tuning of LWR's parameters. We therefore develop an efficient meta-learning procedure that performs online model-selection and tunes LWR's parameters based on the Bellman error. Our algorithm is fully implemented and tested in a realistic simulation of the HVAC control domain, and results in significant energy savings. ", wwwnote={Official <a href="http://link.springer.com/chapter/10.1007/978-3-642-40988-2_5">publisher version</a>}, }

Generated by bib2html.pl (written by Patrick Riley ) on Sat Nov 28, 2015 21:47:25