UTCS Artificial Intelligence
courses
talks/events
demos
people
projects
publications
software/data
labs
areas
admin
On Learning with Imperfect Representations (2011)
Shivaram Kalyanakrishnan
and
Peter Stone
In this paper we present a perspective on the relationship between learning and representation in sequential decision making tasks. We undertake a brief survey of existing real-world applications, which demonstrates that the classical ``tabular'' representation seldom applies in practice. Specifically, several practical tasks suffer from state aliasing, and most demand some form of generalization and function approximation. Coping with these representational aspects thus becomes an important direction for furthering the advent of reinforcement learning in practice. The central thesis we present in this position paper is that in practice, learning methods specifically developed to work with imperfect representations are likely to perform better than those developed for perfect representations and then applied in imperfect-representation settings. We specify an evaluation criterion for learning methods in practice, and propose a framework for their synthesis. In particular, we highlight the degrees of ``representational bias'' prevalent in different learning methods. We reference a variety of relevant literature as a background for this introspective essay.
View:
PDF
,
PS
,
HTML
Citation:
In
Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning
, April 2011.
Bibtex:
@inproceedings{ADPRL11-shivaram, title={On Learning with Imperfect Representations}, author={Shivaram Kalyanakrishnan and Peter Stone}, booktitle={Proceedings of the 2011 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning}, month={April}, url="http://www.cs.utexas.edu/users/ai-lab?ADPRL11-shivaram", year={2011} }
People
Shivaram Kalyanakrishnan
Ph.D. Alumni
shivaram [at] cs utexas edu
Peter Stone
Faculty
pstone [at] cs utexas edu
Areas of Interest
Reinforcement Learning
Labs
Learning Agents