We introduce the first algorithm for learning predictive state representations PSRs, which are a way of representing the state of a controlled dynamical system. The state representation in a PSR is a vector of predictions of tests, where tests are sequences of actions and observations said to be true if and only if all the observations occur given that all the actions are taken. The problem of finding a good PSR---one that is a sufficient statistic for the dynamical system---can be divided into two parts: 1) discovery of a good set of tests, and 2) learning to make accurate predictions for those tests. In this paper, we present detailed empirical results using a gradient-based algorithm for addressing the second problem. Our results demonstrate several sample systems in which the algorithm learns to make correct predictions and several situations in which the algorithm is less successful. Our analysis reveals challenges that will need to be addressed in future PSR learning algorithms.

PDF, PS, HTML

In Proceedings of the Twentieth International Conference on Machine Learning, August 2003.

@InProceedings{singh:icml03, title={Learning Predictive State Representations}, author={Satinder Singh and Michael L. Littman and Nicholas K. Jong and David Pardoe and Peter Stone}, booktitle={Proceedings of the Twentieth International Conference on Machine Learning}, month={August}, url="http://www.cs.utexas.edu/users/ai-lab?singh:icml03", year={2003} }