Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


State Abstraction Discovery from Irrelevant State Variables

Nicholas K. Jong and Peter Stone. State Abstraction Discovery from Irrelevant State Variables. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 752–757, August 2005.

Download

[PDF]102.1kB  [postscript]249.8kB  

Abstract

Intelligent agents can cope with complexity by abstracting away irrelevant details. Prior work in abstraction typically relied on users to provide abstractions manually. We propose an alternative approach built around the idea of policy irrelevance, in which we abstract away a feature if it is possible to behave optimally without it. We show how this approach permits statistical inference methods to discover policy irrelevant state variables, and we give two such methods that optimize respectively for computational complexity and sample complexity. However, although policy irrelevance guarantees the ability to represent an optimal policy, a naive application possibly loses the ability to learn an optimal policy. We demonstrate that encapsulating a state abstraction inside an abstract action preserves optimality while offering the possibility of learning more efficiently. Finally, since our approach currently discovers abstractions post hoc, we show how abstractions discovered in an easy problem can help learn difficult related problems.

BibTeX Entry

@InProceedings(IJCAI05,
        author="Nicholas K.\ Jong and Peter Stone",
        title="State Abstraction Discovery from Irrelevant State Variables",
        booktitle="Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence",
	   pages="752--757",
        month="August",year="2005",
       abstract={
                 Intelligent agents can cope with complexity by
                 abstracting away irrelevant details.  Prior work in
                 abstraction typically relied on users to provide
                 abstractions manually.  We propose an alternative
                 approach built around the idea of \emph{policy
                 irrelevance}, in which we abstract away a feature if
                 it is possible to behave optimally without it.  We
                 show how this approach permits statistical inference
                 methods to discover policy irrelevant state
                 variables, and we give two such methods that optimize
                 respectively for computational complexity and sample
                 complexity.  However, although policy irrelevance
                 guarantees the ability to represent an optimal
                 policy, a naive application possibly loses the
                 ability to learn an optimal policy.  We demonstrate
                 that encapsulating a state abstraction inside an
                 abstract action preserves optimality while offering
                 the possibility of learning more efficiently.
                 Finally, since our approach currently discovers
                 abstractions post hoc, we show how abstractions
                 discovered in an easy problem can help learn
                 difficult related problems.
       },
)

Generated by bib2html.pl (written by Patrick Riley ) on Mon Sep 22, 2014 22:47:23