• Classified by Topic • Classified by Publication Type • Sorted by Date • Sorted by First Author Last Name • Classified by Funding Source •
State Abstraction Discovery from Irrelevant State Variables.
Nicholas
K. Jong and Peter Stone.
In Proceedings of the Nineteenth International
Joint Conference on Artificial Intelligence, pp. 752–757, August 2005.
[PDF]102.1kB [postscript]249.8kB [slides.pdf]323.4kB
Intelligent agents can cope with complexity by abstracting away irrelevant details. Prior work in abstraction typically relied on users to provide abstractions manually. We propose an alternative approach built around the idea of policy irrelevance, in which we abstract away a feature if it is possible to behave optimally without it. We show how this approach permits statistical inference methods to discover policy irrelevant state variables, and we give two such methods that optimize respectively for computational complexity and sample complexity. However, although policy irrelevance guarantees the ability to represent an optimal policy, a naive application possibly loses the ability to learn an optimal policy. We demonstrate that encapsulating a state abstraction inside an abstract action preserves optimality while offering the possibility of learning more efficiently. Finally, since our approach currently discovers abstractions post hoc, we show how abstractions discovered in an easy problem can help learn difficult related problems.
@InProceedings(IJCAI05,
author="Nicholas K.\ Jong and Peter Stone",
title="State Abstraction Discovery from Irrelevant State Variables",
booktitle="Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence",
pages="752--757",
month="August",year="2005",
abstract={
Intelligent agents can cope with complexity by
abstracting away irrelevant details. Prior work in
abstraction typically relied on users to provide
abstractions manually. We propose an alternative
approach built around the idea of \emph{policy
irrelevance}, in which we abstract away a feature if
it is possible to behave optimally without it. We
show how this approach permits statistical inference
methods to discover policy irrelevant state
variables, and we give two such methods that optimize
respectively for computational complexity and sample
complexity. However, although policy irrelevance
guarantees the ability to represent an optimal
policy, a naive application possibly loses the
ability to learn an optimal policy. We demonstrate
that encapsulating a state abstraction inside an
abstract action preserves optimality while offering
the possibility of learning more efficiently.
Finally, since our approach currently discovers
abstractions post hoc, we show how abstractions
discovered in an easy problem can help learn
difficult related problems.
},
)
Generated by bib2html.pl (written by Patrick Riley ) on Wed Apr 17, 2024 18:42:57