UT ML Group: Publication: Advice-Taking Learners

Adaptive systems learn in dynamic environments by repeatedly sensing the world, performing an action, and receiving feedback from the environment. The area of reinforcement learning concerns agents that learn sequential behaviors from experience; however, learning in complex domains is excruciatingly slow. We are developing reinforcement learning methods that can be guided both by reinforcements provided by the environment and abstract advice provided by a human teacher. In particular, we are developing methods in which advice is given in ordinary natural language (which is translated into formal advice using a learned semantic parser). By taking advantage of general advice on actions to perform in certain situations, the agent's learning rate can be greatly accelerated. This work is related to our work on theory refinement and natural language learning.

Learning from natural-language advice and reinforcements is the topic of the PILLAR research project.

Publications

  1. Learning a Compositional Semantic Parser using an Existing Syntactic Parser [Abstract] [PDF] [Slide]
    Ruifang Ge and Raymond J. Mooney
    In the Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (ACL-IJCNLP 2009), pp. 611--619, Suntec, Singapore, August 2009.

  2. Learning for Semantic Parsing with Kernels under Various Forms of Supervision [Abstract] [PDF]
    Rohit J. Kate
    Ph.D. Thesis, Department of Computer Sciences, University of Texas at Austin, August 2007.
    159 pages.

  3. Learning for Semantic Parsing and Natural Language Generation Using Statistical Machine Translation Techniques [Abstract] [PDF]
    Yuk Wah Wong
    Ph.D. Thesis, Department of Computer Sciences, University of Texas at Austin, August 2007.
    188 pages.
    Also appears as Technical Report AI07-343, Artificial Intelligence Lab, University of Texas at Austin, August 2007.

  4. Learning Language Semantics from Ambiguous Supervision [Abstract] [PDF]
    Rohit J. Kate and Raymond J. Mooney
    In Proceedings of the 22nd AAAI Conference on Artificial Intelligence (AAAI-2007), Vancouver, BC, pp. 895-900, July 2007.

  5. Learning Synchronous Grammars for Semantic Parsing with Lambda Calculus [Abstract] [PDF]
    Yuk Wah Wong and Raymond J. Mooney
    Best Paper Award
    In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL-2007), pp. 960-967, Prague, Czech Republic, June 2007.

  6. Semi-Supervised Learning for Semantic Parsing using Support Vector Machines [Abstract] [PDF]
    Rohit J. Kate and Raymond J. Mooney
    In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, Short Papers (NAACL/HLT-2007), pp. 81-84, Rochester, NY, April 2007.

  7. Generation by Inverting a Semantic Parser That Uses Statistical Machine Translation [Abstract] [PDF]
    Yuk Wah Wong and Raymond J. Mooney
    In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (NAACL/HLT-2007), pp. 172-179, Rochester, NY, April 2007.

  8. Learning for Semantic Parsing [Abstract] [PDF]
    Raymond J. Mooney
    Computational Linguistics and Intelligent Text Processing: Proceedings of the 8th International Conference, CICLing 2007, Mexico City (invited paper), A. Gelbukh (Ed.), pp. 311-324, Springer, Berlin, Germany, February 2007.

  9. Using String-Kernels for Learning Semantic Parsers [Abstract] [PDF]
    Rohit J. Kate and Raymond J. Mooney
    In Proceedings of the Joint 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics (COLING/ACL-2006), pp. 913-920, Sydney, Australia, July 2006.

  10. Discriminative Reranking for Semantic Parsing [Abstract] [PDF]
    Ruifang Ge and Raymond J. Mooney
    In Proceedings of the COLING/ACL-2006 Main Conference Poster Sessions, pp. 263-270, Sydney, Australia, July 2006.

  11. Learning for Semantic Parsing with Statistical Machine Translation [Abstract] [PDF]
    Yuk Wah Wong and Raymond J. Mooney
    In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL-2006), pp. 439-446, New York City, NY, June 2006.

  12. Learning Semantic Parsers Using Statistical Syntactic Parsing Techniques [Abstract] [PDF]
    Ruifang Ge
    Ph.D. proposal, Department of Computer Sciences, University of Texas at Austin, February 2006.
    41 pages.
    Also appears as Technical Report UT-AI-TR-06-327, Artificial Intelligence Lab, University of Texas at Austin, February 2006.

  13. Using Active Relocation to Aid Reinforcement Learning [Abstract] [PDF]
    Lilyana Mihalkova and Raymond Mooney
    In Proceedings of the 19th International FLAIRS Conference (FLAIRS-2006), pp. 580-585, Melbourne Beach, Florida, May 2006.

  14. A Kernel-based Approach to Learning Semantic Parsers [Abstract] [PDF]
    Rohit J. Kate
    Ph.D. proposal, Department of Computer Sciences, University of Texas at Austin, November 2005.
    34 pages.
    Also appears as Technical Report UT-AI-05-326, Artificial Intelligence Lab, University of Texas at Austin, November 2005.

  15. Learning for Semantic Parsing Using Statistical Machine Translation Techniques [Abstract] [PDF]
    Yuk Wah Wong
    Ph.D. proposal, Department of Computer Sciences, University of Texas at Austin, October 2005.
    53 pages.
    Also appears as Technical Report UT-AI-05-323, Artificial Intelligence Lab, University of Texas at Austin, October 2005.

  16. A Statistical Semantic Parser that Integrates Syntax and Semantics [Abstract] [PDF]
    Ge, R. and Mooney, R.J.
    Proceedings of the Ninth Conference on Computational Natural Language Learning, Ann Arbor, MI, pp. 9--16, June 2005.

  17. Learning to Transform Natural to Formal Languages [Abstract] [PDF]
    Kate, R.J., Wong, Y. W., and Mooney, R.J.
    Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05), Pittsburgh, PA, pp. 1062--1068, July 2005.

  18. Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer [Abstract] [PDF]
    Gregory Kuhlmann, Peter Stone, Raymond J. Mooney, and Jude W. Shavlik
    Proceedings of the AAAI-2004 Workshop on Supervisory Control of Learning and Adaptive Systems, pp. 30-35, San Jose, CA, July 2004.

  19. Learning Transformation Rules for Semantic Parsing [Abstract] [PDF]
    Rohit J. Kate, Yuk Wah Wong, Ruifang Ge, and Raymond J. Mooney
    Unpublished Technical Note, April 2004.

mooney@cs.utexas.edu