The UT Bootstrap Learning Reading Group meets alternate Mondays, 2:30 pm, to read and discuss papers relevant to the DARPA Bootstrap Learning project. The focus is on methods for transferring knowledge from humans to machines (both robots and virtual agents) through “natural instruction methods” instead of programming (e.g., by demonstration, giving feedback, gesturing or “showing”, etc.)  We are particularly interested in teaching skills in multiple lessons, either by composing simpler skills in a hierarchical fashion or improving / refining simpler skills in stages, in order to build up increasingly complex abilities.

The meetings are scheduled to be in the robot lab meeting room, but changes may be announced before the meeting via email to the mailing list.

To subscribe to the mailing list, go to https://utlists.utexas.edu/sympa/subscribe/bootstrap-learning and enter your email address.

Future meetings will be on: May 5, May 19

Upcoming Papers:

Monday, May 19 – Meet at 3:00 PM

Hinton, G. E. and Salakhutdinov, R. R. "Reducing the dimensionality of  data with neural networks", Science 2006.

[paper] [supplementary material]

I also will refer to this paper (it is short but I won’t assume you’ve read it for the meeting):

Salakhutdinov, R. R. and Hinton, G. E., "Learning a Nonlinear  Embedding by Preserving Class Neighbourhood Structure", AISTATS 2007.   [pdf]

More related material is at this NIPS 2007 tutorial page on deep belief networks.

Previous Papers:

Monday, May 5

Discussion about RL algorithms, their strengths and weaknesses.  This week will focus on LSTD, LSPI and fitted value iteration.

Monday, April 14

Ian Fasel, Michael Quinlan, Peter Stone, “A General Purpose Task Specification Language for Bootstrap Learning”, UTCS Technical Report UT-AI-08-1, 2008. [pdf]

Discussion: Beating the no free lunch theorem: towards a strategy for general-purpose learning from feedback.  (Automatic?) discovery of MDP categories. Possible teacher utterances and what RL algorithms work for those utterances. 

Monday, March 31

Umar Syed, Robert Schapire, “A Game-Theoretic Approach to Apprenticeship Learning.” Advances in Neural Information Processing Systems. 2007.

[ps.gz] [pdf] [bibtex] [supplemental] [slide]

Monday, March 3

J. Zico Kolter, Pieter Abbeel, Andrew Ng, “Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion.” Advances in Neural Information Processing Systems. 2007. [ps.gz] [pdf] [bibtex] [slide] [audio]

You probably also want to look at the earlier apprenticeship learning paper:

Pieter Abbeel, Andrew Ng, “Apprenticeship learning via inverse reinforcement learning.” In 21st International Conference on Machine Learning (ICML). 2005.  PDF

Wednesday, December 5

Cynthia Breazeal, Matt Berlin, Andrew Brooks, Jesse Gray and Andrea L. Thomaz. "Using perspective taking to learn from ambiguous demonstrations". Robotics and Autonomous Systems, Volume 54, Issue 5, 31 May 2006, Pages 385-393    (pdf)

Monday, November 19

Wesley Kerr. Shane Hoversten. Daniel Hewlett. Paul R. Cohen. Yu-Han Chang. Learning in Wubble World. IEEE International Conference on Development and Learning (ICDL), 2007. PDF

(see the rest of the site: http://www.wubble-world.com/?page_id=11)

Monday, November 5

J. Triesch, H. Jasso, and G. Deák. Emergence of mirror neurons in a model of gaze following. Adaptive Behavior, accepted 2007. (pdf)

(This is a reinforcement learning model in which an agent learns to follow the teacher’s line of regard.  There isn’t any vision science here, the interesting part is learning which cues from the teacher to pay attention to and how to use them.)

Monday, October 22

D. Grollman and O. Jenkins. Dogged learning for robots. In International Conference on Robotics and Automation (ICRA 2007), Rome, Italy, Apr 2007.  http://www.cs.brown.edu/~cjenkins/papers/ICRA2007.pdf

Monday, October 8, 2007, in ACES 2.404B

Darrin C. Bentivegna, Christopher G. Atkeson, and Gordon Cheng. Learning from Observation and Practice using Primitives. In AAAI Fall Symposium Series, Symposium on "Real-life Reinforcement Learning", October 22-24, 2004. (pdf)

D. C. Bentivegna and C. G. Atkeson. A framework for learning from observation using primitives. In RoboCup 2002: Robot Soccer World Cup VI, volume 2752 of Lecture Notes in Computer Science, pages 263–270. Springer, Berlin / Heidelberg, 2003.  (pdf)   (original link)

   for movies and more papers, go to http://www.andrew.cmu.edu/user/darrinb/

Monday, September 24, 2007

Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6), December 1994. (pdf)   (original link)

Future Suggested Papers:

Please email any suggested future readings to: bootstrap-learning-request@utlists.utexas.edu

Chen Yu, Hui Zhang and Linda B. Smith, "Learning through Multimodal Interaction", Proceedings of the 5th International Conference of Development and Learning, Bloomington, IN, 2006. (pdf)

J. A. Clouse and P. E. Utgoff. A teaching method for reinforcement learning. In Proc. International Conference on Machine Learning, pages 92–101, 1992.   (could not find a pdf)

Special issue on Imitation Learning / Programming by Demonstration:

     Robotics and Autonomous Systems, Volume 54, Issue 5, Pages 351-418 (31 May 2006)

     The Social Mechanisms of Robot Programming from Demonstration