Transfer Learning Reading Group

Paper for Next Meeting (4 June 2008)

• Transfer in Reinforcement Learning via Markov Logic Networks. L. Torrey, J. Shavlik, S. Natarajan, P. Kuppili & T. Walker. AAAI-08 TL in Complex Tasks Workshop. [PDF]

Candidate Papers

• Convex Point Estimation using Undirected Bayesian Transfer Hierarchies. G. Elidan, B. Packer, G. Heitz, and D. Koller. UAI 08 [PDF]
• Deep Transfer via Second Order Markov Logic. Jesse Davis and Pedro Domingos. AAAI-08 TL in Complex Tasks Workshop. [PDF]
• Revising First-Order Logic Theories from Examples Through Stochastic Local Search. Aline Paes, Gerson Zaverucha and Vitor Santos Costa. ECML-07 http://www.springerlink.com/content/8pp344k41g7735k7/fulltext.pdf
• Effective Control Knowledge Transfer Through Learning Skill and Representation Hierarchies. Asadi and Huber. IJCAI-07. http://www.ijcai.org/papers07/Papers/IJCAI07-331.pdf
• Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling. Jan Ramon, Kurt Driessens, and Tom Croonenborghs. ECML-07.Here
• Learning Multiple Related Tasks Using Latent Independent Component Analysis. J. Zhang, Z. Ghahramani, and Y. Yang. http://nyc.lti.cs.cmu.edu/yiming/Publications/zgy-nips05.pdf NIPS-05
• Semi-Supervised Multitask Learning: Liu, Liao, Carin
• Multi-Task Learning via Conic Programming: Kato, Kashima, Sugiyama, Asai
• A Spectral Regularization Framework for Multi-Task Structure Learning: Argyriou, Micchelli, Pontil, Ying
• Bayesian Co-Traning: Yu, Krishnapuram, Rosales, Steck, Rao
• Multi-task Gaussian Process Prediction: Williams, Chai, Bonilla

Past readings (Some papers discussed in the RL reading group are also related to transfer)

• Approximate Policy Iteration with a Policy Language Bias: Solving Relational Markov Decision Processes. Alan Fern, Sungwook Yoon, and Robert Givan. [PDF]
• Learning Relational Options for Inductive Transfer in Relational Reinforcement Learning. Tom Croonenborghs, Kurt Driessens, and Maurice Bruynooghe. ILP-07. Here
• Learning from Relevant Tasks Only. Samuel Kaski, Jaakko Peltonen http://www.springerlink.com/content/v71r033t631g3401/fulltext.pdf
• 3/19/08: Dynamic Weighted Majority: An Ensemble Method for Drifting Concepts. J. Zico Kolter, Marcus A. Maloof http://www.jmlr.org/papers/volume8/kolter07a/kolter07a.pdf
• 2/20/08: Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification. John Blitzer, Mark Dredze, and Fernando Pereira. http://www.cis.upenn.edu/~mdredze/publications/sentiment_acl07.pdf
• 2/6/08: Internal paper (draft of an ICML-08 submission)
• 1/23/08: Learning Bounds for Domain Adaptation. J. Blitzer, K. Crammer, A. Kulesza. F. Pereira, and J. Wortman. http://www.cis.upenn.edu/~blitzer/papers/adaptation_weighting.pdf
• 12/14/07: Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations. M. M. Mahmud and Sylvian Ray. http://books.nips.cc/papers/files/nips20/NIPS2007_0327.pdf
• 11/30/07: Boosting for Transfer Learning. Wenyuan Dai, Quang Yang, Gui-Rong Xue, and Yong Yu. http://imls.engr.oregonstate.edu/www/htdocs/proceedings/icml2007/papers/72.pdf
• 11/16/07:Learning a meta-level prior for feature relevance from multiple related tasks. Su-In Lee, Vassil Chatalbashev, David Vickrey, and Daphne Koller. http://ai.stanford.edu/~vasco/pubs/metaprior.pdf
• 10/25/07: Probabilistic Planning in the Graphplan Framework. Avrim L. Blum and John C. Langford. http://www.cs.cmu.edu/~jcl/papers/planning/ecp.ps
• 10/12/07: Relational Macros for Transfer in Reinforcement Learning. Lisa Torrey, Jude Shavlik, Trevor Walker, and Richard Maclin. ILP-07.
• 9/20/07: Learning Probabilistic Relational Dynamics for Multiple Tasks. Ashwin Deshpande, Brian Milch, Luke Zettlemoyer and Leslie Kaelbling. UAI-07. http://people.csail.mit.edu/lsz/papers/dmzk-uai07.pdf
• Transfer of Experience between Reinforcement Learning Environments with Progressive Difficulty: Madden & Howley (2004)
• Constructing Informative Priors using Transfer Learning.  Rajat Raina, Andrew Ng, Daphne Koller. ICML-06.
  

• Asadi M, Huber M (2005), Accelerating Action Dependent Hierarchical Reinforcement Learning Through Autonomous Subgoal Discovery, In ICML 2005 Workshop on Rich Representations for Reinforcement Learning, Bonn, Germany.
  
  
• Asadi M, Huber M (2004), State Space Reduction For Hierarchical Reinforcement Learning, In Proceedings of the 17th International FLAIRS Conference, pp. 509 - 514, Miami Beach, FL.
  
  
• G. Casella.An Introduction to Empirical Bayes Data Analysis, American Statistician, 39(2): pp. 83-87, 1985.
  
  
• Y. W. Teh, M. I. Jordan, M. J. Beal and D. M. Blei. (2005).  Hierarchical Dirichlet processes. In Advances in Neural Information Processing Systems (NIPS) 17, 2005.  (Long tech report version)  (Related NIPS'05 tutorial slides)
  
  
• Alexandru Niculescu-Mizil, Rich Caruana: Learning the Structure of Related Tasks. In Inductive Transfer NIPS Workshop.
  
  
• Neville Mehta, Sriraam Natarajan, Prasad Tadepalli and Alan Fern. Transfer in Variable Reward Hierarchical Reinforcement Learning. Inductive Transfer NIPS workshop 2005.
  
  
• Robert E. Wray, John E. Laird, Andrew Nuxoll, Devvan Stokes, Alex Kerfoot, Synthetic Adversaries for Urban Combat Training, AI Magazine, 26(3):82-92, 2005.
  
  
• Ontology Matching: A Machine Learning Approach, A. Doan, J. Madhavan, P. Domingos, and A. Halevy. Handbook on Ontologies in Information Systems, S. Staab and R. Studer (eds.), Springer-Velag, 2004. Invited paper. Pages 397-416.
  
  
• O. Ilghami, H. Muñoz-Avila, D. S. Nau, and D. W. Aha. Learning approximate preconditions for methods in hierarchical plans. Proceedings of the International Conference on Machine Learning (ICML), Aug. 2005.
  
  
• O. Ilghami, D. S. Nau, H. Muñoz-Avila, and D. W. Aha. Learning preconditions for planning from plan traces and HTN structure. Computational Intelligence 21(4):388–413, Nov. 2005.
  
  
• Matthew Richardson, Pedro Domingos (2006). Markov logic networks. Machine Learning 62(1-2): 107-136.
  
  
• Stanley Kok, Pedro Domingos (2005). Learning the structure of Markov logic networks. ICML 2005: 441-448.
  
  
• Langley, P. (in press). Cognitive architectures and general intelligent systems. AI Magazine.
  
  
• Langley, P., Choi, D., & Rogers, S. (2005). Interleaving learning, problem solving, and execution in the Icarus architecture (Technical Report). Computational Learning Laboratory, CSLI, Stanford University, CA.
  

Suggested readings from people on the team (and not read yet)

• Ken Forbus
• Forbus, K. and Hinrichs, T. 2004. Companion Cognitive Systems: A step towards human-level AI. AAAI Fall Symposium on Achieving Human-level Intelligence through Integrated Systems and Research, October, Washington, DC.
• Pisan, Y. (1996). Using qualitative representations in controlling engineering problem solving. Qualitative Reasoning Workshop.
• Forbus, K. Usher, J. and Chapman, V. (2003). Qualitative spatial reasoning about sketch maps.  Proceedings of the Fifteenth Annual Conference on Innovative Applications of Artificial Intelligence, Acapulco, Mexico.
• Larry Holder
• Interfacing the D'Artagnan Cognitive Architecture to the Urban Terror First-Person Shooter Game, by Bharat Kondeti, Maheswar Nallacharu, Michael Youngblood, & Lawrence Holder.  In IJCAI 2005 Workshop on Reasoning, Representation, and Learning in Computer Games.
  
• John Laird
• Lehman, J. F., Laird, J. E., Rosenbloom, P., E. (2005) A Gentle Introduction to Soar: 2005 Update. Unpublished. An update to an earlier paper (published in 1996).
• Nason, S. and Laird, J. E.,Soar-RL, Integrating Reinforcement Learning with Soar, Cognitive Systems Research, 6 (1), 2005, pp. 51-59.
  
• Nuxoll, A., Laird, J., A Cognitive Model of Episodic Memory Integrated With a General Cognitive Architecture,  International Conference on Cognitive Modeling 2004.
• Ray Mooney
  
• Theory Refinement for Bayesian Networks with Hidden Variables, Sowmya Ramachandran and Raymond J. Mooney, Proceedings of the Fifteenth International Conference on Machine Learning (ICML-98), Madison, WI, pp. 454-462, July 1998.
• Improving Shared Rules in Multiple Category Domain Theories, Dirk Ourston and Raymond J. Mooney, Proceedings of the Eighth International Machine Learning Workshop, pp. 534-538, Evanston, IL, June 1991.
• Automated Refinement of First-Order Horn-Clause Domain Theories, Bradley L. Richards and Raymond J. Mooney, Machine Learning 19,2 (1995), pp. 95-131.
• Michael Pazzani
  
• Pazzani, M. (1993). Learning causal patterns: Making a transition from data-driven to theory-driven learning. Machine Learning, 11, 173-194.
• Pazzani, M. (1991). A computational theory of learning causal relationships. Cognitive Science, 15, 401-424.
• Peter Stone
  
• Alexander A. Sherstov and Peter Stone. Improving Action Selection in MDP's via Knowledge Transfer. In Proceedings of the Twentieth National Conference on Artificial Intelligence, July 2005.
• Matthew E. Taylor, Peter Stone, and Yaxin Liu. Value Functions for RL-Based Behavior Transfer: A Comparative Study. In Proceedings of the Twentieth National Conference on Artificial Intelligence, July 2005.
  
• Nicholas K. Jong and Peter Stone. State Abstraction Discovery from Irrelevant State Variables. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, pp. 752–757, August 2005.
• David Stracuzzi
• Stracuzzi, D.J. (2005), Scalable knowledge acquisition through memory organization. International and Interdisciplinary Conference on Knowledge Representation and Reasoning (AKRR05).  57-64.
• Michael Witbrock
• Matuszek, Cynthia, M. Witbrock, R. Kahlert, J. Cabral, D. Schneider, P. Shah and D. Lenat. Searching for Common Sense: Populating Cyc from the Web. In Proceedings of the Twentieth National Conference on Artificial
  Intelligence, Pittsburgh, Pennsylvania, July 2005.
  
• Cabral, John, R.C. Kahlert, C. Matuszek, M. Witbrock, B. Summers. Converting Semantic Meta-Knowledge into Inductive Bias. In Proceedings of the 15th International Conference on Inductive Logic Programming, Bonn, Germany, August 2005.

Current Group Members

• Tuyen Huynh
• Lilyana Mihalkova
• Raymond Mooney
• Peter Stone
• Matthew Taylor

Related resources

• ICML-06 Workshop on Structural Reinforcement Learning.  Online Proceedings.
• NIPS-05 Workshop on Inductive Learning.
• NIPS-05 Workshop on Interclass Transfer.
• Berkeley seminar on transfer.