A Question-Answering System for AP Chemistry: Assessing KR&R Technologies (2004)
Ken Barker, Vinay K. Chaudhri, Shaw Yi Chaw, Peter E. Clark, James Fan, David Israel, Sunil Mishra, Bruce Porter, Pedro Romero, Dan Tecuci, Peter Yeh
Basic research in knowledge representation and reasoning (KR&R) has steadily advanced over the years, but it has been difficult to assess the capability of fielded systems derived from this research. In this paper, we present a knowledge-based question-answering system that we developed as part of a broader effort by Vulcan Inc. to assess KR&R technologies, and the result of its assessment. The challenge problem presented significant new challenges for knowledge representation, compared with earlier such assessments, due to the wide variability of question types that the system was expected to answer. Our solution integrated several modern KR&R technologies, in particular semantically well-defined frame systems, automatic classification methods, reusable ontologies, a methodology for knowledge base construction, and a novel extension of methods for explanation generation. The resulting system exhibited high performance, achieving scores for both accuracy and explanation which were comparable to human performance on similar tests. While there are qualifications to this result, it is a significant achievement and an informative data point about the state of the art in KR&R, and reflects significant progress by the field.
In The Ninth International Conference on the Principles of Knowledge Representation and Reasoning (KR2004) 2004.

Ken Barker Formerly affiliated Research Scientist kbarker [at] cs utexas edu
Peter Clark Formerly affiliated Research Scientist peterc [at] vulcan com
James Jumin Fan Ph.D. Alumni jfan [at] cs utexas edu
Bruce Porter Faculty porter [at] cs utexas edu
Dan Tecuci Ph.D. Alumni tecuci [at] cs utexas edu
Peter Zei-Chan Yeh Ph.D. Alumni pzyeh [at] cs utexas edu