During the week of June 4th, the following members of the SRI team: Vinay Chaudhri Jim Blythe Bruce Porter Art Souther delivered training to four SME's on the GMU campus in Manassass, VA. Three of the SME's are Bio-informatics graduate students, and one is a senior-level biology student. All are bright and openly enthusiastic about the project. Our goal in writing this report is to communicate to the SRI team the week's activities and results, and to propose a plan for improving SHAKEN during the next two weeks. (The SME trials begin 6/25.) Activities and Results ---------------------- A1) We delivered the following units of instruction: (The instruction materials and labs will soon be available at: www.cs.utexas.edu/users/mfkb/RKF/training/Summer-2001 ) Each unit incorporated a significant lab, almost all of them using SHAKEN. The SME's evaluated SHAKEN and our training with a 3 page questionnaire. The primary results are discussed in the following items. We plan to scan the questionnaires and make them available to you soon. A2) SHAKEN has improved tremendously during the past few weeks - good work team! Only occassionally during the week did the SME's experience problems in the basic functions of browsing, editing, and creating representations. Most of these problems were fixed during the week. A3) The SME's found SHAKEN to be intuitive and easy to use. The graphical reprsentations and the compositional approach were well received. A4) The SME's found the pump-priming knowledge helpful, although we need better ways of presenting it to them. (See item P1, below.) A5) KANAL and question-answering were well received, and the SME's found them to be easy to use and potentially very powerful. We discussed numerous ways we might improve them in short order. (See item P3, below.) Proposed Plan ------------- The training team held lengthy in-depth discussions of high-leverage ways to improve SHAKEN during the next two weeks. These discussions were influenced by frequent feedback from the SME's. We propose focusing on the following issues, roughly sorted in order of decreasing priority: P1) Improve the user's control over the set of relations to show for a particular concept. Currently, SHAKEN offers an abridged description and a complete description. The SME's would like to select the relations from a menu. P2) Enable users to delete relationships (and possibly constraints) in a CMAP. This would allow knowledge entry by copy/edit, as well as correcting mistakes along the way, with more control than the UNDO button offers. This might be problematic to implement, so we need to assess the costs and risks. P3) Improve and thoroughly test KANAL, question answering, and explanation generation. In concert with developing more scenarios for testing these subsystems (see P4), we need a focused effort involving many team members. P4) Use SHAKEN to build scenarios for all the subsections of 7.1 that will be used in the summer trials. Ideally, this will be done by several team members working independently. P5) Develop some additional training materials. The SME's requested more worked-out examples of encoding knowledge from Alberts. Also, we need to provide many more examples of using the Q/A subsystem to answer the range of questions the SME's will face this summer. P6) During the week, but only on occassion, SHAKEN became very sluggish. We experimented with several factors, but never isolated the problem. We need to keep a close eye on this. Personal Highlights (Bruce) --------------------------- H1) I used SHAKEN to build several scenarios for Friday's all day lab. The largest one used 5 generic subevents and numerous entities from the pump priming KB. The exercise required 5 minutes, at most, and it was fun! Afterwards, to investigate an issue in KANAL, I coded the scenario directly in KM using Emacs (as I've done for years). This coding effort took about 1.5 hours to enter and debug! H2) The four SME's are outstanding. If they are as enthusiastic in two months as they are now, that will be more telling than all the empirical evaluations combined. H3) Last week was the one year anniversary of our project (as marked by the kickoff meeting in New Orleans). We have accomplished a lot, progress accelerating, and we have a great group of SME's working this summer to give us feedback. Let's push hard, together, these next two weeks and we'll have every reason to be optimistic about this summer's trial and beyond.