Hypothesis
The software industry has particular difficulty producing large software systems on schedule, of high quality and at a pre-established budget. Watts Humphrey, of the Software Engineering Institute (SEI), has approached this problem by developing the Personal Software Process (PSP) development methodology. PSP requires each individual software engineer to strictly adhere to a defined, disciplined software development process. PSP has met with great success in industry and is known for consistently and efficiently producing high-quality products. The value of PSP has been documented in several industrial case studies and is also taught at many universities, including the University of Utah.
Though not as well documented, accepted, or proven as PSP, the Extreme Programming (XP) process, developed primarily by a Smalltalk code developer and consultant Kent Beck, attributes great success to the use of "pair programming". With pair programming, two programmers work side-by-side at one computer, collaborating on the same design, algorithm, code or test. Results demonstrate that the two programmers work together more than twice as fast and think of more than twice as many solutions to a problem as two working alone, and have higher defect prevention and defect removal leading to a higher quality product. The Extreme Programming process employs many other techniques counter to most practices accepted by the mainstream software engineering community. Therefore, isolating which factor to attribute the success to is difficult. The research outlined here will help to dissect the contributing factors.
This research will define and validate the Collaborative Software Process (CSP) by incorporating pair programming practices as new or changed sub-processes of the PSP. The CSP will give a pair of programmers a framework of techniques and measurement-based feedback to improve their joint performance, and thereby the performance of their organization. This research aims to show that a two-person CSP-trained team can work more than twice as fast and with higher quality than one PSP-trained person working alone.
Prior Research
Research and results on the Personal Software Process (PSP) and Extreme Programming(XP) has been discussed above. Two other studies support the use of collaborative programming. Larry Constantine, a programmer, consultant, and magazine columnist reports on observing "Dynamic Duos" during a visit to P. J. Plaugherís software company, Whitesmiths, Ltd, providing anecdotal support for collaborative programming. He immediately noticed that at each terminal were two programmers working on the same code. He reports, "Having adopted this approach, they were delivering finished and tested code faster than ever . . . The code that came out the back of the two programmer terminals was nearly 100% bug free . . . it was better code, tighter and more efficient, having benefited from the thinking of two bright minds and the steady dialogue between two trusted terminal-mates . . . Two programmers in tandem is not redundancy; itís a direct route to greater efficiency and better quality."
An experiment studied 15 full-time, experienced programmers working on a challenging problem, important to their organization, in their own environment, and with their own equipment. Five worked individually, ten worked collaboratively in five pairs. Conditions and materials used were the same for both the experimental (team) and control (individual) groups. This study provided statistically significant results, using a two-sided t-test. "To the surprise of the managers and participants, all the teams outperformed the individual programmers, enjoyed the problem-solving process more, and had greater confidence in their solutions." The groups completed the task 40% more quickly and effectively by producing better algorithms and code in less time. The majority of the programmers were skeptical of the value of collaboration in working on the same problem and thought it would not be an enjoyable process. However, results show collaboration improved both their performance and their enjoyment of the problem solving process. .
Research Methodology
Process Document. As stated earlier, the PSP is a thorough, documented software development process. Each sub-process has a set of scripts, forms and templates to give the programmer specific procedures and to obtain data for measurement-based feedback. Based on theoretical research on collaboration and on actual experiences with collaborative programming, these sub-process descriptions, scripts, forms and templates will be adjusted to give a framework for the pairs of programmers to follow the Collaborative Programming Process.
Information will be gathered on actual experiences with collaborative programming via a web-based questionnaire. Requests for questionnaire respondents will be sent to specific people who are known to have done pair programming, to the "pair programming" mailing list, and to graduate students at the University of Utah. Additionally, the questionnaire will be publicized on the two main Extreme Programming web sites.
Empirical Study. The validation of this new process will be based on an empirical study of students at the University of Utah. In the summer of 1999, an exploratory course will be taught to pairs of graduate student working on research projects in the department. They will be taught CSP processes in the classroom, which they will apply to the programming they are doing for their research. They will enter data to complete all forms and templates. At course completion, each student will write a paper critiquing the CSP. These critiques by experienced graduate students will be used to update and enhance the CSP process.
The structured experiment to validate the effectiveness of the Collaborative Software Process will take place in Fall 1999. The senior "Software Engineering Laboratory" at the University of Utah will be used for the study. Approximately 60-70 students will take this class. The majority of the student will be familiar with PSP because they had been instructed on it in their CS1 and CS2 class using the Introduction to the Personal Software Process book .
The student will learn of the experiment and receive an overview of the CSP during the first class. They will then immediately be given a survey to assess their attitudes towards collaborative programming, which will be contrasted with a similar survey given at the end of the semester. The survey will probe the studentsí feeling towards how much they will enjoy working collaboratively, if they believe working collaboratively will improve their confidence in their solutions, and if they believe it will improve their productivity and quality.
Classes will proceed in the class using the framework and instructorís guide for the more advanced PSP book, A Discipline for Software Engineering . Many of the PSP sub-process and scripts will be used in the CSP process and will taught to the class. In the cases where a particular sub-process differs between the CSP and the PSP, both sub-processes will be taught to the entire class. For example, there will be a class dedicated to the code review sub-process. The procedures for doing code review alone (PSP) and for doing code review collaboratively (CSP) will both be discussed and contrasted. All aspects of the development cycles for both PSP and CSP will be taught likewise.
The students will be randomly divided evenly into Group A and to Group B. Group A students will independently complete three of Assignment #1 through Assignment #6. Group B students will be divided up into self-selected pairs to work collaboratively on all of Assignment #1 through Assignment #6. The PSP and CSP processes require the students to record data about the time they spend and the defects they inject and remove. This data will allow quality and productivity measures to be tracked and analyzed throughout the semester as the students learn to apply more and more sub-processes of PSP and CSP. Historically, PSP data has shown a productivity improvement and a decrease in defects through a semester as the students apply more of the techniques. This experiment should show the same trend for improved productivity and defects density for all students. However, the results for the CSP projects and for the PSP projects will be analyzed separately and compared. The relationship of the measurements for the two processes will be used to validate the research hypothesis.
Additionally, results of the pre- and post-class survey will provide for a quantitative study of the attitude adjustment of the students concerning collaborative programming, after actually practicing the techniques. The final exam for the class will be a thoughtful, written evaluation of the CSP which will provide further qualitative evidence of the effectiveness of the Collaborative Software Process. Specific points that must covered in this evaluation will be distributed to the students.
Often empirical software engineering studies involving students are not highly regarded research because it is not viewed that projects done in a semester need deal with issues of scope or scale that often complicate real, industrial projections. Even Watts Humphrey ñ working in an industrial organization, did his initial PSP studies on students, because he could not find any real project that would risk its success on a new process. However, he says "You can apply PSP principles to almost any software-engineering task because its structure is simple and independent of technology -- it prescribes no specific languages, tools or design methods. His study, involving students, was highly regarded, respectable research. CSP, though on a slightly larger scale, can be considered likewise. This study involves the interactions between and efficiencies of two programmers working collaboratively. Issues of complexity and scale are not inhibitors to the external validity of a study of CSP with students at the University of Utah
Research Contributions
Mature engineering disciplines have documented, repeatable processes to predictably produce products on-schedule and of high quality. The Personal Software Process has proven to have made a strong contribution in this area to the maturity of software engineering. Through this research, the Collaborative Software Process will add another documented, repeatable process. The empirical study will compare the effectiveness of these two methodologies in terms of programmer productivity and system quality. All possible outcomes of this study will provide valuable information to programming organizations making resource allocation decisions, always striving to produce high quality products on-time. If CSP outperforms PSP, they will know that the use of collaborative programming could improve their results. Also, computer science students could also be taught to program collaboratively in the future. If CSP yields the similar results to PSP, they could use PSP or CSP interchangeably, potentially the choice of the individual programmers. Lastly, if PSP outperforms CSP, they will know that collaborative programming is a bad resource decision. The research will also provide a tool to support the use of the Collaborative Software Process.
Bibliography
Beck, K., Cunningham, Ward (1999). Extreme Programming Roadmap, http://c2.com/cgi/wiki/ExtremeProgramming. 1999.
Constantine, L. L. (1995). Constantine on Peopleware. Englewood Cliffs,
NJ, Yourdon Press.
Ferguson, P., Humphrey, Watts S., Khajenoori, Soheil, Macke, Susan and Matvya,
Annette (1997). Results of Applying the Personal Software Process. Computer.
May 1997: 24-31.
Humphrey, W. S. (1995). A Discipline for Software Engineering, Addison
Wesley Longman, Inc.
Humphrey, W. S. (1996). Using a Defined and Measured Personal Software Process.
IEEE Software. May 1996: 77-88.
Humphrey, W. S. (1997). Introduction to the Personal Software Process,
Addison-Wesley.
Nosek, J. T. (1998). The Case for Collaborative Programming. Communications
of the ACM. March 1998: 105-108.