Donald Bren School of Information and Computer Sciences

The Cleanroom software development approach is intended to produce highly reliable software by integrating formal methods for specification and design, nonexecutlon-based program development, and statistically-based independent testing. In an empirical study, 15 three-person teams developed versions of the same software system (800 - 2300 source lines); ten teams applied Cleanroom, while five applied a more traditional approach. This analysis characterizes the effect of Cleanroom on the delivered product, the software development process, and the developers.

The major results of this study are the following, (1) Most of the developers were able to apply the techniques of Cleanroom effectively (six of the ten Cleanroom teams delivered at least 91% of the required system functions). (2) The Cleanroom teams' products met system requirements more completely and had a higher percentage of successful operationally generated test cases. (3) The source code developed using Cleanroom had more comments and less dense control-flow complexity. (4) The more successful Cleanroom developers modified their use of the implementation language; they used more procedure calls and IF statements, used fewer CASE and WHILE statements, and had a lower frequency of variable reuse (average number of occurrences per variable). (5) All ten Cleanroom teams made all of their scheduled intermediate product deliveries, while only two of the five non-Cleanroom teams did. (6) Although 86% of the Clean room developers indicated that they missed the satisfaction of program execution to some extent, this had no relation to the product quality measures of implementation completeness and successful operational tests. (7) Eighty-one percent of the Cleanroom developers said that they would use the approach again.