Cleanroom Method

1. Cleanroom Method CS 415, Software Engineering II Mark Ardis, Rose-Hulman Institute March 20, 2003

2. Outline • Harlan Mills • Cleanroom method • Industrial use of cleanroom

3. Harlan Mills 1919 - 1996

4. Mathematics and Programming • Roman accounting • "to go from programming as an instinctive, intuitive process to a more systematic, constructive process"

5. Cleanroom Method • Incremental (spiral) • Box structure specification and design • Design verification • No debugging • Statistical testing

6. Box Structures • Black boxes: behavior only • State Boxes: behavior + state • Clear boxes: procedures

7. S1S2...Sn R stimulus history  response Black Boxes

8. State Boxes State Data S R stimulus, old state  response, new state

9. Clear Boxes State Data S R Procedures stimulus, old state  response, new state

10. Box Description Language (BDL) • Invocation: use <type> <name> <args> • Sequence: do B1; B2od • Alternation: if <cond> then B1else B2fi • Iteration: while <cond> do B od

11. Box Structure Hierarchy BB SB CB BB BB BB SB SB SB CB CB CB

12. Cartoon of the Day (1/3)

13. Cartoon of the Day (2/3)

14. Cartoon of the Day (3/3)

15. Design Verification • Procedures in BDL are checked for correctness with their higher-level descriptions • All boxes (and all procedures) describe functions • Formal proofs of correctness can be performed (but often informal proofs are done, instead)

16. Verification of Sequence Given a high-level function [f] for statement: do [g]; [h] od Does [g] followed by [h] compute the same function as [f]? Example: [f](x) = 2 * x + 7 [g](x) = 2 * x [h](x) = x + 7

17. Verification of Selection Given a high-level function [f] for statement: if <cond> then[g]else[h]fi • Whenever <cond> is true, does [g] compute the same function as [f]? • Whenever <cond> is false, does [h] compute the same function as [f]?

18. Verification of Iteration Given a high-level function [f] for statement: while <cond> do[g]od • Whenever <cond> is true, does [g] followed by [f] compute the same function as [f]? • Does the loop always terminate? • Whenever <cond> is false, does the empty function compute the same function as [f]?

19. Usage Testing • Develop an operational profile of use • Generate random tests that fit the probabilities

20. Example

21. Test Generation

22. Industrial Use • Used in a few areas of IBM • Used by some military contractors • Tried at NASA

23. Software Engineering Laboratory (SEL) • Joint program of NASA Goddard Space Center, Computer Sciences Corporation, and the University of Maryland • Conduct experiments and case studies on new software technology

24. SEL Experience • First trial at University of Maryland • controlled experiment (10 experiment teams,5 control teams • FORTRAN • 1.5 KLOC • 3 case studies at Goddard • flight-dynamics ground support systems • FORTRAN • 40 KLOC, 22 KLOC, 160 KLOC

25. SEL Results – University Experiment • Cleanroom teams • use fewer computer resources • satisfy requirements more successfully • make higher percentage of scheduled deliveries

26. SEL Results – Goddard • More effort spent in design • Better reliability of final product • Smaller projects achieve higher productivity, but large project just average

27. Summary • Cleanroom may be an effective method for achieving higher reliability • Requires some culture change (no debugging) • Still being investigated by researchers and practitioners

28. References Victor Basili and Scott Green, "Software process evolution at the SEL", IEEE Software 11(4), 58-66, July 1994.