Chapter 26 Cleanroom Software Engineering

1. Chapter 26Cleanroom Software Engineering

2. Cleanroom • Developed in early 80’s by Harlan Mills • Reported very good results • reliable, high-quality software • inexpensively and quickly produced • Not used very much • Moderate use of formal methods 329-27

3. Moderate? • Extreme use of formal methods • prove theorems in a formal language • a program checks all proofs • Moderate use of formal methods • prove theorems on a whiteboard • a group of people talk about the proof until all are satisfied 329-27

4. Cleanroom increment • Requirements gathering • Box structure specification • Formal design • Correctness verification (proofs) • Code generation • Code inspection • Statistical use testing • Certification 329-27

5. Testing • Purpose is to estimate quality • Purpose is NOT to improve quality • if there are a significant number of bugs, do it over • Tests are generated based on • what users actually do • probability that event will occur 329-27

6. Statistical use testing • Make model of how the system will be used • List the set of stimuli that cause the software to change its behavior • Estimate the probability of each stimuli • Generate tests based on probability 329-27

7. The formal part • Box structure specification • Formal design • Correctness verification (proofs) • Goal: produce code that matches specification 329-27

8. Box Specification • BB - black box • sequence of stimuli (input events) • response • rules that map stimuli to response 329-27

9. Black box bank account • Stimuli • deposit x, withdraw y, check-balance • Results • OK, BOUNCE, BALANCE z • BankAccount is a function BankAccount( stimulusHistory: Seq of Stimuli, stimulus: Stimuli) -> Results 329-27

10. Black box bank account Define function balance(Seq of Stimuli) • balance({}) = 0 • balance(SS+S) = • if (S = withdraw X) and X <= balance(SS) then balance(SS) - X • if (S = deposit X) then balance(SS)+X • ottherwise, balance(SS) 329-27

11. Black box bank account BankAccount(stimH, s) if s = balance then BALANCE stim(H) else if s = deposit X then OK else “s = withdrawal X” if X <= balance(stimH) then OK else BOUNCE 329-27

12. Box specification • SB - state box • single stimulus (input event) • response • state • rules that map stimulus and old state to response and new state 329-27

13. State box bank account Bank account has one variable: balance BankAccount(s) if s = balance then BALANCE balance else if s = deposit X then balance’ = balance + X and OK else if x = withdraw X then if X <= balance then balance’ = balance - X and OK else BOUNCE 329-27

14. Boxes • BB: S, T => R where S is a sequence of stimuli, T is a stimulus, and R is a result • SB: S, P => R, Q where S is a stimulus, P and Q are states, and R is a result. • CB: Clear box can use any code to specify the function from stimuli to responses. 329-27

15. Design • Design is the step of converting a Black Box or State Box into a Clear Box. • Clear Box is usually described by pseudocode. • For each step of the design, the designers prove that the step is correct. • Each kind of step has a rule for proving it correct. 329-27

16. Code generation • Once a design is expressed only as Clear Boxes, it is easy to translate into a programming language like C or Java. • The programmers translate the design into code. 329-27

17. Advantages of Cleanroom • Verification becomes a finite process • Improves quality • Can verify every line of design and code • It results in a near zero defect level • It scales up • It produces better code than unit testing 329-27

18. Near Zero Defect Level? KLOC, error/KLOC Ericsson OS-32: 350 1 1.7 improvement HP 3.5 1.4 IBM 107 2.6 486 LOC/PM IBM 86 1.2 US Army 75 0.8 4.8 improvement 329-27

19. Summary • If reliability is very important, Cleanroom techniques should be considered • Reasonably efficient of programmer time • Works for groups of 70 programmers • Not popular, and there are probably reasons 329-27