CSE403 Software Engineering Autumn 2001 More Project Management and Testing Overview

1. CSE403 Software Engineering Autumn 2001More Project Management and Testing Overview Gary Kimura Lecture #9 October 19, 2001

2. Today’s quote “I have a pop quiz and I’m not afraid to use it!”

3. Today • Due today • Final Requirements document • Weekly status report • Two separate emails please • Two purposes for information hiding • The academic and engineering arena • The competitive environment • A little more on group management • Testing overview

4. Group management • Good group management is a big factor in doing a large multi-person project • Management works both ways (i.e., managing subordinates and managing your manager) • When you give someone the responsibility over a project then you’ve also given the authority to make decisions for that project. Another way of saying this is that “Responsibility implies authority” • Pay versus other rewards

5. Group rivalries • Rivalries both inter- and intra-company (for a large company) are a fact of life. This doesn’t mean they are necessarily always good or bad • Good because • Hopefully the best engineered design wins • Some people really enjoy and perform well under competition (e.g., Win98 versus WinNT) • Bad because • Waste of resources • Hard feelings and undercutting (this happened a lot at DEC) • Other reasons?

6. More design issues • Having a well understood and well communicated architecture is vital • Consensus on a design is not always practical • Architects at various levels of the system are needed to formulate and communicate the design to others on the team • A well designed API set helps define the division of modules, labor, and necessary information hiding

7. Modularization and information hiding are abstractly good; but they do have a cost • Performance might be sacrificed by a clean design. For example, • “Heavy weight” procedure cost time and space, and identifying these cost is not always easy. Even the number of parameters has a cost. Using procedures cannot to be avoided, but must be respected. • Internal data structures are often conceptually different allocations but each allocation has a small overhead cost • Using well designed submodules can make debugging conceptually easier; however experience shows that programmers often need to know fairly minute subcomponent details to debug their new code

8. Functional module layout and temporal module organization go hand-in-hand • A clean design also needs a clean workspace • Modules can be organized separating exported from local routines • Global header files need to be well organized • Smart editors make locating places where functions and structures are defined and used easier

9. Often we need to sleep on the design • As with other engineering aspects the group must not become paralyzed trying to settle design issues. An educated decision is better than no decision. • Experience and education helps us all make better educated decisions • Try and anticipate changes. Here are some NT design issues that have cropped up through the years. Not all of them are changeable or fixable. • 32 bit to 64 bit expansion • Big and little endian support • Time format without time zone information • String format is a counted string limited to 64KB in length and with Unicode the string length no longer equals character count.

10. A few things to keep in mind • Turnover (via desertion or job growth) • Not everyone can work at 100% throughout a long (or even short) project • Training new hires • Buy-in and communication is always vital

11. Now onto testing • Educational objective • Understand why testing is the primary vehicle for quality assurance • Understand the inherent flaw with testing • Understand that there are many different levels and styles of testing, each with different objectives

12. Verification and validation • Validation: “Did we build the right system?” • Primarily a requirements-level (upper lifecycle) issue • Verification: “Did we build the system right?” • Primarily a lower lifecycle issue (design, implementation, testing)

13. Approaches to verifying software systems • Testing • A dynamic approach • Program verification • Use math to show an equivalence between a specification and a program • Process • Improving the likelihood that code is correct

14. Process • Process includes a broad set of ideas and approaches • Software inspections, walkthroughs, reviews • Capability maturity model, ISO 9000 • Etc.

15. Testing vs. proving • Dynamic • Builds confidence • Can only show the presence of bugs, not their absence • Used widely in practice • Costly • Static • It’s a proof • Proofs are human processes that aren’t foolproof • Applicability is practically limited • Extremely costly

16. An aside • Dynamic techniques are unattractive because they are unsound—you can believe something is true when it’s not • Static techniques are unattractive because they are often very costly (and there are other reasons) • The truth is that they should be considered to be complementary, not competitive

17. Testing • In any case, testing is by far the dominant approach to demonstrating that code does what it supposed to (whatever that means!) • Testing is a lot like the weather: everybody complains about it, but nobody seems to do much about it

18. Two kinds of improvements • One goal is to improve testing to increase the quality of the software that is produced • Another goal is to reduce the costs of testing while maintaining the current quality of the software that is produced. Even if people don't admit it, this is a common focus

19. Terminology • A failure occurs when a program doesn't satisfy its specification • A fault occurs when a program's internal state is inconsistent with what is expected (usually an informal notion) • A defect is the code that leads to a fault (and perhaps to a failure) • An error is the mistake the programmer made in creating the defect

20. More terminology • A test case is a specific set of data that exercises the program • A test suite is a set of test cases • Old terminology • A test case (suite) fails if it demonstrates a problem • New terminology • A test case (suite) succeeds if it demonstrates a problem

21. Root cause analysis • Tries to track a failure to an error • Identifying errors is important because it can • help identify and remove other related defects • help a programmer (and perhaps a team) avoid making the same or a similar error again

22. Unit White-box Black-box Gray-box Bottom-up Top-down Boundary condition Syntax-driven Big bang Integration Acceptance Stress Regression Alpha Beta Kinds of testing

23. A simple problem • The program reads three integer values. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is isosceles, equilateral, or scalene. • Write a set of test cases that you feel would adequately test this program

24. In practice • 13 kinds of errors were found in actual programs • When highly experienced programmers are given this example, on the average they figure out about half of the kinds of errors

25. The lucky thirteen... • Valid scalene triangle • Valid equilateral triangle • Valid isosceles triangle • Three cases that represent valid isosceles triangles in all permutations • One side is zero • One side is negative • 3 positive integers where two sum to the third • All permutations of the previous case

26. The remaining ones • 3 positive integers where two sum to less than the third • 3 permutations of the previous case • All sides are zero • A non-integer side • An incorrect number of inputs • Machine overflow (my own)