Pop Quiz

1. Pop Quiz • Define Fan-in and Fan-out • Does it matter if (what should be done about) code (which) has a high Fan-in and a high Fan-out content? • What is the goal of a Software Quality Management Model? • Why is the Rayleigh model good for quality management?

2. Software Quality EngineeringCS410 Class 11 Complexity Metrics and Models

3. Complexity Metrics and Models • Complexity Metrics and Models: • Provide clues to help focus quality improvement efforts by looking at the program-module level. • Focused on internal dynamics of design and code • Studied by Computer Scientists and Software engineers • In contrast, Reliability and Quality Management Models are: • Unit of analysis is not as granular • Focused on external behavior of process (eg. inspection defects), or product (eg. failure data). • Studied by researchers, reliability experts, and project managers

4. Lines of Code (LOC) • A representation of program size • Can be measured in different ways (chap. 4) • HLL Source Statements • Assembler Statements • Executable lines • Executable lines plus data definitions

5. Lines of Code (LOC) • General assumption: • The more lines of code, the more defects are expected, and a higher defect density (defects per KLOC) is expected • Research has shown that the general assumption may not be true in all cases. • A optimum code size may exist in which expected defect rates would be contained within an acceptable upper limit. • Such an optimum may depend on language, project, product, and environment

6. Lines of Code (LOC) • A curvilinear (p. 76-77) relationship more accurately describes the relationship between code size and defect density: • Example: Fig. 10.1 p. 255 • Module size affects defect density: • Small modules require more external interfaces • Large modules become very complex • Key is to use empirical data, historical data, and comparative data to establish a guideline for module size

7. Halstead’s Software Science • Halstead (1977) stated that Software Science is different from computer Science and that Software Science consists of programming tasks. • A programming task is the act of selecting and arranging a finite number of program ‘tokens’ which are basic syntactic units distinguishable by a compiler.

8. Halstead’s Software Science • Tokens can be classified as either operators or operands. • Primary measures for Halstead’s Software Science are: n1 = the number of distinct operators that appear in a program n2 = the number of distinct operands that appear in a program N1 = the total number of operator occurrences N2 = the total number of operand occurrences

9. Halstead’s Software Science • Based on the four primitives a system of equations describing the program can be applied: • Total Vocabulary Vocabulary (n) = n1 + n2 • Overall Program Length Length (N) = N1 + N2 • Potential Minimum Volume (in bits) for algorithm Volume (V) = NLog2(n1 + n2)

10. Halstead’s Software Science • Program Level (a complexity measure) Level (L) = n1 + n2 • Program Difficulty Difficulty (D) =(n1/2) x (N2/n2) • Development Effort Effort =V / L • Projected number of faults Faults (B) =V / S where S equals a mean number of decisions between errors (3000 used as default)

11. Halstead’s Software Science

12. Halstead’s Software Science • Problems with Halstead’s Software Science: • Faults equation is oversimplified. It simply states that the number of faults in a program is a function of its volume. • Equations do not provide relevant information, only good for comparison • Data for predictions must be available to perform equations, which means code must already be written.

13. Cyclomatic Complexity • McCabe (1976) designed a system to indicate a programs testability and understandability (maintainability). • A.K.A. McCabes Complexity Index or CPX • Based on graph theory of cyclomatic number of regions in a (flow) graph • Represents the number of linearly independent paths comprising a program • Gives an upper bound to the number of test-cases that would be required for path testing

14. Cyclomatic Complexity • General formula: V(G) = e - n + 2p where: e = number of edges n = number of nodes p = number of unconnected parts to the graph or V(G) = e - n + 2 If all parts are connected

15. Cyclomatic Complexity • Example: set of processing a; If cond1 Then set of processing b else set of processing c; set of processing d; If cond2 then set of processing e; else set of processing f; set of processing g;

16. Cyclomatic Complexity • Example: More complexity

17. Cyclomatic Complexity • Cyclomatic Complexity also can be computed based on the decision in a program: • n binary decisions: V(g) = n + 1 • three-way decision: counts as 2n + 1 binary decisions • n-way decision: counts as n - 1 binary decisions • loops: count as 1 binary decision • note: does not distinguish between different type of control flow (eg. loops vs. IF-THEN-ELSE) • Cyclomatic complexities are additive • The Cyclomatic Complexity of one large graph is the sum of the individual graph’s complexities.

18. Cyclomatic Complexity • McCabes recommendation: • To achieve a good testability and maintainability, no program module should have a Cyclomatic Complexity greater than 10. • Cyclomatic complexity correlates strongly with program size (LOC). • There also tends to be a positive correlation between Cyclomatic Complexity and defects. • Cyclomatic Complexity appeals to many software developers because it is tied to decisions and branches (logic).

19. Syntactic Constructs • Studies that look at syntactic makeup of a program. • Shen (1985) • Found a correlation between the number of unique operands and the presence of defects • Lo (1992) • Found correlation between field defects in modules, and LOC, IF-THEN-ELSE’s, DO-WHILE’s, unique operands, and number of calls • DO-WHILE turned out to be the greatest factor, and a positive correlation between DO-WHILE use and defects was discovered (programmers needed training)

20. Structure Metrics • Previous metrics all assume that each module is a independent entity. • Structure metrics take into account that (significant) interactions between modules exist. • Yourdon/Constantine (1979) & Myers (1978) both proposed fan-in and fan-out metrics based on the idea of coupling.

21. Structure Metrics • Fan-out = number of modules called • Fan-in = number of modules called by • Module Coupling = “connectedness” • Other Factors: • Inputs, Outputs, and Global Variables • Low Module Coupling = relatively low number of inputs, outputs, and calls • High Coupling = relatively high number of inputs, outputs, and calls

22. Structure Metrics • Small/Simple modules are expected to have high fan-in. These modules are usually located at the lowest levels of the system structure. • Large/Complex modules are expected to have low fan-in. • Modules should generally not have both high fan-in and high fan-out. If so - then module is good candidate for redesign (functional decomposition).

23. Structure Metrics • Modules with high fan-in are expected to have low defect levels. • Fan-in is expected to have negative or insignificant correlation with defects • Modules with high fan-out are expected to have higher defect levels. • Fan-out is expected to have positive correlation with defects

24. Structure Metrics • Card and Glass (1990) System Complexity Model: • System Complexity is equal to the sum of structural complexity plus data complexity • where: • structural complexity is equal to mean of squared values of fan-out (fan-in is insignificant) • data complexity is average I/O variables

25. Complexity Metrics and Models Criteria for Evaluation • Explanatory Powers - the metrics/model ability to explain the interrelationships among complexity, quality, and other programming and design parameters. • Applicability - the degree to which the metrics/models can be applied by software engineers to improve the quality of design, code and test.

26. Complexity Summary • Complexity metrics and models describe the complexity characteristics of software components (modules). • Software complexity traditionally has a positive correlation to defects. • A key to achieving good quality software is to reduce the complexity of software designs and implementations.

27. Pop Quiz • What are metrics and models for? • List some advantages of Halstead’s Software Science (you may want some disadvantages, too…) • What is the Back End? • So what’s the Front End? • How good is “good enough”?

28. Software Quality EngineeringCS410 Class 11b Measuring and Analyzing Customer Satisfaction

29. Customer Satisfaction • Customer satisfaction is the ultimate validation of quality! • Product quality and customer satisfaction together form the total meaning of quality. • TQM links customer satisfaction to product quality, and focuses on long-term business success. • Customer Focus - achieve high Customer Satisfaction • Process - reduce process variation and achieve continuous process improvement • Human side of Quality - quality culture • Measurement and Analysis - goal-oriented measurement

30. Customer Satisfaction • Customer satisfaction is important because: • Enhancing customer satisfaction is the bottom line of business success • Retaining existing customers is becoming tougher with ever-increasing market competition • Studies show that it is five times more costly to recruit a new customer than it is to retain an old customer • Dissatisfied customers tell 17 to 20 people about their (negative) experiences • Satisfied customers tell 3 to 5 people about their (positive) experiences

31. Customer Satisfaction Surveys • Three type of Survey: • Face-to-Face • Telephone Interviews • Mailed Questionnaires • Face-to-face interviews • interviewer asks questions from pre-structured questionnaire and records the answers • advantage - high degree of validity of the data because the interviewer can note specific reactions and eliminate misunderstandings about questions being asked • disadvantages - cost, interviewer bias, recording errors, training

32. Customer Satisfaction Surveys • Telephone Interviews • Similar to face-to-face interviews • interviewer asks questions from pre-structured questionnaire and records the answers over the phone • advantage - interviews can be monitored to ensure consistency and accuracy • advantage - less expensive than face-to-face • advantage - can be automated (computer-aided) • disadvantage - lack of direct observations, and lack of contacts (due to not scheduling a meeting)

33. Customer Satisfaction Surveys • Mailed Questionnaire • Does not require interviewers and is therefore less expensive • disadvantage - lower response rates, and samples may not be representative of the population (skewed results - only the people who really have something to say may respond) • Questionnaire must be carefully constructed, validated and pre-tested before being used.

34. Customer Satisfaction Surveys • Comparison of three survey methods:

35. Customer Satisfaction Surveys • Sampling methods: • When customer base is very large it is not possible (or feasible) to sample every customer • Customer satisfaction must be estimated based on a sub-set of population • Four types of probability sampling: • Simple random sampling • Systematic sampling • Stratified sampling • Cluster sampling

36. Customer Satisfaction Surveys • Simple random sampling • Every sample of size n has the same chance of being selected from the population • Method • List each individual is listed once (and only once) • Some mechanical (I.e. a random number generator) process is used to draw the sample • On each draw the already selected individuals are removed from the list • Probabilities of being selected are always equal for each of the remaining individuals

37. Customer Satisfaction Surveys • Systematic sampling • Similar to simple random except that a ratio of samples is selected and a random number is only used for the starting point • Method • Determine population (p) • Determine desired sample size (s) • Determine sampling ratio (k = p/s) • Determine the sampling fraction (f = 1/k) • Randomly select starting point between 1 and k • Example:

38. Customer Satisfaction Surveys • Stratified sampling • Individuals are classified into non-overlapping groups called strata and then simple random samples are selected from each stratum. • Strata selection is usually based on some aspects of the customer environment. • Cluster Sampling • Individuals are group into many clusters (groups) based on some characteristic and then a cluster is selected and sampled.

39. Customer Satisfaction Surveys • Sampling considerations: • Simple random sampling is generally the least expensive sampling method. • Systematic sampling can be biased if: • The list is ordered • Cyclical characteristics conflict with k • Stratified Sampling is usually more efficient, and yields greater accuracy than simple sampling (individuals in each stratum are represented). • Cluster sampling is generally less efficient, but also less expensive than stratified sampling.

40. Customer Satisfaction Surveys • Sample size: • How large should sample size be? • Dependant on: • Confidence level desired • Margin of error that can be tolerated • Larger sample sizes give higher confidence levels and lower margins of error. • Power of sample is dependant on absolute size rather than percentage of population.

41. Customer Satisfaction Surveys • Analyzing Satisfaction Data: • Five-point (Likert scale) is the most commonly used measure. • Run charts can be used to show graphical representation of survey results. • Note - margin of error can also be included. • Example - fig. 11.3 p. 280 • Data can be viewed as: • percent satisfied: - where are we doing well • percent unsatisfied (neutral, dissatisfied, and very dissatisfied) - where do we need to focus

42. Customer Satisfaction Surveys • Specific Attributes: • Measuring specific attributes of the software helps to provide data which will indicate areas of improvement. • The profile of customer satisfaction with those attributes indicates the areas of strength and weakness of the software product. • Notes: • Must be careful with correlating weakness with priority, it may not always be the desirable case. • More important is correlation of specific attribute to overall satisfaction level.

43. IBM - CUPRIMDSO Capability Usability Performance Reliability Installability Maintainability Documentation Service Overall Hewlett-Packard - FURPS Functionality Usability Reliability Performance Serviceability Customer Satisfaction SurveysSpecific Attributes Examples:

44. Customer Satisfaction Surveys • Overall Satisfaction • Each attribute has some correlation to the overall satisfaction rating. • Determining the correlation is the key to prioritizing improvement plans. • Methods for determining correlation (pp. 281-287): • Least squares multiple regression • Logistic regression

45. Customer Satisfaction Surveys • Method for determining priorities: 1. Determine the order of significance of each quality attribute on overall satisfaction by statistical modeling. 2. Plot the coefficient of each attribute from the model (Y-axis) against its satisfaction level (X-axis). 3. Use the plot to determine priority by: Top to Bottom Left to right if same coefficients Example: Fig. 11.4 p. 287

46. Customer Satisfaction Surveys • Satisfaction with the company: • Adds a broader view to Customer Satisfaction • Overall satisfaction and loyalty is attributed to a set of common attributes of the company: • Ease of doing business • Partnership • Responsiveness • Knowledge of the customer’s business • Customer driven

47. Customer Satisfaction Surveys • Key dimensions of company satisfaction: • Technical solutions: quality.reliability, availability, ease of use, pricing, installation, new technology, etc. • Support and Service: flexible, accessible, product knowledge, etc. • Marketing: solution, central point of contact, information, etc. • Administration: purchasing procedure, billing procedure, warranty expiration notification, etc

48. Customer Satisfaction Surveys • Key dimensions of company satisfaction: • Delivery: on time, accurate, post-delivery process, etc. • Company image: technology leader, financial stability, executives images, etc. • Perceived performance is equally as important as actual performance. • Company level data is an important aspect of overall customer satisfaction.

49. Customer Satisfaction Surveys • How good is good enough? • Is it worth it to spend $x to gain y amount of satisfaction? • The key to this question lies in the relationship between customer satisfaction and market share. • Basic assumption - Satisfied customers continue to purchase products from the same company and dissatisfied customers will purchase from other companies.

50. Customer Satisfaction Surveys • As long as market competition exists, then customer satisfaction will be the key to customer loyalty. • Even in a monopoly, customer dissatisfaction will encourage the development and emergence of competition. • Answer to “How good is good enough” Better than the competition.