SENG 521 Software Reliability & Testing

1. SENG 521Software Reliability & Testing Defining Necessary Reliability (Part 3a) Department of Electrical & Computer Engineering, University of Calgary B.H. Far （far@enel.ucalgary.ca） http://www.enel.ucalgary.ca/~far/Lectures/SENG521/03a/ far@enel.ucalgary.ca

2. Error cause Fault cause Failure Error - Fault - Failure • An error is a human action that results in software containing a fault. • A fault (bug) is a cause for either a failure of the program or an internal error (e.g., an incorrect state, incorrect timing). • Among the 3 factors only failure is observable. far@enel.ucalgary.ca

3. Necessary Reliability: How to • Define failure with “failure severity classes (FSC)” for the product. • Choose a common measure for all associated systems. • Set a “failure intensity objective (FIO)” for each system to be tested. • Find the developed software failure intensity objective. • Engineer strategies to meet the software failure intensity objective. far@enel.ucalgary.ca

4. Failure Severity Classes • Failures usually differ by their impact on the system • A failure Severity Class (FSC) is a set of failures that have the same per-failure impact on users using a failure classification criteria • Common classification criteria: • cost, system capability, human life, environment • Failure severity is different from its complexity • Severity can change with the time of failure far@enel.ucalgary.ca

5. FSC: Common Classification • Common classification criteria: Cost • What does this failure cost in terms of operational cost, repair cost, loss of business, disruption, etc. • Severity classes based on cost may be scaled by a factor of 10. • Usually 4 ranges are enough. far@enel.ucalgary.ca

6. FSC: Common Classification • Common classification criteria: System capability • May include factors such as loss of data, downtime, recoverability, etc. far@enel.ucalgary.ca

7. FSC: Common Classification • Common classification criteria: Environment • May include factors such as harmful to environment, loss of wild life, etc. • Applicable to nuclear, chemical industry, etc. far@enel.ucalgary.ca

8. FSC: Common Classification • Common classification criteria: Human life • May include factors such as harmful to human or environment, loss of human life, etc. • Applicable to aeronautical, automotive, nuclear, health care industry, military systems, etc. far@enel.ucalgary.ca

9. How to Define FSC • Experience based. • List all factors that may be considered as failure severity for the project • Narrow the list down to the most critical and/or measurable ones • Some factors may be hard to measure, such as impact on company reputation, etc. far@enel.ucalgary.ca

10. Failure Intensity Objective • Failure intensity is defined as failure per natural units, e.g. • 3 alarms per 100 hours of operation. • 5 failures per 1000 print jobs, etc. • Failure intensity of a system is the sum of failure intensities for all of the components of the system. far@enel.ucalgary.ca

11. How to Set FIO /1 • Mainly experience based. • Depends on the project. • Depends on the trade-off among quality characteristics (development time and development cost) and functionality and technology. far@enel.ucalgary.ca

12. How to Set FIO /2 • Typical FIO for various projects far@enel.ucalgary.ca

13. How to Set FIO /3 • Setting FIO in terms of reliability λ is failure intensity R is reliability t is natural unit (time, etc.) • For λ=0.001 reliability is around 0.992 for 8 natural units. far@enel.ucalgary.ca