Chapter 14 Testing Reusable Software Components in Safety-Critical Real-Time Systems

1. Chapter 14 Testing Reusable Software Components in Safety-Critical Real-Time Systems Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

2. Overview • Introduction • Reuse and Exhaustive Testing • Reuse and Statistical Evidence • Component Reuse, Statistical Evidence and Failure Behavior Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

3. Introduction • How dynamic verification of real-time software relates to component reuse in safety-critical real-time systems. • Re-testing cannot be eliminated in general. • Ariane 5 • Therac 25 • Contract • Pre-conditions • Post-conditions • Invariants Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

4. Reuse and Exhaustive Testing • Provide evidence based on the component’s: • Contracts, • Experience accumulated, • That a component can be reused immediately, • That only parts can be reused or that it cannot be reused. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

5. First Use Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

6. New Environment Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

7. Overlapping Input Domain Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

8. Pre- and Post-conditions Telephone A 0...1027 345...640 G...P Pre-condition ( (0 £ input1 £1027) && (”G” £input2 £”P”) ) // pre-condition statement 1; . . . statement n; Post-condition(345 £output £640 ) // post-condition A component with Pre- and Post-conditions Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

9. Updated Pre- and Post-conditions Telephone B -17...778 5...123 A...F Pre-condition ( (-17 £input1 £1027) && (”A” £input2 £”P”) ) // pre-condition statement 1; . . . statement n; Post-condition (45 < output < 640 ) // post-condition A new environment would violate the pre- and post-conditions unless they are updated Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

10. Reliability and Confidence for a Input Domain R(c) C(c) I(c) 0 1027 A graph representing the reliability and the confidence for a input domain Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

11. Lower Reliability Requirements R(c) C(c) I(c) 0 1027 A component reused in a context with lower reliability requirements Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

12. Reaching Desired Reliability R(c) C(c) I(c) 0 1027 The component must be run for a longer time to reach the desired reliability Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

13. Previously Experienced Reliability R(c) C(c) I(c) 0 1027 Previously experienced reliability cannot be utilized if input domains are outside historical use of the component Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

14. Component Reuse, Statistical Evidence and Failure Behavior • Failure • The inability of a system or component to perform its intended function as defined by the specification. • A failure is a consequence of a fault, which has been executed. • When a fault in a computer program is executed an error arise. • Finally, if the error propagates and becomes externally visible for an observer of a system or component, a failure occurs. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

15. Byzantine and Arbitrary Failures • This failure mode is characterized by a non-assumption: • Meaning that there is absolutely no restriction with respect to which effects the component user may perceive. • The failure mode has therefore been called malicious or fail-uncontrolled. • This failure mode includes two-faced behavior: a component can output “X is true” to one component user, and “X is false” to another component user. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

16. Sequential Failure Behavior • Control failures: • Selecting the wrong branch in an if-then-else statement. • Value failures: • Assigning an incorrect value to a correct (intended) variable. • Addressing failures: • Assigning a correct (intended) value to an incorrect variable. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

17. Sequential Failure Behavior • Termination failures: • A loop statement failing to complete because the termination condition is never satisfied. • Input failures: • Receiving an (undetected) erroneous value from a sensor. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

18. Failure Behaviors R(c) C(c) Addressing failure Failure behavior The confidence in the measured reliability is decreased when new failure behaviors can develop Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

19. Timing Failure Behavior • This failure mode yields a correct result (value), although the procurement of the result is time-wise incorrect. • For example, deadline violations, start of task too early, incorrect period time, too much jitter, too many interrupts. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

20. Deadline Requirements • If we reuse a component with only a deadline requirement in a new environment in which the execution time is shorter, the component can be reused without re-testing. Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

21. Deadline Requirements R(c) C(c) Worst case execution time New old The deadline requirement is still fulfilled since the new execution time is shorter Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems

22. Response Time R(c) C(c) Tol min Tol Max Response time The response time for the reused component is within the tolerance Building Reliable Component-based SystemsChapter 14 - Testing Reusable Software Components in Safety-Critical Real-Time Systems