Safety-Critical Systems 5 Testing and V&V

1. Safety-Critical Systems 5Testing and V&V T 79.232

2. Requirements Model Requirements Analysis Test Scenarios Test Scenarios System Acceptance Requirements Document Functional / Architechural - Model System Integration & Test Systems Analysis & Design Specification Document Knowledge Base * Software Design Module Integration & Test Software Implementation & Unit Test * Configuration controlled Knowledge that is increasing in Understanding until Completion of the System: • Requirements Documentation • Requirements Traceability • Model Data/Parameters • Test Definition/Vectors V - Lifecycle model

3. Testing Testing is a process used to verify or validate system or its components. Testing is performed during various stage of system development. V-lifecycle diagram. Module testing – evaluation of a small function of the hardware/software. System integration testing – investigates correct interaction of modules. System validation testing – a complete system satisfies its requirements.

4. Testing forms Dynamic testing - execution of the system or component in the natural/simulated environment. Functional – test all functions Structural – test signal/test cases (glass-box) Random – n-dimensional input space Static testing - reviews, inspections and walkthroughs. Static code analysis for software. Modelling - mathematical representation of the behaviour of a system or its environment.

5. Testing methods Black-box testing – requirements-based, no information of the system, what is inside. White-box testing – more information about the system design to guide testing. Open view glass box.

6. Dymanic testing techniques Dynamic testing standards IEC1508, BCS (British Computer Society) and DO-178B. Process simulation Error seeding/guessing Timing and memory tests Performance/stress testing Probabilistic testing – values for failure rates

7. Test planning Lifecycle Phase Activity Safety case Requirements Hazard identification Analysis results Test planning Identify tests integrity Strategy for V/V Req/Design/Test Trace hazards to specs. Risk reduction Req/Design Define specs Design analysis Safety Functional Requirements are the actual safety- related functions which the system, sub-system or item of equipments required to carry out. (Cenelec)

8. Operational Thread: Elaboration of operational requirements in textual form Elaboration of requirements model based on operational requirements Until validated: Validate requirements model against operational requirements Update model as needed Transformation of the requirements model into a verifiable form Safety Thread: Identification of safety requirements based on hazard analysis Formalization into a safety model based on safety requirements Until validated: Validate safety model against hazard analysis Update model as needed Transformation of the safety model into a verifiable form • Until verified: • Model verification based on safety an liveness requirements • Update model as needed Development Process for V & V

9. Validation Textual requirements Operational requirements Terms & definitions Safety requirements Validation Informal Verification Validation Informal Verification (Initial) Requirements model Use cases & control cases Domain objects Safety properties Formal Verification Important interactions Dynamic behavior (Final) Requirements model Development Process for V & V

10. e.g. „What use cases are available to the signaler?“ Domain Expert Requirements Modeling Language Question Validation Support Tool Validator Confirmer Requirements Model • e.g. • prepare to train arrival • set reserved path • monitor situation Model Validation

11. Validation/Confirmers • Confirmer: A property of a system derived from a model and subject to human evaluation. • Types of confirmers: • Static, derived (i.e. implicit) model information (e.g. implicit use cases or required conditions for a transition/action) • Dynamic state requests („is the model now in the right state?“) • Dynamic event responses („does the model react correctly?“) • Possible representations for confirmers: • Natural language sentences • Algebraic expressions • Traces / sequence diagrams • Dynamic simulation

12. e.g. „A point may never move when a route is locked.“  Challenger Domain Expert Requirements Modeling Language Proof Verifier Verification Support Tool • e.g. challenger is false in the following case: • User: set route A • System: steer point 1 left • HW: point 1 at left • User: set point 1 right • System: steer point 1 right • CONFLICT!!! Requirements Model Model Verification

13. S S S S t t t Languages of Logic • Propositional LogicStatements • (1st Order) Predicate Logic (FOPL)Statements quantified (, ) over things (objects!) • Linear Temporal Logic (LTL)Statements quantified (, , G, F, H, P) over things and time • Computational Tree Logic (CTL)Statements quantified (, , G, F, H, P, , ) over things, time and worlds (modal logic) • Enhanced Regular Expression Logic (ERE)Statements about occurrence patterns (seq, sel, itr, par) of events and conditions causing actions • Note: The list above is neither complete nor it does necessarily imply any hierarchy!

14. (Some) Languages of Logic CTL ERE? Time G, F, H, P Temporal Logic (LTL) Worlds ,  Modal Logic DL Predicate Logic Objects ,  Propositional Logic

15. Model Checking Theorem Proving CTL ERE? Time G, F, H, P Temporal Logic (LTL) Worlds ,  Modal Logic DL Predicate Logic Objects ,  Propositional Logic Verification Technologies

16. Testing and V&V Home assignments: - 12.7 Dynamic testing 12.20 Constructed environment Please email to herttua@eurolock.org by 12 of May 2005 References: KnowGravity, I-Logix