Safety - definitions

1. Safety - definitions Accident - an unanticipated loss of life, injury, or other cost beyond a pre-determined threshhold. • If you expect it, it’s not an accident. • Cost is high - generally higher than the value of the service provided by the system. Safety - absence of accidents.

2. What is a system? • Black box view • environment • Glass box view (structure) • Collection of components (which are themselves subsystems) • Software components • Computers • Sensors, actuators • Humans • Communications • System structure can be dynamic

3. Role of software in system safety • Software can be a source of failures • Software can deal with error diagnosis and recovery. • Interaction between software and rest of system can affect safety (e.g., GUI issues).

4. System safety Principle: software safety is meaningless without consideration of system-wide effects. • GUI confuses operator. • Wrong sensor reading causes inappropriate response from software. • Bad error recovery code aggravates error. • Loading the wrong software.

5. Hazards • Hazard - a condition that inevitably leads to an accident under some combination of environmental conditions (Leveson). • Hazard may or may not lead to accident • . . . but it is a matter of luck - assumes system cannot control environment. • Example: Two aircraft are separated by less than minimum required distance. • If exact position and velocity is “right”, they will collide.

6. Hazard identification • Cannot make a system safe unless we know how it can be unsafe. • Hazard identification attempts to make explicit the safety risks of the system • This requires thinking about the system globally as well as locally • Should begin in early phases of design, when problems are relatively easy to fix.

7. Hazard identification, cont • After identification • System can be redesigned to remove hazard • . . . or reduce probability • . . .and/or reduce effects of hazard. • Hazards should be tracked • Hazard analysis should be updated • To reflect design and implementation changes • Increased understanding and experience (e.g. from observed accidents or malfunctions). • Also, remedies for hazards should be logged.

8. Hazard identification methods • There is no magic bullet • Draw on past experience • Study problems from previous systems. • There may be checklists of common hazards. • “Structured brainstorming” • Form a committee of experienced people with a variety of expertise and viewpoints • Present the system in various ways • Try to think of possible problems

9. Preliminary Hazard Analysis (PHA) • Goal: Early identification of hazards so they can be taken into account in system specification and design. • However, analysis should be updated to reflect increased understanding and design changes. • Inputs • System design objectives • equipment specs • environmental conditions • regulations

10. <<Space shuttle example>>

11. Hazard Level: severity and likelihood Example DoD MIL-STD-882B • Category I: Catastrophic: may cause death or loss of system. • Category II: Critical: May cause severe injury, severe occupational illnusess, or major system damage. • Category III: Marginal: may cause minor injury, minor occupational illness, or minor system damage. • Category IV: Negligable: will not result in injury, occupational illness, or system damage.

12. Hazard likelihood British Std. 00-56 • Frequent: Likely to be continually experienced. • Probable: Likely to occur often. • Occasional: Likely to occur several times. • Remote: Likely to occur some time. • Improbable: Unlikely, but may exceptionally occur. • Incredible: Extremely unlikely that the event will ever occur.

13. Hazard Aversion/Containment • Hazard elimination: e.g., robot arm is too slow to break a hole in the wall. • Hazard reduction: e.g., add interlocks • Hazard control: e.g., pressure valve • Damage reduction: e.g., evacuation procedures.

14. PHA evaluation Advantages • Focuses on hazards Disadvantages • Determining hazard categories may be hard. • May make unrealistic assumptions about engineering, testing, operational procedures, etc.

15. HAZOP • Goal: To establish, by systematic examination of system components and operations, failure modes which can lead to hazards • When: during design • Inputs: results of previous safety analysis • Outputs: identification and elimination of events and failure modes leading to hazards.

16. HAZOP guide words (00-58) • No: Complete negation of design intention. No part of the intention is achieved, nothing else happens • More: a quantitative increase • Less: a quantitative decrease • As well as: Design intent achieved, with additions • Part of: Only some of design intention is achieved. • Reverse: The logical opposite of the intention is achieved.

17. HAZOP evaluation Advantages • Simplicity and ease of application • Combines members from different teams Disadvantages • Labor intensive • Results vary depending on quality of team.

18. FMECA (FMEA) “Failure Modes, Effects (and Criticality) Analysis” Goal: Systematically analyze the components of the target system with respect to attributes relevant to safety. When: Early design, after components identified. Input: Schematics, functional diagrams, component relationships Output: List of failure modes and effects.

19. FMECA example/more info

20. FMECA evaluation Advantages • Effective for analyzing single units • Completeness • Used to identify redundancy. Disadvantages • Multiple failures/common mode failures overlooked. • Time consuming and costly • Failure modes must be known in advance. • Hard for complex components