Safety Assurance in Risk Management for Federal Aviation Administration (FAA)

1. Policy (Structure) Risk Management Safety Assurance Safety Promotion Federal Aviation Administration SMS Details: Safety Assurance Component

2. Objective of Safety Assurance • Assurance: “something that gives confidence”1 • Safety Assurance: “something that gives confidence in safety risk controls.” 1 Black’s Law Dictionary

3. Safety Assurance Functions: • Collect and analyze information to determine that process requirements are continuously being met. • Assess performance and effectiveness of risk controls. • Works in partnership with Safety Risk Management (SRM). AC 120-92

4. S.A. is similar to Q.A. • QA focuses on product conformity & customer satisfaction on a continuous basis. • SA ensures that risk controls, once designed and put to place, perform in a way that continue to meet their safety objectives. • Integration of management systems may be beneficial. AC 120-92

5. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action System Monitoring Written documentation to describe: Who, What, When, Where, Why, How • The system monitoring includes: • System description, including risk controls added during SRM. This forms the basis for SA functions such as audits and analysis. • Monitoring of risk controls during operations, AC 120-92

6. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Data Acquisition and Processing Types of Data Sources • Continuous Monitoring • Internal Audits • Internal Evaluation • External Audits • Investigations • Employee Reporting Systems SMS Framework Elements 3.1.1 through 3.1.6

7. Accomplish continuous monitoring of day-to-day activities & processes Have direct responsibility for process control Must ensure that processes in their areas function as designed. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Continuous Monitoring Where SRM and SA interface - risk controls Line managers of operational departments:

8. Continuous Monitoring - Operational Data Sources • Flight dispatch records • Flight schedules • Financial data • Crew schedules and records • Warranty return reports • Aircraft discrepancy reports • Flight cancellation and delay reports

9. This is where: Deficiencies in processes contribute to risk Audits provide feedback to process owners Direct supervisory control and resource allocation can help to maintain effectiveness of risk controls System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Internal Audits The day-to-day responsibility for safety management rests with those who “own” the technical processes.

10. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Internal Audit Design • Performed by each department. • Department Director/Manager is responsible. • Regularly scheduled • Include contractors & vendors • Determine: • Conformity with safety risk controls • Performance of safety risk controls • Performance to meet safety objectives • Deficiencies always get action! AC 120-92

11. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Internal Evaluation • Performed by a functionally independent person or organization (e.g. QA, Safety) • A process-oriented control function • Backs up the internal audit function • Uses sampling to validate SA processes

12. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action External Audits Conducted by: • Code-share partners • Industry organizations (e.g. C.A.S.E.) • Third parties: consultants • The regulator (FAA) = “Safety Oversight”

13. Safety investigations Traditional Approach – put the event behind us (everything is okay): To reassert trust and faith in the system (as is) To resume normal activities To fulfil political purposes SMS Approach - Improved system reliability: To learn about system vulnerability To develop strategies for change To prioritize investment of resources System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action

14. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Employee Reporting • Employee safety reporting is required & a feedback system is encouraged. • Must provide confidentiality. • Employees must be encouraged to use the system. • Data may identify emerging hazards. • Data must be included in analysis. AC 120-92, App. 1

15. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Types of analysis • Against criteria/objectives • Compared to “norms” • Patterns from multiple data points • Trends over time • “Trends” is one of the most misused term in analysis • Must have stable, reliable measures at each time sample for a valid trend

16. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action System Assessment • Are objectives being met? (“Happy loop”) • Risk controls failing due to: • Lack of supervision • Lack of resources • Lack of training • Poor job aids • New Hazard/failed Risk Controls (Redesign - back to SRM) • Prioritize according to safety criticality (triage)

17. System Operation Data Acquisition & Process Analysis System Assmt Preventive/Corrective Action Preventive/Corrective Actions • Increased supervision • Emphasized procedures • Equipment maintenance • Remedial training • Schedule changes • Personnel assignments • Health and wellness

18. Management Review Top management must conduct regular reviews of the SMS, including: • The outputs of SRM & SA • Lessons learned • Need for changes

19. Continuous Improvement The organization shall continuously improve the effectiveness of the SMS through: • Safety and Quality Policies • Safety Objectives • Audit & Evaluations • Analysis of Data • Corrective and Preventive Actions • Management Reviews AC 120-92

20. Internal Evaluation SA Process Circle Diagram Continuous Monitoring: Day to day, flight by flight Job by job – supervise! • Design/Re-design • Changes • Failures • Back to SRM Internal/External Audits: Go out and look Analysis, Assessment, & Decision Employee Reporting: Listen the front line Preventive/Corrective Action: Getting back on track Affirmed: Expectations met Investigation: Learn from error