An Update Of the ICAO Universal Safety Oversight Audit Programme

1. An Update Of the ICAO Universal Safety Oversight Audit Programme

2. Outline • Background • Establishment of USOAP • Universal programme - principles and mandate • Programme activities and personnel • Audit Findings and implications • Future of the program COSCAP-NA Briefing on USOAP

3. Safety Oversight - a definition • Safety oversight is a function by which States ensure the effective implementation: • Standards and Recommended Practices (SARPs) • The critical elements of a safety oversight system • Relevant safety practices and procedures COSCAP-NA Briefing on USOAP

4. Why an ICAO programme? • Increased concern over the level of safety world-wide • Various reports in the early 1990s on the lack of Implementation of ICAO SARPs by States • Need to reduce accident rates to off-set the rapid increase in traffic COSCAP-NA Briefing on USOAP

5. Directors General of Civil Aviation Conference • Montreal, November 1997. • Attended by DGCAs from 147 Contracting States and one non-Contracting State, and by 13 international organizations. • Objective: to develop a global strategy on safety oversight. • 38 recommendations in all. COSCAP-NA Briefing on USOAP

6. What is audited? Compliance with: • Chicago Convention • State Regulations Conformance with: • ICAO Standards Adherence to: • Recommended practices • Related procedures • Guidance material • Relevant industry practices in general use COSCAP-NA Briefing on USOAP

7. Audits and Audit Follow-ups Completed(as of 31 October 2003) COSCAP-NA Briefing on USOAP

8. Analysis of the findings Audit Findings and Differences Database (AFDD) • Based on factual and recent data • Enables an accurate identification of deficiencies impacting on safety • Consistent with regional accident/ • incident rates • Reliable safety tool, both for ICAO and its Contracting States • 181 audit interim reports analyzed • Updated as audit follow-up validation missions are completed COSCAP-NA Briefing on USOAP

9. APAC/33 – 24.9% GLOBAL/181 – 28.8% EUR/NAT/51 – 19.65% MID/17 – 30.52% NACC/21 – 24.04% SAM/13 – 23.24% WACAF/21 – 49.37% 70 ESAF/21 – 40.2% 60 50 40 30 20 10 0 USOAP – findings and analysis Lack of effective implementation of the critical elements (%) Primary Aviation Legislation Specific Operating Regulations CAA Structure & Safety Oversight Functions Technical Guidance Material Qualified Technical Personnel Licensing & Certification Obligations Continued Surveillance Obligations Resolution of Safety Issues COSCAP-NA Briefing on USOAP

10. What does this all mean? • What is the impact of the results of the audits on safety? • What can and should be done? COSCAP-NA Briefing on USOAP

11. Audit Findings – Lack of Effective Implementation of SARPs, (178 audit reports) Accident rates per 1 000 000 departures - scheduled (International and Domestic, 1996) Accident rates per 1 000 000 departures - scheduled (International and Domestic, 2000) Parallel between findings and regional accident rates 80 70 60 50 40 30 20 10 0 Asia and Pacific Region Eastern and Southern African Region European and N. Atlantic Region N. / C. American and Caribbean Region Southern American Region Western and Central African Region Middle East Region COSCAP-NA Briefing on USOAP

12. Critical Elements of a Safety Oversight System - Lack of Effective Implementation Comparison of Audit and Audit Follow-up Results Global -Initial Audit = 30.3% 85 States - Initial Audit = 22.7% Global - Revised = 21.4% 85 States - Follow-up = 7.7% 40 35 30 25 20 15 10 5 0 Primary Aviation Legislation Specific Operating Regulations CAA Structure and Oversight Functions Technical Guidance Material Qualified Technical Personnel Licensing and Certification Obligations Continued Surveillance Obligations Resolution of Safety Issues COSCAP-NA Briefing on USOAP

13. COSCAP-NA Briefing on USOAP

14. Future of the Programme • Complete the remaining follow-up missions to validate the implementation of States’ corrective action plans • Continue the implementation of the SOA Quality Management System • Expansion of the Programme COSCAP-NA Briefing on USOAP

15. Expansion to Annexes 11, 13 and 14 • Expansion due to start in 2004 • Provisions in Annexes 11 and 14 are highly interrelated with those in other Annexes (1,2, 4, 10, 12 and 15) • Annexes 11 and 14 cannot be audited in isolation. Related provisions in other Annexes must be considered • Expansion also entails the provision of: • Seminar/workshops • Guidance material COSCAP-NA Briefing on USOAP

16. ICAO Auditing practices to-date • Annex-by-Annex approach until now. • Adding new Annexes would inflate the resource requirements of the Programme • Lengthy and expensive proposition • Momentum lost when area of concentration changes • Entails replacing staff – and losing expertise at every audit cycle. Cannot keep receiving and monitoring progress from States • Detrimental to the continuity and credibility of the Programme COSCAP-NA Briefing on USOAP

17. Systems approach • A Systems Approach for the conduct of audits is being proposed for the ICAO Safety Oversight Audit Programme • It moves away from the current approach of Annex-by-Annex auditing • It will focus both on Oversight capability of States and the safety oversight critical elements • It will be designed to cover all safety-related Annexes and will provide a more objective and cost-effective approach to auditing COSCAP-NA Briefing on USOAP

18. Systems approach to conducting audits • Implementation of Annex provisions and status of differences would be determined BEFORE the audit mission, through a series of tools and relevant reports • A review of the State’s legislation, regulations, documentation and organization would also be done BEFORE the mission • The final assessment of the organization and the remaining critical elements of a safety oversight system would be made on-site COSCAP-NA Briefing on USOAP

19. Systems approach to conducting audits • On site audit would validate the organization, procedures and processes established to help the State fulfill its safety oversight obligations • Audits will be tailored to the level of activity and complexity of aviation activities in each State COSCAP-NA Briefing on USOAP

20. Systems approach to conducting audits • All Contracting States to be audited at least once in a six-year period • Number, duration, frequency of visits and team size tailored to each State • Core audit staff at Headquarters complemented with Regional Office and State-seconded staff will conduct the audits • Systems approach offers cost advantages over Annex-by-Annex approach as the cost per Annex audited will be significantly reduced COSCAP-NA Briefing on USOAP

21. Conclusions • USOAP’s activities related to Annexes 1, 6 and 8 cannot be abandoned. States have a vested interest and ICAO’s credibility would be at stake • Audits in the expansion areas of Annexes 11, 13 and 14 cannot be conducted in isolation • Support to States with seminar/workshops and guidance material must be continued and increased COSCAP-NA Briefing on USOAP

22. Conclusions • Audits based on the systems approach can be launched in 2005 with a six-year cycle schedule • Council has accepted this concept and the Assembly will be invited to endorse it • The Preparatory work for launching the Systems Approach can be finalized in 2004 • If the systems approach is endorsed there will be no need to start audits relating to Annexes 11, 13 and 14, in 2004 COSCAP-NA Briefing on USOAP

23. ICAO Universal Safety Oversight Audit Programme THANK YOU