USOAP Comprehensive Systems Approach

1. By COSCAP-NA USOAP Comprehensive Systems Approach Module 3

2. Contents • From an Annex-based to systems-based audits • The Comprehensive Systems Approach • Tools used in the implementation of the comprehensive systems approach Introduction to USOAP

3. An Annex by Annex-based approach • An Annex by Annex-based approach was practiced from 1996 – start of the voluntary assessment to 2004 – end of first cycle of mandatory audits • Experience shows that it was successfully implemented and served the intended purpose: • Raised State awareness of their safety oversight responsibilities; • Provided reliable information on the status of implementation of ICAO SARPs; • Enabled the identification of deficiencies in the global safety of aviation activities in the three areas Introduction to USOAP

4. Initial Audit - 181 Contracting States (32.62%) Audit Follow-up – 181 Contracting States (17.46%) Graphic presentation of the results of the first audit cycle 40 30 20 10 0 Primary Aviation Legislation Specific Operating Regulations Civil Aviation System And safety Oversight Functions Qualification and Training of Technical Staff Procedures and Technical Guidance Licensing and Certification Obligations Surveillance Obligations Resolution of Safety Issues Introduction to USOAP

5. An Annex by Annex-based approach • The challenge represented: • Aviation activities are interrelated. A problem in one area affects the “overall” safety of the whole system; • Safe aircraft operation is not limited to licensing, operations and airworthiness; • State oversight capabilities with respect to other aviation activities have an equal impact on safety; Introduction to USOAP

6. An Annex by Annex-based approach • The challenge represented: • Continuing on an Annex-by-Annex approach would have been very lengthy and expensive; • It would also represent an administrative problem, replacing experts on a three-year or six year cycle; • Overall picture of the health of the aviation system would have been impossible to assess. Introduction to USOAP

7. Comprehensive systems approach • A systems approach: • Looks at States’ safety oversight systems from a process perspective; • Provides a timely picture of the overall health of a State’s safety oversight system and its effectiveness; • Focuses attention on criticalsafety areas; • Allows for flexibility depending on the size and complexity of the aviation system established in a State. Introduction to USOAP

8. Comprehensive systems approach Endorsed by Assembly Resolution A35-6 (2004) • USOAP to be further expanded to include the safety-related provisions in all safety-related Annexes; • USOAP to adopt a comprehensive systems approach in conducting safety oversight audits; • PEL, OPS, AIR, ATS, AGA and AIG to constitute the core elements (subjects) of the audit Programme. Introduction to USOAP

9. Comprehensive systems approach Endorsed by Assembly Resolution A35-6 (2004) • Audit reports to reflect the critical elements of a safety oversight system; • Final audit reports to be made available to all Contracting States; • USOAP to continuously ensure the maintenance of the quality assurance system. Introduction to USOAP

10. Comprehensive systems approach Audit tools: • Audit tools are used throughout the three phases of the audit process starting with the initial planning stage to the publishing of the final audit report; • Audit tools developed include: • SOA Quality Manual; • State Aviation Activities Questionnaire (SAAQ); • Compliance Checklist (CC); and • Audit Protocols Introduction to USOAP

11. Comprehensive systems approach State Aviation Activities Questionnaire (SAAQ) • Designed to enable ICAO to collect useful information on the organization and system established by a State to meet its safety-related obligations as a signatory to the Convention on International Civil Aviation. Introduction to USOAP

12. Comprehensive systems approach State Aviation Activities Questionnaire (SAAQ) • Completed SAAQs to be submitted to ICAO by 31 May 2005 (extended for some States on request); • States to update information contained in their respective SAAQ whenever there is a change in their civil aviation system; • An updated SAAQ to be submitted at least 90 days prior to the conduct of the on-site phase of the audit; • SAAQ questionnaire available at:http://www.icao.int/icaonet/ Introduction to USOAP

13. Comprehensive systems approach Compliance Checklist: • A tool which will provide ICAO with information on a State’s level of implementation of international Standards and Recommended Practices (SARPs); • A tool which will enable States to identify any difference which may exist between their own practices and those established by international standards(Article 38 of the Chicago Convention) Introduction to USOAP

14. Comprehensive systems approach Compliance Checklist: • States submitting a completed compliance checklist will enable ICAO to maintain a database on a States level of compliance to the ICAO SARPs; • States are required to maintain the compliance checklist up-to-date and/or update the compliance checklist at least 90 days prior to the scheduled audit. Introduction to USOAP

15. Comprehensive systems approach Audit Protocols (PQ): • Developed by SOA with the assistance of relevant ANB Sections; • Used for the conduct of the on-site audit; • Enables auditing against the critical elements of a safety oversight system; • Provides guidelines to the auditor on what evidence should be reviewed; • Could also be used by States to conduct an internal audit (Doc 9735). Introduction to USOAP

16. Comprehensive systems approach Audit Protocols (PQ): • Use of standardized protocols as the primary tool for the conduct of the on-site audit ensures transparency, consistency and standardization; • It enhances confidence and reliability in the conduct of audits; • Designed to reflect the core areas of the audit Programme. Introduction to USOAP

17. Comprehensive systems approach Areas covered by the Audit Protocols: • Primary Aviation legislation and civil aviation regulations • State aviation system and safety oversight functions • Personnel licensing system • Operation of aircraft • Airworthiness of aircraft • Aircraft accident and incident investigations • Air navigation services • Aerodromes Introduction to USOAP

18. INTRODUCTION TO USOAP By COSCAP-NA Questions ? Module 3