Draft CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes

1. Draft CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes “Quality for Tomorrow Begins Today” NAM/CAR/SAM Air Traffic Services Quality Assurance Programme Seminar Mr. Mark Rios Regional Officer, Air Traffic Management ICAO NACC Regional Office (16 to 20 October 2000, Mexico City, Mexico)

2. INTERNATIONAL CIVIL AVIATION ORGANIZATION NORTH AMERICAN, CENTRAL AMERICAN AND CARIBBEAN OFFICE CAR/SAM REGIONAL GUIDANCE MATERIAL ON AIR TRAFFIC SERVICES QUALITY ASSURANCE PROGRAMMES DRAFT Version 0.1 October 2000

3. THIS MATERIAL IS UNDER DEVELOPMENT AND IS SUBJECT TO CHANGE BEFORE ADOPTION BY GREPECAS AS REGIONAL GUIDANCE MATERIAL

4. Forward • CAR/SAM/3 RAN (October 1999) • ATS Quality Assurance Programme • Purpose • ATS Incidents • Quality of Services • CAR/SAM/3 Recommendation 5/36 • ICAO develop guidance material on ATS quality assurance programmes for worldwide use • CAR/SAM Regional Guidance Material on ATS QA Programmes • first phase

5. Contents • Chapter 1. Definitions • Chapter 2. Background • Chapter 3. ATS Quality Assurance Programmes • Chapter 4. Controller Proficiency Checks • Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Chapter 6. Air Traffic Incident Prevention Programmes • Chapter 7. Air Traffic Services Evaluation Programme • Chapter 8. Quality of Services Improvement Programmes • Chapter 9. Proficiency Training Programmes • Chapter 10. ATS Safety Management • Chapter 11. Human Factors

6. ATC proficiency training ATS operational duties Desk audit Follow-up ATS evaluation Full-unit ATS evaluation In-flight/preflight evaluations Operational deviation Operational error Pilot deviation Refresher training Runway incursion Simulation training Skill enhancement training Special evaluations Supplemental training Three-step closure process Chapter 1. Definitions New Definitions

7. Chapter 2. Background • Important to prepare ATS QA programme guidelines for States • Heightened awareness of ATS incidents and recommendations made by: • CAR/SAM/2 RAN (Santiago, 1989) • Recommendation 6/11 – Investigation of ATS Incidents • Recommendation 6/12 – Reporting of ATS Incidents • GREPECAS 8 (Dominican Republic, 1998) • Conclusion 8/7 – Actions to be taken by States to reduce ATS incidents in the CAR/SAM Regions • Conclusion 8/8 – Actions to be taken by the users to reduce ATS incidents in the CAR/SAM Regions • Conclusion 8/9 – Actions to be taken by the ICAO Regional Offices to reduce ATS incidents in the CAR/SAM Regions

8. Chapter 2. Background • Informal working groups • International organizations • Airlines • Reports received by ICAO Regional Offices • Increase of ATS incidents by 43 per cent between 1996 and 1998 • Justified the need for introduction of ATS QA programme for the CAR/SAM Regions • ICAO NACC/SAM Regional Offices began work programme

9. Chapter 3. ATS Quality Assurance Programmes • Introduction • ATS QA programmes should focus on Identifying and correcting deficiencies before they result in an ATS incident or accident; and • Should continually work towards improving the overall quality of air traffic services • Chapter contains proactive quality assurance strategies

10. Chapter 3. ATS Quality Assurance Programmes • Scope and purpose • Specific guidance on reporting, investigating and resolving various types of ATS incidents that impact the quality of ATS • Designed to work in conjunction with • ICAO SARPs • State’s regulations • Objectives • Prevent ATS incidents • Continually improve overall quality of air traffic services

11. Chapter 3. ATS Quality Assurance Programmes • Structure • Depends on size and composition of ATS provider • Acceptable and productive programme structure may involve • ATS chief designating/selecting experienced ATC expert to be ATS unit quality assurance officer/specialist • ATS unit QA specialist assumes duties and responsibilities of QA for ATS unit • Larger ATS units could develop ATS QA unit/department • Several ATS QA specialists • Assistant chief with extensive ATC experience

12. Chapter 3. ATS Quality Assurance Programmes • Implementation • CAR/SAM/3 Recommendation 5/38 – Implementation of an ATS quality assurance programme That, a) States/ATS service providers within the CAR/SAM Regions implement an ATS quality Assurance programme with associated supporting documents. The programme should outline the purpose, goals, objectives and responsibilities of the programme within the State/ATS service provider; and b) Each ATS unit should establish such a programme

13. Chapter 3. ATS Quality Assurance Programmes • Responsibilities • ICAO NACC and SAM Regional Offices • provide guidance and assistance to States/ATS service providers in the development of ATS quality assurance programmes • States/ATS service providers • Should implement an ATS quality assurance programme with associated supporting documents • Programme should outline the purpose, goals, objectives and responsibilities within the State/ATS service provider and each ATS unit should establish a programme • National ATS QA programmes are maintained and evaluated for effectiveness

14. Chapter 3. ATS Quality Assurance Programmes • ATS unit chiefs • Maintain a level of awareness and involvement in their ATS unit’s operations/programmes so as to ensure maximum quality and efficiency • All ATS personnel • Responsible for maintaining the highest level of quality performance

15. Chapter 3. ATS Quality Assurance Programmes • Programme content • ATS QA programmes should establish methods to identify and correct shortcomings and recognize successes in the following areas: • ATS incident prevention • Teamwork • Communications • Customer service/feedback

16. ATS incident prevention ATS refresher training Aeronautical phraseology improvement English language proficiency Hearback/readback ATS surface incident Incentive/recognition List of good operating practices Recordings monitor evaluation Simulation training Personal accounts of lessons learned Periodic quality assurance briefings in ATS units covering trends, customers input, evaluations, etc. Aggressive resolution of problems identified Incorporation of past ATS incidents scenarios into training Internal, national, and regional ATS evaluation Chapter 3. ATS Quality Assurance Programmes ATS incident prevention

17. Air traffic service teamwork training Teamwork incentive/recognition programmes Roles of different positions/jobs ATS Operational Supervisor training course Team meetings/briefings Expectations of all employees clearly communicated Chapter 3. ATS Quality Assurance Programmes Teamwork

18. Electronic bulletin board system Internet/intranet access to data National database containing national and local ATS QA data Newsletters ATS QA seminars, conferences and workshops All staff meetings Reports from International organizations, i.e. IFATCA, IATA, IFALPA, PAAST, etc., and other industry safety reports Chapter 3. ATS Quality Assurance Programmes Communications

19. Pilot education programmes Surveys of internal and external customers Interaction with other aviation organizations Air traffic controller/Operational Supervisor evaluation of shift performance All staff meetings Familiarization flights Contacts with customer organizations (i.e. local flight schools, airlines, aviation organizations, etc.) Pilot safety seminars and airport management workgroups Chapter 3. ATS Quality Assurance Programmes Customer service/feedback

20. Chapter 4. Controller Proficiency Checks • Introduction • To provide continuous enhancement of technical proficiency in the ATS environment, individualized training requirements for technical performance should be identified and accomplished • Controller proficiency checks are intended to provide feedback by ATS supervisors/ATS QA officers/specialists to controllers and supervisors regarding their proficiency, and to develop plans to enhance their proficiency

21. Chapter 4. Controller Proficiency Checks • Responsibilities • ATS unit chiefs to establish and maintain ATS unit proficiency standards • ATS authority should develop guidelines specifying required level of knowledge (theoretical and practical) • ATS operational personnel should be required to periodically demonstrate that their performance meets proficiency standards • Controller proficiency checks should be accomplished on each controller and supervisor

22. Chapter 4. Controller Proficiency Checks • ATS unit chief, supervisor or quality assurance specialist should conduct task of controller proficiency checks • Develop proficiency check rosters to ensure all ATS operational personnel are checked biannually • ATS operational supervisors and QA specialist should continuously assess controllers’ performance through direct or indirect methods • Indirect – remote monitoring, voice recordings, observations by other supervisors or ATS unit QA specialists

23. Chapter 4. Controller Proficiency Checks • What if results of controller proficiency check indicates controller would benefit from proficiency training? • Use references to determine type of training needed: • ICAO Doc 9426, Air Traffic Services Planning Manual, Chapter 3 – Training and Proficiency Requirements • Draft CAR/SAM Regional Guidance Material for ATS QA Programmes, Chapter 9. Proficiency Training • Assignment of proficiency training may not necessarily mean a deficiency was noted • A controller may demonstrate an acceptable level of technical performance, but might still benefit from proficiency training in a particular skill or task

24. Chapter 4. Controller Proficiency Checks • ATS supervisor or QA specialist should discuss the completed controller proficiency check with the controller • Proficiency checks are not intended to be pass/fail or satisfactory/unsatisfactory • What if a controller’s performance is found to be unsatisfactory? • Rating should be suspended and appropriate proficiency training, followed by a re-certification process should be completed

25. Chapter 4. Controller Proficiency Checks • ATS units should conduct an annual review of all controller proficiency checks with a view to identify recurring and significant proficiency needs • Documentation • Discuss proficiency check results with the controller and document in the controller’s training record

26. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Introduction • Air traffic incidents • AIRPROX • Operational error • Operational deviation • Runway incursion • Pilot deviation • Casual factor • Procedural • Ground facilities (equipment) • Human

27. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • General reporting procedures • Reporting by pilots • Reporting by ATS • ATS incident report form for use by pilots and controllers • Investigation and documentation • Analysis of ATS incident • Release of information • In the interest of accident/incident prevention • Publish report as soon as possible

28. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Air traffic incident investigation process • Step-by-step process • Suspected ATS incident event • Identify and report ATS deficiencies • That problems could be corrected • Reporting of ATS incidents • Requirement within States • Responsibility of all ATS personnel • Immediately report suspected ATS incidents to ATS supervisor or appropriate ATS administration official • Without fear of reprisal

29. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Preliminary ATS incident investigation • Fact finding • Determine what occurred in the ATS system • Determine validity of the suspected ATS incident • Confirmation of ATS incident • Take appropriate action • Advise other ATS unit if involved • Relieve controller(s) may be involved • Gather pertinent information (e.g. flight progress strips, etc.) • Review voice recordings and prepare working copy

30. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Review computer and radar data • Conduct preliminary interviews • Notify ATS management • Complete preliminary ATS Incident Investigation Worksheet • Follow-up investigation • ATS unit chief should • Ensure ATS incident investigations are conducted • Coordinate with other ATS unit chief if involved • Designate ATS incident Investigator-In-Charge (ATS-IIIC) • Assemble ATS incident investigation team • Ensure complete, thorough, objective and timely investigation

31. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • ATS-IIIC • Conduct complete investigation • Forward findings and recommendations to ATS unit chief • Determine scope of other ATS unit investigation if involved and exchange information • Assign duties to team members • Conduct interviews in accordance with national and local requirements • ATS incident investigation team • Assist ATS-IIIC • Remain under guidance of ATS-IIIC during investigation process

32. Procedures Training Supervision Equipment Control room environment External factors Controller actions Airspace configuration Traffic flows Pilot actions Consequence of ACAS/TCAS event Meteorological conditions Control position configuration Coordination procedures Aerodrome environment Human factors Accuracy of automated radar systems Radar data Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Investigation process • Fact finding • In-depth inquiry into all causal factors

33. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Investigation process (cont..) • Interviews • Conduct interviews with all possible personnel involved or who may have knowledgeable of the incident • Conduct interview in atmosphere of shared concern • Consider national and/or local interview procedures • Allow written comments and recommendations from interviewee • ATS-IIIC or investigation team should conduct interviews

34. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Investigation process (cont.) • Voice recordings • Review voice recording • Make certified copy of original recording • 5 minutes before to 5 minutes after last contact • Include track time if available • Written transcript of communications only when needed • Radar data • Radar data may be used to validate occurrence of ATS incident • Written procedures should be developed explaining process • Consider automation capabilities and accuracy of data

35. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Reclassification of ATS incident • What if during the investigation data indicates a reclassification is warranted? • The incident may be reclassified as: • Pilot deviation • Other ATS unit deviation • From operational error to operational deviation • From operational deviation to operational error • No occurrence

36. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Return to operational duties • ATS chiefs should remain involved in the post ATS incident process to ensure complete and consistent handling • Reasons for suspension of controller’s rating and assignment of proficiency training • Assist and enhance controller’s performance • Best of his/her abilities • Meet minimum quality standards • Improve overall performance and quality of ATS

37. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Return to operational duties (cont.) • Consider the following when returning a controller to operational duties • Events leading up to and surrounding ATS incident • Controller’s statement • Procedure or separation minimum involved • Voice recordings • Controller’s training record • Controller proficiency checks • Proficiency training • Controller currency on the control position • Computer data • Involvement in previous ATS incidents • Results of interviews and recommendations made by ATS-IIIC and investigation team

38. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Return to operational duties (cont.) • Deficiencies in controller’s performance • Determine whether to suspend controller’s rating • Suspension of rating not based solely on involvement in ATS incident but based on performance • Determine appropriate actions and refresher training • Complete refresher training • Re-certify controller through controller proficiency check

39. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Final ATS incident reports • ATS unit chief • Analyze data submitted by ATS-IIIC and investigation team in Final ATS Incident Investigation Report to determine and/or endorse classification of incident and • Categorization of ATS incident (i.e, procedural, facility, human factor, etc.) • Causal factors • Recommendation and corrective actions • Complete Final ATS Incident Report within 30 days

40. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • ATS incident tracking process • Annual ATS incidents tracking process • Will assist CAAs with follow-up investigations and trend analysis • Sample ATS Incident Report # XXXX-E-00-001 ATS Incident Report # XXXX-T-00-002 ATS Incident Report # XXXX-F-00-003 Legend: “XXXX” = ATS unit identification “E, T, or F” = Type of ATS unit (E = Enroute (ACC); T = Approach (APP) and Aerodrome Control Tower; F = FIC, AFIS, etc.) “00” = Last two digits of the year “001” = ATS incident number in sequence by year for the unit

41. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Training and proficiency records • Document in training and proficiency records • ATS incident records retention • Determine retention period • 2 ½ years is recommended • National ATS authority responsibilities • Maintain central source of ATS incident data • Review final ATS incident investigation reports • Develop and distribute annual ATS incident analysis report • Determine effectiveness and efficiency of programme • Maintain liaison with ATS unit chiefs • Maintain oversight of ATS incident reports

42. Chapter 6. Air Traffic Incident Prevention Programmes • Introduction • Establish policies and programmes that result in quality incident free ATS • Requires commitment of all ATS personnel • Consistently perform duties and responsibilities in accordance with ICAO SARPs, CAA and ATS unit directives and procedures • Accept responsibility for maintaining the safest and highest levels of operational performance • Goals/objectives • ATS incident free environment • Safe, orderly, and expeditious ATS to users • Everyone is responsible for ATS incident prevention • Know, apply, and adhere to requirements in performance of operational duties and responsibilities

43. Chapter 6. Air Traffic Incident Prevention Programmes • ATS incident prevention • Many initiatives can be pursued • Five primary areas • Communications • Phraseology • Supervision • Teamwork • ATC proficiency

44. Identify activities/efforts which enhance individual, ATS unit, and system performance Recognize commendable activities/efforts Identify deficiencies (individual, equipment, procedural) Promptly correct deficiencies Ensure actions taken corrected the deficiencies or problems Initiatives to ensure effective controller performance, teamwork, communications Communicate performance expectations Maintain summary of ATS incidents and incorporate into training Brief ATS personnel on ATS incidents Monitor/evaluate voice recordings Review local/national/regional ATS incidents Annual QA review ATS unit staff meetings Improve communications among all ATS personnel to create atmosphere conducive to sharing information Attendance to local, national, regional QA seminars, conferences, etc. Chapter 6. Air Traffic Incident Prevention Programmes Prevention objectives

45. Communicate performance expectations with controllers Take prompt action when performance does not meet expectations (on-the-spot corrections, etc.) Address individual/team accountability Efficient/consistent oversight of ATS unit operation Ensure distractions and noise levels in ATS unit are kept at a minimum Require personnel to maintain high degree of professionalism, teamwork, control position discipline, awareness Promote open flow of communications Conduct ATS unit team meetings Place emphasis on hearback/readback Keep ATS unit chief/support staff informed (operations concerns, etc.) Chapter 6. Air Traffic Incident Prevention Programmes ATS supervisors

46. Random voice recording reviews Brief ATS unit on trends and causal factors related to ATS incidents, random recording reviews, evaluations, etc. Maintain awareness of ATS operational environment Provide staff expertise and support Assist in ensuring QA goals and objectives are being met Ensure scenarios involving hearback/readback errors are included in training Prepare/distribute ATS incident summaries to all ATS personnel Ensure ATS unit maintains summary of ATS units, causal factors, trends, etc. Incorporate into classroom and annual proficiency training Chapter 6. Air Traffic Incident Prevention Programmes ATS quality assurance officers/specialist

47. Keep ATS supervisors advised of traffic problems and equipment limitations Suggest improvements to ATS unit and/or incident prevention Maintain situational awareness Assist busier control positions Assist busier control positions Review own operating techniques and ATS unit procedures to effect highest quality of performance Promptly report ATS incidents Utilize memory aids Chapter 6. Air Traffic Incident Prevention Programmes ATC personnel

48. Chapter 6. Air Traffic Incident Prevention Programmes • Voice recording monitor evaluations • To ensure proper phraseology, good operating practices, adherence to ICAO standards, national/local directives and practices • Guidelines • Conduct semi-annually on all ATS operational personnel • Review of voice recording by ATS supervisor/ATS QA specialist • Document comments • Develop action plan to address performance deficiencies • ATS supervisor/QA specialist review and discuss voice recording with controller

49. Chapter 6. Air Traffic Incident Prevention Programmes • Review of ATC procedures and practices that may contribute to ATS incidents • Constant review of ATC procedures and practices • Identify, report, recommend, and implement amendments • Ensure safety of aviation and prevention of ATS incidents

50. Chapter 6. Air Traffic Incident Prevention Programmes • ATS incident review groups • Implement ATS incident review groups • National/local levels • Purpose • Periodic reviews of prior ATS incidents • Identify other potential problem areas requiring attention • Effort to prevent ATS incidents • Composition • ATS unit chief • Controllers • Other ATS staff • ATS unit QA specialist • Pilots/airlines when appropriate