HQ AF Flight Standards Agency Air Traffic System Evaluation Program (ATSEP)

1. HQ AF Flight Standards AgencyAir Traffic System Evaluation Program (ATSEP) HQ AFFSA/XAFS SMSgt Mike Beckles

2. Overview • ATSEP 101 • Current ATSEP Report Trends • Unit Preparation Strategies

3. True or False? 1) ATSEP is an inspection of ATC (T/F) 2) “Observations” result from failure to comply with proper policies/procedures (T/F) 3) Most observations are against ATC (T/F) 4) Most “problems” are against ATC (T/F) 5) If you can correct a deficiency “on-the-spot,” you get a “SAT” (T/F) 6) ATSEP provides “ratings” (T/F)

4. What is an Air Traffic System? System Environment People MISSION Procedures Equipment System

5. Why ATSEP? • History lessons Honest broker Validation Advocacy • Your primary functional area evaluation—and more Helps fix what you know about Helps fix what you don’t know about • Importance today… Manning/experience levels Ops tempo Risk management

6. ATSEP Purpose Evaluate the ability of the Air Traffic System to meet the standards and operational requirements of all users (AFI 13-218)

7. ATSEP Responsibilities • AFFSA provides oversight • AFI 13-218 OPR, SIIs and checklists • Observe/augment MAJCOM teams • MAJCOMs implement ATSEP • Schedule, evaluate and report • Coordinate problem/obs/SII closures • Provide feedback on checklists • Units prepare with checklists, X-feed and on-going management

8. ATSEP Scope • Where MAJCOMs have ATC, AM or ATCALS functional responsibility • Evaluates support to AT System users and compliance with standards • Provides trend info and a process to resolve system deficiencies • NOT the IG, WX or Aircrew Stan/Eval, but can be run concurrently

9. System Evaluation TERPS ATCALS ATC USERS SUPPORT AIRSPACE AIRFIELD WX Assess all air traffic system elements from an operational viewpoint

10. System Evaluation “Observations” are system deficiencies that clearly indicate actual or potential flying mission impact or an adverse affect on flight safety

11. System Evaluation • Special Interest Item (SII) • Focus attention on AF-wide issues • Derived from ATSEP trend data and “emergent” issues • Annual validation with MAJCOMs • Checklists available to all • UNSAT SII results in single observation

12. Compliance Evaluation AOF ATCALS Maint Evaluate specific functional areas for compliance with directives

13. Compliance Evaluation “Problems” are ATSEP checklist items graded unsatisfactory, indicating noncompliance with established standards “Off-Checklist Problems” are also compliance deficiencies

14. “Normal” ATSE Method TERPS ATCALS ATS USERS SUPPORT AIRSPACE AIRFIELD WX AOF ATCALS Maint

15. AFI 13-218 and MAJCOM Supplements

16. ATSE Process • Unit prep and notifications • Customer surveys Pre-visit • Inbrief • Customer interviews • Compliance checklists • System Eval and SII checklists • Outbrief ATSE Post-visit • Report & reply actions

17. ATSE Process • MAJCOM-specific requirments • Inbrief includes at least: • Team composition and intros • Overview of ATSEP and reporting • Definition of observation, SII, problem, and closure process • Process for daily review of identified/potential deficiencies

18. ATSE Process • Team reviews customer surveys • Team chief interviews customers • Flying organizations • Adjacent airports/ATC facilities • On-base support organizations • AOF staff personnel • Facility personnel

19. ATSE Process • Evaluators run functional area checklists (Sat, Unsat, N/A, N/O) • Observe daily facility operations • Completing evaluation checklists • Looking for trends--incorrect procedures or application • Looking for trends--correct procedures and application • Daily debrief/validation, team meeting

20. ATSE Process • Out-brief mandatory items • Overall assessment of ATS • SII results, observations & problems • Required follow-up actions • Out-brief optional items • Testing results • Commendable items • Various “rating” processes

21. ATSEP Report Content I. Executive Information II. System Evaluation Observations SII results III. Compliance Evaluation Problems Off-checklist problems IV. General Information

22. Post-ATSEP Actions • AOB addresses Observations/SIIs • Convenes to assign actions & OPR w/in 30 days of receiving report • Tracks status & recommends closure • Documents closure by MAJCOM/DO • Problems/Off-Checklist Problems • Unit documents fix actions • MAJCOM determines closure authority • Written closure request & approval

23. CY 2001 CY 2002 CY2003 CY2004 113 Observations in 36 Evals (3 av.) 2348 Problems in 33 Evals (71 av.) 161 Observations in 48 Evals (3 av.) 2766 Problems in 54 Evals (51 av.) 157 Observations in 56 evals (3 av.) 2509 Problems in 56 evals (45 av) 225 Observations in 77 evals (3 av.) 3056 Problems in 77 evals (40 av.) ATSEP Activity

24. Observation Balance Sheet

25. Jan-Dec 04 ATSEP Activity Evaluations at 77 units by MAJCOM

26. Jan-Dec 04 Observations/SIIs 224 Obs/SIIs in 77 evals (2.9 av; range 0-11)

27. Jan-Dec 04* Problems 3056 problems in 77 evals (39.7 average; range 2-132) OCPs Not included

28. Trend Reporting • Semi-annual detailed trends (positive & negative) to MAJCOM 3-letters • Annual Executive Summary prepared for Air Staff and MAJCOM 2-letters • Units should use data to avoid common errors!

29. Jan-Dec 04 Obs/SII Trends Negative Trends • Airfield planning and design deficiencies • ATCALS maintenance management • ATCALS grounding and lightning protection • Airfield maintenance and repair Corrective Trends • Airfield signs and markings • AOF procedural guidance • TERPS products

30. Jan-Dec 04 Problem Trends ATC Facilities (45 Towers and 27 Radars) • ATC 100 Phraseology (12R) • ATC 162 Equipment Checks (6R) • ATC 170 RADAR display photos (5R) • ATC 058 Equipment checks (13T) • ATC 019 Phraseology (11T) • ATC 047 Visibility checkpoint chart (9T) • ATC 006 Crew change procedures (9T) • ATC 018 RIF (9T)

31. Jan-Dec 04 Problem Trends ATC Training & QA (31 Evaluations) • ATC 299 Periodic facility evaluations (13) • ATC 316 Training record reviews (11) • ATC 215 Simulator in PCGs (11) • ATC 250 Realistic scenarios (9) • ATC Appointed 7 level review (10)

32. ATSEP Perspective: • Mission • Safety • Standards

33. Summary • ATSEP focuses attention and resources to fix ATS deficiencies • Evaluation trend information can help you identify common deficiencies • Strong self-inspection and follow-up is key to ATSE preparation

34. HQ AF Flight Standards AgencyAir Traffic System Evaluation Program HQ AFFSA/XAFS