1 / 18

Instructor Calibration

Instructor Calibration. Building Grading Consistency with QC Observers and Emergency Procedures Instructors. Presenters. Greg Hegarty Manager, Emergency Procedures Training & Programs United Airlines Chris Hallman Founder and Principal Consultant, Great Circle Consulting.

kaycee
Download Presentation

Instructor Calibration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Instructor Calibration Building Grading Consistency with QC Observers and Emergency Procedures Instructors

  2. Presenters • Greg Hegarty • Manager, Emergency Procedures Training & Programs United Airlines • Chris Hallman • Founder and Principal Consultant, Great Circle Consulting

  3. Why Focus on Calibration? • Standardize Training Delivery • United Airlines trains flight attendants in six locations worldwide • Ensure Grading Consistency • United employs more than thirty instructors and quality control observers for emergency procedures training • Enhance Data Quality • Provides training managers with more accurate picture of training effectiveness • Supports Transition to AQP

  4. Departmental Organization

  5. Emergency Procedures Training Department 2010 Manager Emergency Procedures Training & Programs Administrative Assistant Supervisor Emergency Procedures Training Program Developers 3 – Flight Training Program Specialist (FTPS) 1 – Multi-media (temp) ORDTK 1 - EPS 10 - EPIs Train ORD, BOS, DCA, JFK, domicile FAs SFOTK 1 - EPS 5 - EPIs Train SFO, LAX, LAS, SEA domicile FAs HNLTK 1 - EPS Train HNL & NRT domicile FAs DENTK 1 - EPS 10 - EPIs Train DEN domicile FAs LHRTK 1 - Sr. Staff Instructor Part-time EPI Train LHR & FRA domicile FAs HKGTK 1 - Sr. Staff Instructor Train HKG domicile FAs EPS: Emergency Procedures Specialist EPI: Emergency Procedures Instructor

  6. Enhanced Quality Control Plan • Develop enhanced grade sheets and quality control observer data collection instruments • Reliable and Valid • Design QC observer calibration training • Design instructor calibration training • Conduct calibration training • Monitor and Evaluate

  7. Instructor Calibration Training includes: • Importance of collecting quality data in evaluating flight attendant performance • Rater-referent reliability • Inter-rater reliability • Video scenarios • Actual grade sheets • Coaching performance improvement to improve calibration

  8. Door PC1 Scenario • 777 door 1L • F/A #1

  9. Please make your selection…PC1. • Pass • Fail • Coach

  10. Quality Control Observer Calibration Training includes: • Importance of collecting quality data in evaluating flight attendant performance • Rater-referent reliability • Inter-rater reliability • Video scenarios • Actual grade sheets • Coaching performance improvement to improve calibration

  11. Results

  12. Audit Activity Summary08Feb-04Mar 2010 • Audits Completed 45 • Modules Audited 203 • Items Audited 1,724 • Items-Deviation from Standard 142 = 8.2% of total items audited • Items-Deviation from Standard 32* = 1.9% of total items audited with Intervention22.5% of total deviation- from-standard items *18 add’l responses under review

  13. Audits Completed per Location 08Feb-04Mar 2010

  14. Audits Completed per Module 08Feb-04Mar 2010

  15. Compliance by Module 08Feb-04Mar 2010

  16. Program Timeline

  17. QC Process Improvement Plan Timeline Immediate Action Steps Dec. - Jan COMPLETE Instructor & QC Observer Calibration Phase 2 COMPLETE Observation & Corrective Actions CONTINUING APR-MAY Organization Redesign CURRENTLY UNDER REVIEW Instructor & QC Observer Calibration Phase 3 JUNE • Compliance: • Leads Call • Instructor Letters • Standardized Daily Briefing/ Debriefing Form • Enhanced Admin.Sign-off Checklist • IG Revised - No Deviation • Written Flow Schedules at Each TK Center • F/A Training Certification Form • Forms: • Supplemental Form for QC Audits • Training Plan: • Remedial • Individual Commitment Review • New Video Vignettes for Instructors & QC Observers • Compliance & Performance Tracking with Q5 Database • Basic Training Review • Intent • Role: Through the eyes of the FAA • Human Factors: Objectivity vs. Subjectivity • Standardiza-tion & consistency • •Yes/No Form • Grading Exercises • Practice QC Observer Training & Calibration Feb. 2-5 COMPLETE • Basic Training Review • Intent • Role: As the arms of the FAA • Regulatory Requirements vs. Individual-ization • QC Form: Shifting to Yes/No Format • QC Responsibility • Video Vignettes • Exercises: Compliance vs. Deviation Instructor Training & Calibration Feb. 3-5 COMPLETE • Observations • •8internal QC observers • QA resource support • All 6 TK Centers • 4-6 Hour Observations • Shortened QC Form • Yes/No Format • Q5 Database • Continuous Improvement Processes Ongoing • QA Observes QC’s QC Observation Blitz Feb-Mar COMPLETE • Data • •Daily input & review • •Refinement of QC document • •Track by Issues/ Module, TK Center & Instructor • Follow-Up: Observers & Instructors • Weekly Mtg SW/TK/QA • Weekly Leads call • On-going development calls Data Review & Corrective Actions IN PROGRESS • Calibration • •Review of findings, issues, concerns • •Calibration update based on findings • At 4/1 start of 2010 RET, QC observers conduct class at SFO, DEN, ORD • All Other Instructors observe • Next step: Instructors teach, QC observes • Continuous Improvement • Tracking, trending & corrective actions continue • 2010 RET incorporates learning • Further refinement of: • •Curriculum • IG & Manuals • QC Document • Data Reporting Systems • HHDs • Calibration with FAA Evolution •Resource efficiency & centralization •Oversight capability in TK Centers •QC Observer Pool •Equipment Maintenance •AQP Program Development QC Program Design Dec. - Jan COMPLETE Calibration •Quarterly calibration continues

  18. Questions? • Greg Hegarty: greg.hegarty@united.com • Chris Hallman: chris.hallman@greatcircleconsulting.com

More Related