Dr Louise Hull Research Associate, Department of Surgery and Cancer, Imperial College London

1. Assessment of Team Skills in the Operating Room: Development and Evaluation of a ‘Train-the-Trainers’ Program Dr Louise Hull Research Associate, Department of Surgery and Cancer, Imperial College London

2. Non-Technical Skills, Teamwork and Performance

3. Assessment ASSESSING NTS & TEAMWORK IN THE OR Availability of faculty TRAINED to assess Availability of assessment tools

4. Availability of Assessment Tools NOTSS, __ ANTS, __SPLINTS ASSESSING NON-TECHNICAL & TEAM SKILLS IS A SKILL IN ITSELF OTAS Revised NOTECHS Oxford NOTECHS

5. Learning from Other Industries LACK OF GUIDELINES/RECOMMENDATIONS CURRENTLY EXIST FOR TRAINING FACULTY TO ASSESS AND DEBRIEF IN SURGERY Training Faculty to Assess and Debrief is CRITICAL

6. Training Guidelines For The OR • AIMS: • To obtain expert consensus on the specifications of a ‘train-the-trainers’ program for Faculty assessing non-technical and team skills • To develop, pilot and evaluate a ‘train-the-trainers’ program (based on expert guidelines & recommendations)

7. Methodology 3 Stage Delphi • UK-wide multistage expert consensus study • Experts in non-technical/team skills assessment (n=19) • 11 clinicians (8 surgeons, 3 anaesthesiologists)and8 psychologists/human factors specialists Stage 1: Interviews Interviews to explore expert views on Training Requirements Stage 2: Survey (Round 1) Analysed for agreement and consensus Stage 3: Survey (Round 2) Survey repeated, identical to first (but incorporating group's second round responses

8. Training Program Development • Based on Expert Guidelines/ Recommendations

9. Results: Training Guidelines Trainers: Who should deliver the ‘Train-the-Trainers’ program? • Multi-disciplinary team consisting of clinicians and psychologists/human factors experts Faculty: Who should the ‘Train-the-Trainers’ programme be aimed at? • Senior clinicians (attending-level) • Clinicians that regularly provide training and assessment • Clinicians and psychologists/human factors experts (alongside clinicians)

10. Training Guidelines Training Program Content • Theoretical background on non-technical skills and human factors applied to healthcare/surgery • Introduction to non-technical skill assessment tools • Limitations of non-technical and team skill assessment (e.g. rating errors) • Implications of non-technical and team skill assessments • Training in the recognition of non-technical and team skills • Practice in rating non-technical and team skills • Training in providing feedback/debriefing following a non-technical and team skill assessment

11. Training Guidelines Training Programme Delivery • Proficiency • To provide performance feedbackwithin clinical practice: Minimum intra-class correlation coefficients of 0.61-0.70 • To provide high-stakes assessments: Minimum intra-class correlation coefficients 0.71-0.80 • Recertification • Faculty should be assessed annuallyif providing high-stakes assessments • Methods: Video clips for practising observing and rating non-technical skills • Duration: 2 full-days, proficiency-based training, support after initial training, and refresher course(s) • Ratio of Trainers to Faculty: 1:12 or more for theoretical components, 1:4-6 for practical components of the training.

12. Training Program Development • Based on Expert Guidelines/ Recommendations • Trainers: Multidisciplinary (Clinicians & Psychologists) • Faculty: Senior Clinicians (surgeons, anaesthesiologists, OR nurses) • Duration: 1 day • Delivery: Workshop • Content: Theoretical and practical components • Team Assessment: Observational Teamwork Assessment for Surgery (OTAS)

13. Practice Observing and Rating Team Skills • Observe Teamwork • Video • Rate Teamwork • Behaviours using OTAS • Group Discussion • followed by • Presentation of Expert Ratings

14. Faculty • 28 faculty attended two training sessions (16 and 12 respectively) • Mix of surgeons, anaesthesiologists, nurses, and OR managers • Attending & Senior Residents

15. Discussion & Conclusion • First training program, developed according to national consensus guidelines, which suggest that faculty are able to reach an acceptable level of proficiency in teamwork assessment • Assessing non-technical skills/performance is a skill in itself – and thus it requires training • Training faculty will be essential to formally integrate team skills assessment into surgical programs ASSESSING TEAMWORK IN THE OR Availability of faculty TRAINED to assess Availability of assessment tools

16. Acknowledgements Expert Consensus Panel SonalArora Andrew Healey Eva Kassab Krishna Moorthy Nick Sevdalis ShabnamUndre Professor Charles Vincent Associate Professor Ken Catchpole Peter McCulloch Amitabh Mishra Ronnie Glavin Professor RhonaFlin Nikki Maran Lucy Mitchell Simon Paterson-Brown Rona Patey Professor David Rowley Steve Yule Professor George Youngson Research Team and Training Faculty SonalArora Professor Bryn Baxendale Professor the Lord AraDarzi RohzJalil Peter McCulloch Krishna Moorthy Stephanie Russ Nick Sevdalis Nick Symons Professor Charles Vincent

17. Thank you for your attention! l.hull@imperial.ac.uk www.cpssq.org