Ethnic Variation in GP Trainees' Performance: Cognitive Interview Study

1. CaHRU@lincoln.ac.uk Understanding reasons for variation by ethnicity in performance of GP specialty trainees in the MRCGP Applied Knowledge Test: cognitive interview study Dr Julie Pattinson University of Lincoln Professor Niroshan Siriwardena, University of Lincoln Dr Bijoy Sinha, Programme Director, Lincoln VTS Dr Carol Blow, AKT core group lead, MRCGP

2. Research programme First study exploring: • variations in performance by ethnicity in the MRGCP AKT • employing cognitive interviewing method Provides valuable insights into the causes of differential attainment • UK graduates (UKGs), BME UKGs, overseas trained (OST). Importance – high stakes exams with severe consequences for failure Understand why OST are more likely to underperform compared to UKGs Concept of ethnicity is sensitive and politically charged “ethnicity implies one or more of the following: shared origins or social background; shared culture and traditions that are distinctive, maintained between generations, and lead to a sense of identity and group; and a common language or religious tradition”. (Senior, S.A & Bhopal R, 1994).

3. CaHRU@lincoln.ac.uk Background • no findings of discrimination based upon examiner judgments • review concluded “the method of the MRCGP assessment is not a reason for the differential outcomes” and noted that “the AKT is a machine marked examination testing applied clinical knowledge” There is a differential pass rate for both BME UKGs and IMGs compared to white UKGs…..(RCGP, 2016).

4. CaHRU@lincoln.ac.uk Aim and Objectives Investigate reasons for differences in performance in the MRCGP AKT related to ethnicity of candidates. Explore differences across three subgroups of doctors in GP speciality trainingby: • Ethnicity • Source of primary medical qualification Comparative analysis • White British/Irish UK trained (UKGs) • Black and Minority Ethnic (BMEUKGs) • Overseas trained (OST)

5. CaHRU@lincoln.ac.uk Methods Qualitative design - semi structured, cognitive interview (Willis, 2005). • 15 live AKT questions • understanding questions, wording and answers • thought processes retrieving information and applying it to solving AKT questions Systematic Grounded Theory (Strauss & Corbin, 1998). 21 Participants • 13 OST • 1 BME UKG • 7 UKGs Recruitment: East Midlands • GP Vocational Training Centres • Speciality Training Years 1 – 3. AKT questions chosen on the basis of similar or different performance according to candidate ethnicity. Provided by RCGP. Not informed if their answers were correct.

6. Emergent Theme 1: Theoretical versus real-life clinical experience “You need theory obviously but the practical exposure makes you remember”. Female, OST All participants Classroom versus clinical experience Sub-theme 1 Exposure “If you are coming from outside Europe, they don’t see any CT in real life.”, Male, OST All participants Sub-theme 2 Clinical Exposure limited minor specialities Easier to recall AKT information OST less exposure “We are taught to memorise things, all the doses and names.” Female OST OSTparticipants Memorisation -TheoreticalAdvantage for OST Sub-theme 3 Rote Learning

7. Emergent Theme 2. ‘Recency, frequency, opportunity and relevance’ “I haven’t worked in paediatrics or come across this[UK].When I was in basic training seven years ago [overseas], that’s the time I read about vaccinations. I have forgotten.” Male, OST. All participants AKT Topics not revisited Sub-theme 1 Recency OST – lengthier intervals All participants “Getting exposed to many different patients makes it easier to remember.” Male OST Frequent exposure to patients reinforced learning Sub-theme 2 Repeated Exposure All participants I’m not going to be doing a rotation through that”. Male UKG Sub-theme 3 Limited Opportunity Rotation, gender, uncommon presentations OSTparticipants disadvantaged: No access in UK orundergraduate training “overseas, we didn’t have much statistics. When we started studying for AKT, we got to know [UK] that we need to study statistics.” OST Male All participants Sub-theme 4 Relevance AKT topics; speciality, stats, exam scenario OSTparticipants disadvantaged; statistics

8. Emergent Theme 3. ‘Cultural barriers’ “If you are not familiar with the system, you don’t know what services are available.” Female OST Sub-theme 1 Unfamiliarity of the process of the NHS system Transition to UK practice “even the simplest things. Abbreviations” OST Female “we read the question, will have got a bit of three way translation” OSTMale Translation in exam - UK training Sub-theme 2 Abbreviations Language barriers; Exam Format “guidance are used more in the UK. So I still have to read a lot ofguidance. Back where I trained, I don’t thinknational guidelines apply.” OST male Sub-theme 3 NICE -working in a clinical framework Point of accessing UK guidelines

9. Implications Key messages • OST participants compared to UKGs appear to face additional difficulties answering AKT questions • Generated insights into reasons for difficulty in answering AKT questions in all participants • Provided a basis for developing interventions to reduce differential attainment in UK specialty training for general practice. • RCGP Fairness project (2013): Observed for IMG candidates limited evidence of insufficient time Opened discussions • Issues related to test timing • Need for a robust test of competence to ensure safety for patients • Identified areas in the AKT where risk is more likely to be acute (e.g. calculating patient risk in practice, OST candidates adaption to UK clinical guidance, NICE) A stronger position/impact • RCGP/BAPIO • Inform the test • Inform training (e.g. limited stats teaching in UK graduate speciality curriculum)

10. References Benjamin, AS, Bjork, RA. (2000). On the relationship between recognition speed and accuracy for words rehearsed via rote versus elaborative rehearsal. J Exp Psychol. Learn Mem Cog, 26, 638. Gill PS. General practitioners, ethnic diversity and racism. In: Coker N, ed. Racism in medicine: an agenda for change. King’s Fund, 2001:99-120. Royal College of General Practitioners. (2013). AKT "fairness" project. MRCGP AKT Core Group. Internal publication. Royal College of General Practitioners. (2016). MRCGP Annual Reports 2014 to 2015. Retrieved from http://www.rcgp.org.uk/training-exams/mrcgp-exams-overview/mrcgp-annual-reports/mrcgp-annual-reports-2014-2015.aspx Senior PA, Bhopal R. Ethnicity as a variable in epidemiological research. BMJ1994;309:327-30 Strauss A L, Corbin J. (1988). Basics of Qualitative Research: Grounded Theory Procedures and Techniques. 2nd edition. Thousand Oaks: Sage. Thorndike, E (1932). The Fundamentals of Learning. AMS Press Inc. ISBN 0-404-06429-9

11. CaHRU@lincoln.ac.uk Thank you for listening! jpattinson@lincoln.ac.uk and http://cahru.org.uk/