‘Do we need exams?’

1. ‘Do we need exams?’ Wendy Reid Medical Director HEE Past – Vice President RCOG

2. Assessment of doctors.... • Demanded by the public • Required by the regulator • Necessary for the definition of ‘profession’ But..... • Opaque methods • No direct input from the public • Examinations are often ‘historical’ not designed for their present purpose

3. What is Assessment ? • A biopsy of knowledgeand skills “clinical competence”

4. Criticalquestions in assessment • WHYare you doing the assessment? • WHATare you going to assess? • HOWare you going to assess it? • HOW WELLis the assessment working?

5. WHYare you doing the assessment? • Is its purpose: • Formative? • Summative? In course/ in training feedback Graduation/ PG Certification

6. Critical questions • WHY are you doing the assessment? • WHAT are you going to assess? • HOW are you going to assess it? • HOW WELL is the assessment working?

7. WHATare we testing? Clinical competence • Knowledge • factual • applied: clinical reasoning • Skills • communication • clinical • Attitudes • professional behaviour

8. Behaviour = Skills + attitude Professionalauthenticity Cognition = knowledge A model of clinical competence Does Shows how Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67.

9. Criticalquestions • WHY are you doing the assessment? • WHAT are you going to assess? • HOW are you going to assess it? • HOW WELL is the assessment working?

10. Professional authenticity Testingformats Performance/hands on assessment Does Shows how Written/ Computer based assessment Knows how Knows Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67.

11. Performance assessment in vivo: WBA eg mini-CEX, DOPs, TBA Does Performance assessment in Vitro: OSCE Knows how (Clinical) Context based tests: SBA, SAQ, (EMQ) Knows Factual tests: SBA, SAQ, (EMQ) Shows how Testing formats Does Shows how Knows how Knows

12. Critical questions • WHY are you doing the assessment? • WHAT are you going to assess? • HOW are you going to assess it? • HOW WELL is the assessment working?

13. Howwell is the assessment working? • Is it valid? • Is it reliable? • Is it doing what it is supposed to be doing? • To answer these questions, we have to consider the characteristics of assessment instruments ** Define the purpose of the assessment

14. Characteristics of assessment instruments • Validity (V) • Reliability (R) • Educational impact (E) • Acceptability (A) • Cost (C)

15. 1 2 Specialty Training & Education Programme Full registration CCT Specialist Training Curriculum Intermediate Advanced Training Modules Foundation Basic 3 4 5 6 7 1 2 * Log Book Women's Health Module Annual Review of Competence (ARCP) Subspecialty 2-3yr Part 2MRCOG Exam Part 1 MRCOG Exam

16. Curriculum • ‘Run-through’ i.e. Appointed once, progress by assessment • Iterative 7 years – average doctor takes 9.8 years • First 2 years – basic knowledge, must pass part 1 of exam • Middle 3 years – intermediate, must pass part 2 of exam • Final 2 years – advanced, continue with core work and learning but add specialist modules

17. Principles of curriculum • Competency based • Performance measured • Iterative time – ‘weigh’ points • Transition clearly defined at each stage • Flexibility in advanced training • Generic skills across core • Log book – e-portfolio • Knowledge and application of knowledge tests • Workplace based assessments

18. Aim of curriculum • Produce well trained Obstetricians & Gynaecologists ready for consultant posts in the NHS • Produce doctors with flexibility of career choice, well advised throughout training • Produce doctors who will advance the care of women • Re-defined in ‘Tomorrow’s Specialist’ publication 2012

19. Options during training Doctors are allowed to: • Work less than full time (50% or more) • Take time out of the programme to work overseas or do research (maximum 3 years) • Can move into formal Academic training pathway • ‘Pause’ – personal reasons, Olympics, Maternity leave • Apply for sub-specialty training from end of year 5 But... Every doctor does the MRCOG examination

20. MRCOG Examination • Any graduate can enter from anywhere in the world, need evidence of medical degree • Part 1 – test of basic knowledge applied to clinical O&G. Written papers (EMQs, MCQs) • Part 2- application of knowledge, 2 written elements require pass before OSCE element • Reviewed in 2013 – new proposal to split part 2 and have oral element as part 3

21. The MRCOG Overseas Centres

22. Part 1 Success Rates

23. Part 2 Success Rates

24. Why Take the MRCOG? “It is one of the most highly recognised and well-respected degrees in my country” [India] “It is a window through which I can have more knowledge and find the chance of training in O&G” [Sudan] “It would give me the best chance at getting first-world training which I could use to advance the level and quality of health care service provided in my coutry” [Trinidad]

25. Why Take the MRCOG? “I wish to have an international degree with expertise in evidence-based medicine, audits and protocols…to serve patients better” [India] “Passing…means that I have achieved an appropriate level to implement RCOG standards to improve women’s health.” [Saudi Arabia] “It is a prestigious and well-recognised qualification.” [Pakistan]

26. Consultant Country of Qualification

27. Principles of Assessment • There isNO perfect assessment: • compromiseis alwaysrequired • dependson the CONTEXTof the assessment • The Quality of assessment is a function of • The overall design of the programme of assessment • The qualityof the individualinstruments • Gatherandscrutinise overall validityevidence

28. Yes, we need exams • Public confidence • Professional recognition • RCOG standard • International credibility – for the college nad for individuals • But they must be fit for purpose, modern, reflect best educational practice and embrace evidence based techniques

29. The End