n MRCGP Workplace-based Assessment

1. nMRCGPWorkplace-based Assessment March 2007

2. nMRCGP • Integrated assessment package comprising: • Applied knowledge test (AKT) • Clinical skills assessment (CSA) • Workplace-based assessment (WPBA)

3. Workplace-based assessment “The evaluation of a doctor’s progress over time in their performance in those areas of professional practice best tested in the workplace.”

4. Some principles of assessment • Validity • Reliability • Educational impact • Acceptability • Feasibility

5. Why workplace-based assessment? • Tests something important and different from other components “Does do versus can do” • Reconnects assessment with learning • Has high educational impact • Valid and reliable • In keeping with PMETB guidance

6. The WPBA framework An integrated package comprising a competency-based training record that applies over an entire training envelope (3 years from August 2007)

7. The educational model of WPBA for nMRCGP

8. The competency-based training record • Competency-based • Developmental • Evidential • Locally assessed • Triangulated

9. Competency-based • 12 competency areas • Best tested in the workplace setting • Developmental progression for each competency area • Competency demonstrated “when ready” • Process is learner led

10. The 12 competency areas 1. Communication and consulting skills 2. Practising holistically 3. Data gathering and interpretation 4. Making a diagnosis/ making decisions 5. Clinical management 6. Managing complexity and promoting health 7. Primary care administration and IMT 8. Working with colleagues and in teams 9. Community orientation 10. Maintaining performance, learning and teaching 11. Maintaining an ethical approach to practice 12.Fitness to practice

11. Developmental progression “a process of monitoring a student’s progress through an area of learning so that decisions can be made about the best way to facilitate future learning”

12. Evidential • Notion of multiple sampling • From multiple perspectives • Tool-box of “approved” methods (locally assessed and national complementary tools) • Sufficiency of evidence defined

13. Locally assessed • Assessed by clinical supervisor in hospital or general practice setting • Regular reviews at 6 month intervals by trainer/educational supervisor • Review all the assessment information gathered • Judge progress against competency areas • Provide developmental feedback

14. Triangulated • Different raters • Many tools (e.g. CBD, COT, mini CEX, DOPS, MSF and PSQ) • Different settings (hospital and general practice)

15. Gathering the evidence about the learner’s developmental progress

16. Evidence from • Locally assessed tools • Complementary tools and… • Naturally occurring information

17. Tools for Evidence CBD (case based discussion) COT (consultation observation tool) mini-CEX (clinical evaluation exercise) DOPS (direct observation of procedural skills) MSF (multi-source feedback) PSQ (patient satisfaction questionnaire)

18. Case-based discussion • Structured oral interview • Designed to assess professional judgement • Across a range of competency areas • Starting point is the written record of cases selected by the trainee • Will be used in general practice and hospital settings

19. COT • Tool to assess consultation skills • Based on MRCGP consulting skills criteria • Can be assessed using video or direct observation during general practice settings

20. Mini CEX • Used instead of COT in hospital settings

21. DOPS • For assessing relevant technical skills during GP training: • Cervical cytology • Complex or intimate examinations (e.g. rectal, pelvic, breast) • Minor surgical skills • Similar to F2 DOPS

22. MSF • Assessment of clinical ability and professional behaviour • ST1 Rated by 5 clinical colleagues, 2 occasions ST3 Rated by 5 clinical and 5 non-clinical colleagues on 2 occasions • Simple web based tool • Is able to discriminate between doctors • Needs skill of trainer in giving feedback

23. PSQ • Measures consultation and relational empathy (CARE) • 30 consecutive consultations in GP setting • Central optical scanning and generation of results • Can differentiate between doctors • Needs skill of trainer in giving feedback

24. Naturally occurring evidence • From direct observation during training • “tagged” against appropriate competency headings • Other practice-based activities

25. Monitoring Progress • Interim reviews with trainer • 6 month intervals • ensure the trainee is making satisfactory progress • agree training needs

26. Monitoring Progress • Deanery Panel meeting at end of ST1 and ST2 • reviews the training records of every trainee • face to face review with trainees when • unsatisfactory achievement in either of the complementary tools • or when requested by the educational supervisor

27. Workplace-based assessment ST1 6 month 12 month Deanery panel if unsatisfactory Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x MSF 1 x PSQ * DOPS ** Clinical supervisors’ report ** Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x MSF 1 x PSQ * DOPS ** Clinical supervisors’ report ** * if GP post ** if appropriate

28. Workplace-based assessment ST2 18 month 24 month Deanery panel if unsatisfactory Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x PSQ * DOPS ** Clinical supervisors’ report ** Interim review Based on evidence: 3 x COT or mini-CEX 3 x CBD 1 x PSQ * DOPS ** Clinical supervisors’ report ** * if GP post ** if appropriate

29. Workplace-based assessment ST3 30 month 34 month Deanery sign off or panel review if unsatisfactory Interim review Based on evidence: 6 x COT 6 x CBD 1 x MSF DOPS ** Final review Based on evidence: 6 x COT 6 x CBD 1 x MSF DOPS ** PSQ ** if appropriate

30. The final judgement • The trainer makes a recommendation as to whether the trainee has achieved competence in all 12 areas at the end of training

31. Review by Deanery Panel • Review of e-portfolio if satisfactory level achieved in training record • Review of e-portfolio and face-to-face meeting with trainee, if satisfactory level not achieved