Workplace based assessment for the nMRCGP

1. Workplace based assessment for the nMRCGP

2. nMRCGP • Integrated assessment package comprising: • Applied knowledge test (AKT) • Clinical skills assessment (CSA) • Workplace based assessment (WPBA)

3. Workplace based assessment

4. Definition “The evaluation of a doctor’s progress over time in their performance in those areas of professional practice best tested in the workplace”

5. Some principles of assessment

6. Utility • Validity • Reliability • Educational impact • Acceptability • Feasibility

7. Why workplace-based assessment? • Tests something important and different from other components • “Does do versus can do” • Reconnects assessment with learning • Has high educational impact • Valid and reliable • In keeping with PMETB guidance

8. The WPBA framework

9. The components of the framework An integrated package comprising: • A competency-based training record and… • Two externally marked work-based tools That applies over an entire training envelope (3 years from August 2007)

10. The educational model of WPBA for nMRCGP

12. The competency-based training record

13. Key features • Competency-based • Developmental • Evidential • Locally assessed • Triangulated

14. Competency-based • 12 competency areas • Best tested in the workplace setting • Developmental progression for each competency area • Competency demonstrated “when ready” • Process is learner led

15. Communication and consulting skills Practising holistically Data gathering and interpretation Making a diagnosis/ making decisions Clinical management Managing complexity and promoting health Primary care administration and IMT Working with colleagues and in teams Community orientation Maintaining performance, learning and teaching Maintaining an ethical approach to practice Fitness to practice The 12 competency areas

16. Developmental progression “a process of monitoring student’s progress through an area of learning so that decisions can be made about the best way to facilitate future learning”

17. Evidential • Notion of multiple sampling • From multiple perspectives • Tool-box of “approved” methods (locally assessed and national external tools) • Sufficiency of evidence defined

18. Locally assessed • Assessed by clinical supervisor in hospital or general practice setting • Regular reviews at 6 month intervals by educational supervisor • Review all the assessment information gathered • Judge progress against competency areas • Provide developmental feedback

19. Triangulated • Internally: • Different raters • Local tools (e.g. CBD, COT, mini CEX, DOPS) • Different settings (hospital and general practice) • Externally: • Multiple raters • Externally marked tools (MSF and PSQ) • Different settings (hospital and general practice)

20. Gathering the evidence about the learner’s developmental progress

21. Evidence from • Locally assessed tools • External tools, and… • Naturally occurring information

22. Local tools CBD (case based discussion) COT (consultation observation tool) mini-CEX (clinical evaluation exercise) DOPS (direct observation of procedural skills)

23. Case based discussion • Structured oral interview • Designed to assess professional judgement • Across a range of competency areas • Starting point is the written record of cases selected by the trainee • Will be used in general practice and hospital settings

24. COT • Tool to assess consultation skills • Based on MRCGP consulting skills criteria • Can be assessed using video or direct observation during general practice settings

25. Mini CEX • Used instead of COT in hospital settings

26. DOPS • For assessing relevant technical skills during GP training: • Cervical cytology • Complex or intimate examinations (e.g. rectal, pelvic, breast) • Minor surgical skills • Similar to F2 DOPS

27. External tools • MSF (multi-source feedback) • PSQ (patient satisfaction survey)

28. MSF • Assessment of clinical ability and professional behaviour • Rated by 5 clinical and 5 non clinical colleagues on 2 occasions in ST1 and ST3 • Simple web based tool • Is able to discriminate between doctors BUT • Needs skill of trainer in giving feedback

29. PSQ • Measures consultation and relational empathy (CARE) • 30 consecutive consultations • Central optical scanning and generation of results • Can differentiate between doctors BUT • Needs skill of trainer in giving feedback

30. Naturally occurring evidence • From direct observation during training • “tagged” against appropriate competency headings • Other practice-based activities

31. Monitoring progress and final decisions

32. During training • Meeting at end of ST1 and ST2 • A Deanery based panel will: • Review the ETR of any trainee who: • “Fails “ in any of the external tools during training • Where requested by the educational supervisor i.e. where there may be a problem with the trainee’s progress in performance against the 12 competency areas • Ensure the trainee is making satisfactory progress

33. The Final Judgement • The trainer makes a recommendation as to whether the registrar has achieved competence in all 12 ETR areas at the end of training • This together with achievement in the external workplace assessments will be reviewed by a Deanery panel

34. 4 possible outcomes: • Pass in ETR and external assessments – clear pass. • Fail in ETR and fail in external assessments – clear fail. Referred to Deanery panel but no change in decision • Pass in ETR but fail in external assessments – referred to Deanery panel for decision • Fail in ETR but pass in external assessments – referred to Deanery panel for decision

35. 6M 6M Interim review Based on evidence: *3 x mini-CEX 3 x CBD **DOPS **Clinical supervisors report Interim review Based on evidence: 3 x mini-CEX 3 x CBD **DOPS **Clinical supervisors report MSF PSQ Workplace-based assessment ST1 Deanery panel if unsatisfactory * COT if GP post ** if appropriate

36. 6M 6M Interim review Based on evidence: *3 x mini-CEX 3 x CBD **DOPS **Clinical supervisors report Interim review Based on evidence: *3 x mini-CEX 3 x CBD **Clinical supervisors report **DOPS Workplace-based assessment ST2 Deanery panel if unsatisfactory * or COT if GP post ** if appropriate

37. 6M 6M Interim review Based on evidence: *6 x COT 6 x CBD **DOPS **Clinical supervisors report Final review Based on evidence: 6 x COT 6 x CBD MSF PSQ **DOPS Workplace-based assessment ST3 Deanery sign off or panel review if unsatisfactory * mini-CEX if hospital post ** if appropriate