Conducting workplace based assessments

1. Conducting workplace based assessments Philip DaCosta Consultant Histopathologist Member, RCPath WBA Working Group

2. Outline • Motivating trainers and trainees • Organising the assessment process • Conducting an assessment • Providing feedback • Workplace-based assessment as a formative process

3. “It’s not all alien…” • Build on existing structures • Day-to-day clinical work/encounters • Supervisory staff (medical, scientific and BMS) • Use existing opportunities to train and assess • WBA formalises existing supervision and improves feedback

4. Motivation • Clarity • about process and method of assessment • about number of assessments • about trainee’s role • about the assessor’s role • about documentation and evidence • Expectation • monitoring arrangements • linkage to the ARCP • broad coverage of professional activities

5. Organisation • Cohort of assessors (appropriate to the relevant tasks) • Training and familiarity with assessment tools • Pacing of assessment process • planner for year • linkage to other processes (portfolio reviews, ESR, ARCP) • remedial processes

6. The mechanics…

7. Mini-PAT CBD Action DOES ECE Mini-CEX Performance SHOWS DOPS Competence KNOWS HOW Knowledge KNOWS Miller’s pyramid of clinical competence

8. Consider the focus of the assessment Mini-CEX DOPS ECE CbD Real time - HOW people do their work Reflective - WHY they did what they did

9. Mini-CEX DOPS ECE CbD Reflective - WHY they did what they did Real time - HOW people do their work Multi-source feedback - global and team-based assessment

10. Number of WBAs during training

11. Undertaking assessments Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

12. “Overview of WBA” document • Customised for each specialty • Similar content • Introduction to WBA • Outline of WBA tools • Who can be an assessor? • Standards for assessment • Record keeping • Examples of scenarios • Hyperlinks to relevant guidance notes and forms

13. Guidance note for each WBA tool • Aimed at assessors and trainees • Customised for each specialty and each tool • Content: • Explanation of the WBA tool • Suitable cases • Who can be an assessor • How the assessment works • Standards for assessment • Outcome of assessment • Feedback • Record keeping

14. Opportunities & scenarios Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

15. Opportunities - examples of DOPS Direct observation of practical skills (DOPS) - Chem Path • use of pipette • use of balance • preparation of buffer • measurement of glucose using meter • urinalysis using ‘dipstick’ • use of blood gas machine • manual (specify), e.g. osmometry • use of bilirubinometer • performance of sweat test • insertion of long line • supervision of dynamic function tests

16. Opportunities - examples of DOPS Direct observation of practical skills (DOPS) - Histopathology • Specimen cut up (e.g. a cancer resection, a group of skin biopsies) • Autopsy procedures (e.g. initial evisceration, dissection of heart or other system, taking samples for toxicology) • Set up and use of microscope • Use of camera and specimen photography • Handling and reporting of frozen section • Systematic microscopic assessment of biopsy/cytology specimen • Practical reporting procedures (e.g. completion of proformas, notification of case for MDT discussion, diagnostic coding, ensuring copies are sent to relevant individuals) • Taking a fine needle aspirate

17. Opportunities - examples of DOPS Direct observation of practical skills (DOPS) - Microbiology • sample handling and preparation (including safe laboratory practice) • safe disposal • microscopy and staining • identification: presumptive or definitive • use of selective media • ability to distinguish between significant and non-significant pathogens and normal commensal flora • ability to spot and separate mixtures of organisms, obtaining pure culture • antimicrobial sensitivity testing: setting up, reading and interpretation • practical aspects of therapeutic drug monitoring • serology • molecular methodologies

18. Opportunities - examples of ECE Evaluation of clinical/management events (ECE) - Chem Path • Presenting audit findings and leading discussion on the action required • Observation of a trainee led teaching event • Demonstration and presentation of cases at MDT • Presentation at ‘grand round’ • Referring a case for a specialist opinion • Providing clinical biochemistry advice in response to enquiry (primary and secondary care) – by letter, by phone • Use of critical incident/non-conformity reporting procedures • Presenting findings and leading discussion on the action required • Preparing a business case • Writing clinical guidelines • Appointment of staff

19. Opportunities - examples of ECE Evaluation of clinical/management events (ECE) - Histopathology • evaluating and reporting a histopathology or cytology case • evaluating and presenting autopsy findings to a clinical supervisor • demonstrating autopsy findings to clinical team • presenting cases in a MDT meeting/clinicopathological conference • presenting cases at a morbidity/mortality meeting or ‘grand round’ • referring a case for specialist opinion • providing clinicopathological advice in response to an enquiry • use of the call/recall and failsafe systems in cervical cytology screening • use of critical incident reporting procedures • handling a patient safety issue (e.g. specimen misidentification) • presenting audit findings and leading discussion on the action required.

20. Opportunities - examples of CbD Case-based discussion (CbD) • Discussion of a case relating to specific areas of clinical practice: • eg. lipidology or nutrition on chemical pathology • eg. blood stream inections or serological diagnosis in microbiology • eg. complex case requiring immunohistochemistry in histopathology • Involvement in critical incident or patient safety event • Case involving diverging diagnostic opinions • Case raising health and safety/risk management issues • Evaluation of QC/QA data

21. How to use opportunities for workplace-based assessment • Numerous opportunities/scenarios available - refer to guidance notes • Be realistic about assessment • duration • complexity • appropriateness • Sampling

22. Undertaking assessments Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

23. Conducting the assessment • Ensure both you and the trainee have read and understood the guidance • Check appropriateness of event • Adequate time to undertake assessment and provide feedback • Appropriate place for feedback • Have the assessment form with you and make notes on it

24. The assessment form

25. Information about the assessment …

26. The assessment …

27. Feedback and outcome …

28. Outcome - satisfactory / unsatisfactory … A “global” judgement required to comply with “Gold Guide” and ARCP requirements. In essence, does the trainee need to repeat the assessment… SATISFACTORY UNSATISFACTORY

29. Personal calibration • The “experienced” trainer • All assessors will know what constitutes acceptable (ie. safe) clinical care/standards • Assessors bring their experience to the assessment • Refer to documentation • “Standards for assessment tools” • Curriculum • Suggested complexity of scenarios at different stages of training • Consistency within cohort of trainees • educational supervisors will receive assessment forms from other assessors and will undertake assessments on a range of trainees

30. Trainees must be assessed against the standard expected of a trainee at the end of the stage of training that they are in The form offers a grading scale from 1–6: 1–2 Below expectations 3 Borderline 4 Meets expectations 5–6 Above expectations “Standards for assessment tools” provides definitions and grade descriptors Standards for assessment

31. Process Standardisation • Triangulation • Checklist -v- Global rating • Discussion amongst assessors • College monitoring and QA processes

32. Benefits of using the assessment form • Gives structure to the encounter • Encourages a systematic approach • Generic forms, so some items will be not applicable (“unable to comment”) • Reduces bias • Helpful in providing feedback • Helpful in identifying developmental points • Helpful in commenting on behaviour

33. Feedback Opportunities and scenarios Assessment process Feedback Guidance notes for assessors and trainees Curriculum Schedules of scenario complexity Assessment form & feedback guidance Personal Development Plan Portfolio Assessment form Standards for Assessment

34. Providing feedback • Immediate! • Part of assessment process • Consider using modified Pendleton framework • what trainee felt went well … • what assessor observed went well … • what trainee would do differently in future … • developmental suggestions from assessor … • Focus on what was done and observed • Use the form as part of the feedback process

35. Using the assessments as a formative process • What they know … • How they do things …. • How they present themselves … • How they learn … • How they are progressing … • Enthuse them ….

36. Focus on good feedback • Trainees consistently report that “senior review” has major educational impact (as well as optimising patient care) • The educational impact depends on good feedback • Using structured feedback can encourage self appraisal, confidence and planned learning by the trainee

37. Paperwork • College intend to implement a web-based system from August 2008 • For now, use the paper based systems • 3 (4) copies of assessment form (whether successful or unsuccessful) • one into trainee’s portfolio • one to educational supervisor • one to RCPath assessment office (part of the WBA standardisation and validation process) • (one copy to assessor, if they ask for it. Sensible to do this if assessment outcome is unsatisfactory) • Educational supervisor can use information from all assessments in drawing up their report for the ARCP process, even though the trainee will only present successful WBAs as evidence to the ARCP panel

38. Summary • Understand the assessment tools and process • Use your existing people, places and opportunities • Train the trainers/assessors and trainees • Allay their anxieties • Use it as a formative process