1 / 23

Introduction to Workplace-based Assessments

Introduction to Workplace-based Assessments. Dr Hani Zakhour Director of Training and Educational Standards The Royal College of Pathologists 19 October 2007. History I. PMETB Standards for postgraduate training and assessment were published in 2005

lucas
Download Presentation

Introduction to Workplace-based Assessments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction to Workplace-based Assessments Dr Hani Zakhour Director of Training and Educational Standards The Royal College of Pathologists 19 October 2007

  2. History I • PMETB • Standards for postgraduate training and assessment were published in 2005 • Training and assessment should be much more integrated – an assessment system • Clear view of PMETB and GMC (joint paper 2005) that these should include continuous monitoring not just periodic College examinations • Workplace-based assessment” PMETB Jan 2005 RCPath 07

  3. History II • RCPath • College has been fully engaged with reforms in Medical Education • January 2005, College Council witnessed a presentation of RCP pilot of WBA, tools were commended by Council • Director of Examinations and Assessment May 2005 • WBA Manager appointed October 2005 • WBA Working group March 2006 RCPath 07

  4. PMETB principle documents • Standards for Curricula Janet Grant Chair Curriculum Sub-Committee Sue Fox, Namita Kumar, Ewen Sim, Members Curriculum Sub-Committee Revised March 2005 • Principles for an assessment system for postgraduate medical Training Lesley SouthgateChair Assessment CommitteeJanet GrantChair Curriculum Sub-Committee 14 September 2004 RCPath 07

  5. Test methods as part of the blueprint • Test methods for assessment programmes can only be selected once the purpose and content are established • What is the best trade-off between fidelity and efficiency. • What is the programme for writing and refining test material? • How are marking schedules developed and reviewed where relevant? • Is new test material piloted before it is used? RCPath 07

  6. How will the assessments be collated? • Assessment tools will build up a profile as the year progresses • Evidence of a response to feedback will be shown by subsequent assessment of the same area • Overall record will form the basis for a submission to the deanery as evidence of satisfactory completion of the year ARCP RCPath 07

  7. Standard Setting • It is only possible to set a sensible standard if the purpose of the assessment is clearly set out. • The most important thing about a standard is who sets it • It is the standard expected of trainee at the end of a particular stage • Understanding principles of the specialty and practices under direct supervision (Stage A) • In-depth knowledge and understanding of principles and practices of the specialty (stage D) RCPath 07

  8. Standardisation • Commonality in the standards that are required of trainees. • Grade descriptor, intended as a guide • Grade scale • 1,2 - Below expectation • 3 - Borderline • 4 - Meets expectations • 5,6 - Above expectations RCPath 07

  9. Tools • Tools taken from F2 programme • CbD Case-based Discussion (Dame Lesley Southgate) • used to enable the documenting of conversations about, and presentations of, cases by trainees • DOPS Directly Observed Procedural Skills(RCP) • a method, similar to the Mini-CEX that has been designed specifically for the assessment of practical skills. • MiniCEX Mini Clinical Evaluation Exercise (John Norcini) • designed to provide feedback on skills essential to the provision of good clinical care by observing an actual clinical encounter RCPath 07

  10. Tools • Tools taken from F2 programme • MSF Multi Source Feedback - ePath SPRAT (Dr Helena Davies) • provides feedback from a range of co-workers across the domains of Good Medical Practice. These can be mapped to the core objectives of the curriculum • ECE Evaluation of Clinical Events (WBA Group Histopathology) • It replaces MiniCEX for Histopathologists RCPath 07

  11. Purpose of Assessment Tools • To assess what cannot be assessed by an examination. • CbD • Indicates trainees’ capability, potential and behaviour • Provides immediate feedback on trainee’s learning needs highlighting strengths and identifying weaknesses • DOPs • Indicates trainees’ acquisition of practical skills and understanding of equipment • Indicates trainee interacts appropriately with patients • MiniCEX • Indicates trainees’ acquisition of clinical skills RCPath 07

  12. Pitfalls of WBA • To avoid time-consuming “busy” assessment forms, • the assessments are often simplified if not simplistic • computerised forms can over-simplify complex judgements • A robust rating can only occur if there are a number of independent observers – which can be difficult in small specialties • They have only been validated in a number of clinical settings but not in all pathology settings RCPath 07

  13. Beware! • Test methods can only be selected once the purpose and content are established • What is the best trade-off between fidelity and efficiency. • What is the programme for writing and refining test material? • How are marking schedules developed and reviewed where relevant? • Is new test material piloted before it is used? RCPath 07

  14. Principles of conduct for WBA • Assessment based on performance in the workplace and measured there • Observations can be combined from a number of assessors to provide a robust rating for the candidate • Assessment occurs during normal work immediately followed by feedback on their routine performance • WBA is not cast in stone – work in progress and subject to improvement following evaluation RCPath 07

  15. Who can be an assessor? • Assessors can be consultants who are supervisors as well as staff of all grades in contact with the trainee • Clinical Scientists • Nurses • BMS • Senior trainees • Autopsy room staff and others • No need for prior approval by RCPath • Need to follow WBA guidelines and are willing to engage in the process RCPath 07

  16. Who is it for? • Mandatory for all StRs (Specialty Registrars) appointed from August 2007 to one of the following: • Specialty training programme with a National Training Number (NTN) • Fixed Term Specialty Training Appointment (FTSTA) • Locum Appointment for Training (LAT) RCPath 07

  17. Feedback • One of the most important aspects of any assessment programme is the effect it has on learning and preparation for individuals and the profession. • How is feedback given to individuals and groups about the outcomes of an assessment? • Finally the mechanisms for appeal about the outcome should be documented and reported. RCPath 07

  18. RCPath 07

  19. Present and future • 2007 – Launch of tools and paper-based assessment forms. • 2008 – Launch of web-based system • Use data generated by all assessments including MSF and OSPE for validation • Ensure compliance with MMC Gold Guide and planned ARCP RCPath 07

  20. Acknowledgments • Sandra Dewar • Dr Trevor Gray • Dr Mary Thompson RCPath 07

  21. Tribute • Professor Dame Lesley Southgate • Professor Janet Grant RCPath 07

  22. RCPath 07

  23. RCPath 07

More Related