Integrated Assessment Framework for Curriculum 2019: Ensuring Performance Levels and Training Completion

1. Curriculum 2019: Programme of assessment Mr Rehan Khan, Assessment Advisor (Workplace Based)

2. rcog.org.uk/ Key points • The new curriculum will be an integrated framework of exams, assessments in the workplace and judgements made about a learner during their approved programme of training. • Current O&G ‘assessment at work’ methods have been fully reviewed as part of the curriculum development. • Assessment tools, which are well established in O&G training, will be both formative and summative and have been selected on the basis of their fitness for purpose and their familiarity to trainees and trainers. • The revised assessment tools will put the trainee at the heart of the assessment process, with a much stronger emphasis on the trainee as a reflective practitioner. • Assessors will make accountable, professional judgements on whether trainees have made sufficient progress throughout the training year to progress to the next year/level of training. • The successful completion of each stage of training will be dependent on achieving the expected level in all Capabilities in Practice (CiPs) and procedural skills, with the programme of assessment being used to monitor and determine progress throughout the programme.

3. rcog.org.uk/ Purpose of the programme of assessment To clearly communicate the expected levels of performance and ensure these are met on an annual basis and at other critical progression points and to demonstrate satisfactory completion of training as required by the curriculum.

4. rcog.org.uk/ What has changed? • Main changes • The successful completion of each stage of training will be dependent on achieving the expected level in all CiPs and procedural skills • Education Supervisors (ES) will now be required to make a global judgement as to whether trainees should progress to the next year of training • The Education Supervisors Report (ESR) has been redesigned to support the global judgement process • With multi-source feedback, a trainee will now be able to self-assessusing a modified TO1 form (SO) • A reflective element has been added to workplace-based assessment tools • A new assessment, the NOTSS tool, will be used to rate and give feedback on non-technical skills.

5. rcog.org.uk/ Curriculum 2019 assessments

6. rcog.org.uk/ MRCOG examination: Part 1, Part 2 and Part 3 Clinical Assessment • Part 1 MRCOG: • covers basic and applied sciences relevant to the clinical practice of O&G • summative assessment of CiP 6. • Part 2 MRCOG: • covers the knowledge required during clinical practice • summative assessment of CiPs 9-12 and elements of CiP 2. • Part 3 MRCOG: • covers the application of knowledge, clinical competencies and attitudes in clinical practice • summative assessment of CiP 1 and CiP 8 (to show the application of medical knowledge, clinical skills and professional values in the provision of high-quality and patient-centred care and effectiveness as a teacher) and CiP 13 and CiP 14 (to assess skills as a champion of healthcare in all groups within society and a promoter of health and disease prevention).

7. rcog.org.uk/ Progression through the training programme • Progression through the programme will continue to be determined by the ARCP process. • The assessment requirements for each indicative year of training are summarised in the table opposite. • The successful completion of each stage of training will be dependent on achieving the expected level in all CiPs and procedural skills. • Decisions on progression fundamentally rely on the professional judgement of the ES. • Towards the end of the training year, trainees will assess their own progression for each CiP and record this in the Training ePortfolio, signposting to the evidence that supports their rating.

8. rcog.org.uk/ Matrix of progression • This has been updated so that it is relevant for the new curriculum. • The matrix provides guidance on all aspects of training, both clinical and non-clinical, for each year of training. • It sets out the requirements for a satisfactory ARCP outcome at the end of each training year and critical progression points. • The new matrix of progression is available here.

9. rcog.org.uk/ Assessment of CiPs • Detailed guidance on the assessment of CiPs will be provided on Training ePortfolio. The guidance will cover: • what the CiP is about • statement of expectations • the kind of evidence that might be relevant to the CiP • when the CiP can be signed off. • The CiP is the fundamental basis of global judgement. • Trainees will be assessed throughout the training programme, allowing them to continually gather evidence of learning and to provide formative feedback. • Clinical Supervisors and others contributing to assessments will provide formative feedback to the trainee on their performance throughout the training year. • The ES will carry out an annual global judgement, and satisfactory sign off will indicate that there are no concerns and that the doctor in training meets or exceeds the minimum level of performance expected for completion of that stage of O&G training • The method of assessing CiPs varies depending on whether they are generic and non-clinical or a clinical CiP.

10. rcog.org.uk/ Assessment of generic and non-clinical specialty CiPs • With these CiPs trainees are required to make a self-assessment as to whether they meet expectations for the stage of training, linking to appropriate evidence in the ePortfolio. • ES will indicate on the ESR whether they feel a trainee is meeting expectations or not. • If trainees are not meeting expectations, an explanation will need to be given by the ES. • Trainees will need to meet expectations for the stage of training as a minimum to be judged satisfactory to progress. • Detailed guidance will be available on the Training ePortfolio. Overall, I consider that the trainee is meeting or exceeding expectations for this stage of training.

11. rcog.org.uk/ Clinical specialty CiPs • With clinical specialty CiPs, the trainee will make a self-assessment to consider whether they meet expectations for the year of training, using the five supervision levels listed opposite and linking to relevant evidence in the ePortfolio. • The ES will then indicate whether they feel the trainee is meeting expectations or not by assigning one of the five supervision levels. • Trainees will need to meet expectations for the year of training as a minimum to be judged satisfactory to progress. Overall, I consider that the trainee’s performance meets 1-5 of the clinical entrustability scale. Overall, I consider that the trainee is meeting or exceeding expectations for this year of training.

12. rcog.org.uk/ Clinical specialty CiPs

13. rcog.org.uk/ Assessment of procedural skills • doctors must demonstrate that they are capable of unsupervised practice in all clinical specialty CiPs and that they meet the minimum requirements for each of the generic CiPs. • Trainees should ideally receive training in procedural skills in a simulated setting before performing these procedures clinically, but this is not mandatory. • Assessment of procedural skills will be made using the OSATS tool. • When a trainee has been signed off as being able to perform a procedure independently, they are not required to have any further assessment (OSATS) of that procedure, unless they or their Educational Supervisor think that this is required (in line with the standard professional conduct).

14. rcog.org.uk/ The global judgement process: principles • The programme of assessment in the new curriculum relies on the ES making a professional judgment as to whether trainees have met the learning outcomes and expected levels of performance set out in the approved curriculum. • This is not a new concept, and actually something which many doctors are already doing daily on an informal basis. The new curriculum seeks to formalise the process. • ES will make accountable, professional judgements on whether progress has been made • The ES will then offer a global judgement as to whether the trainee should progress to the next year of training.

15. rcog.org.uk/ The global judgement process: principles (cont’d) • Each CiP will be globally judged against the expectations for the particular stage of training. • Detailed guidance on the expectations for each CiP will be provided on the Training ePortfolio. • There will be a difference between the global judgement of generic and non-clinical specialty CiPs, and clinical specialty CiPs. • The global judgement decision will be recorded on the ESR, which has been redesigned to support the new Curriculum. • Trainees will be given the opportunity to agree or disagree with the recommendations of the ES. Educational Supervisors report (ESR) The Educational Supervisors records annually a longitudinal, global report of a trainee’s progress over the full range of CiPs based on a range of assessments, including exams and observations in practice or reflection on behaviour by those who have appropriate expertise and experience. The ESR can incorporate commentary or reports from observations, such as from supervisors, or formative assessments demonstrating progress over time.

16. rcog.org.uk/ The role of evidence in the global judgement process • Evidence is a crucial concept in the new curriculum • Trainees will collect evidence to support their self-assessment, • ES will use evidence to reach a global judgement. • Summative assessment • The MRCOG examination: Part 1, Part 2 and Part 3 Clinical Assessment • Objective Structured Assessment of Technical Skills (OSATS) • Formative assessment • Case-Based Discussions (CbD) • Mini-Clinical Evaluation Exercise (mini-CEX) • OSATS • Team Observation (TO1), TO2 and Self-observation (SO) • Non-Technical Skills for Surgeons (NOTSS)  • Supervisor report • Educational Supervisor Report (ESR)

17. rcog.org.uk/ The global judgement process: how it will work • Trainee uploads evidence on to the Training ePortfolio throughout the training year. • At any stage, trainees can assess their progress for each CiP signposting to the evidence that supports this. • Once they feel they have enough evidence to complete the CiP, they add a self-assessment rating of their performance, stating that it is: • not meeting expectations • meeting expectations OR • exceeding expectations. • Before the ARCP, the ES will generate the ESR, which pulls together a summary of the trainees progress within each CiP. • The ES will review the evidence in the ePortfolio including workplace-based assessments, the TO2 and the trainee’s self-assessment and record their global judgement of the trainee’s performance in the ESR with commentary. • The ESR will make a recommendation to the ARCP panel as to whether the trainee has met the defined levels of achievement for the CiPs and acquired the procedural competence required for that year of training. • The ARCP panel will then make the final decision on whether the trainee can be signed off and progress to the next year or level of training.

18. rcog.org.uk/ Workplace-based Assessments and reflection • In line with Foundation training, all formative workplace-based assessments in the new curriculum will include a reflective element: • TO1 • Mini-CEX • OSATS • CBD • NOTSS. • Ensures reflection and feedback are an integral component of the assessment process. • Every clinical encounter will provide an opportunity for reflection and feedback, which should take place as soon as possible after events to maximise the benefit for the trainee. • The redesigned formative assessment forms have been piloted nationally and a box on Trainee’s reflection has been added to each of these forms on the Training ePortfolio.

19. rcog.org.uk/ Non-TechnicalSkills for Surgeons (NOTSS) system • Provides feedback on non-technical skills. • It enables behaviour in theatre to be observed and rated in a structured manner. • A pilot version of the framework in use since 2015 and the tool will be rolled out more widely with the introduction of the new curriculum.