Competency Based Fellowship Program CBFP Congress Presentation

4. Medical workforce issues growing need for mental health specialists; Current time taken to complete program Unmet needs across multiple levels including Provision of health care �Crisis� orientation focusing on acute settings Medical workforce issues growing need for mental health specialists; Current time taken to complete program Unmet needs across multiple levels including Provision of health care �Crisis� orientation focusing on acute settings

8. A number of processes were undertaken in determining the new curriculum and assessment program, including: a review of the current curriculum, involving focus groups with trainees and DOTs, and an online survey. The outcomes of this review identified some significant problems within the current curriculum. a comprehensive literature review was undertaken examining various curricula frameworks for contemporary training programs. Based on this literature review it was determined that the CanMEDS framework was the most appropriate. The adoption of the CanMEDS framework was subsequently ratified by GC in 2008 extensive (and continuing) discussion with the Royal College of Psychiatrists (UK) and the Royal College of Physicians and Surgeons of Canada regarding curriculum and curriculum implementation an extensive consultation process involving meetings in each of the states to liaise with trainees, DOTs and supervisors about competency-based curriculum and workplace based assessment � Collective understanding generated through the above processes informed the CIP Reference Committee�s development work for the new curriculum. Underpinning this process was a developmental model for training. This model is based on the premise that learning is a developmental process, with competencies being acquired over a developmental trajectory, from novice to advanced Ongoing development work is been undertaken through dedicated working groups, as required. A number of processes were undertaken in determining the new curriculum and assessment program, including: a review of the current curriculum, involving focus groups with trainees and DOTs, and an online survey. The outcomes of this review identified some significant problems within the current curriculum. a comprehensive literature review was undertaken examining various curricula frameworks for contemporary training programs. Based on this literature review it was determined that the CanMEDS framework was the most appropriate. The adoption of the CanMEDS framework was subsequently ratified by GC in 2008 extensive (and continuing) discussion with the Royal College of Psychiatrists (UK) and the Royal College of Physicians and Surgeons of Canada regarding curriculum and curriculum implementation an extensive consultation process involving meetings in each of the states to liaise with trainees, DOTs and supervisors about competency-based curriculum and workplace based assessment � Collective understanding generated through the above processes informed the CIP Reference Committee�s development work for the new curriculum. Underpinning this process was a developmental model for training. This model is based on the premise that learning is a developmental process, with competencies being acquired over a developmental trajectory, from novice to advanced Ongoing development work is been undertaken through dedicated working groups, as required.

13. AIM: to explain that progression of competence is reflected by the developmental trajectory As the trainee progresses through training, their level of competence will be reflected at a position on the developmental trajectory. In the ideal competency-based education model, progression through training would have no time boundaries and trainees would develop at the speed they are able to demonstrate their abilities, however in the RANZCP training program trainees will be expected to spend 60 months in training. This also allows for the development of tacit knowledge and the building of expertise. This slide highlights three stages of training. For progression from Stage 1 to Stage 2 and Stage 2 to Stage 3, trainees will need to demonstrate competent performance. Attainment of Fellowship is contingent on acquiring advanced competencies. (Please note that there are additional requirements to progression through training such as passing the OCI; this example only focuses on competencies). AIM: to explain that progression of competence is reflected by the developmental trajectory As the trainee progresses through training, their level of competence will be reflected at a position on the developmental trajectory. In the ideal competency-based education model, progression through training would have no time boundaries and trainees would develop at the speed they are able to demonstrate their abilities, however in the RANZCP training program trainees will be expected to spend 60 months in training. This also allows for the development of tacit knowledge and the building of expertise. This slide highlights three stages of training. For progression from Stage 1 to Stage 2 and Stage 2 to Stage 3, trainees will need to demonstrate competent performance. Attainment of Fellowship is contingent on acquiring advanced competencies. (Please note that there are additional requirements to progression through training such as passing the OCI; this example only focuses on competencies).

15. The light blue colour denotes these assessments as different from they way they are currently structured - more of a workplace based assessment. Also need to address this in terms of potential costs and impacts on health services. The CFE plans for local OCIs from 2012 is a distracter issue but may also need to be addressed especially with DOTs but service support would also be helpful for CFE and CBFP. Need to support local OCI concept but differentiate from workplace OCIs for CBFPThe light blue colour denotes these assessments as different from they way they are currently structured - more of a workplace based assessment. Also need to address this in terms of potential costs and impacts on health services. The CFE plans for local OCIs from 2012 is a distracter issue but may also need to be addressed especially with DOTs but service support would also be helpful for CFE and CBFP. Need to support local OCI concept but differentiate from workplace OCIs for CBFP

17. AIM: To provide an overview of the summative assessment program that was ratified by GC in November 2010 AIM: To provide an overview of the summative assessment program that was ratified by GC in November 2010

21.

36. AIM: to give a picture of what supervision time could look like. Many of the supervision sessions will occur as usual but some sessions will be structured meeting times following a workplace-based assessment. This time is to offer feedback and to discuss a plan to enhance future learning. The supervisor observes the trainee. At the end of the session the supervisor and the trainee meet � the meeting time and venue is pre-planned. The supervisor starts by asking the trainee to comment on their performance, asking the trainee to reflect on and evaluate/assess their work. The supervisor offers the trainee feedback based on their observations. In addition to verbal feedback the supervisor records comments and provides a rank score where appropriate on the WBA form. The supervisor and trainee discuss potential pathways to enhance further learning and agree on a course of action and learning goals. In collaboration, a personal learning plan is detailed. Both supervisors and trainees will receive training in this process. AIM: to give a picture of what supervision time could look like. Many of the supervision sessions will occur as usual but some sessions will be structured meeting times following a workplace-based assessment. This time is to offer feedback and to discuss a plan to enhance future learning. The supervisor observes the trainee. At the end of the session the supervisor and the trainee meet � the meeting time and venue is pre-planned. The supervisor starts by asking the trainee to comment on their performance, asking the trainee to reflect on and evaluate/assess their work. The supervisor offers the trainee feedback based on their observations. In addition to verbal feedback the supervisor records comments and provides a rank score where appropriate on the WBA form. The supervisor and trainee discuss potential pathways to enhance further learning and agree on a course of action and learning goals. In collaboration, a personal learning plan is detailed. Both supervisors and trainees will receive training in this process.

37. AIM: to introduce the 5 A�s model to support supervisors as a process when conducting workplace-based assessments The 5 A�s model (6) is used to develop a personal action plan for patients. This model and its processes can be utilised by supervisors when conducting workplace-based assessments. Assess: the supervisor observes the trainee in consultation with the patient. Using one of the workplace-based assessments tools the supervisor makes an assessment of the trainees performance Advice: the supervisor gives advice to the trainee Agree: collaboratively, the supervisor and trainee set goals Assist: the supervisor will assist with suggestions to achieve training goals. This may include removing barriers, problem solving activities, further reading etc. Arrange: specify a time and date to re-assess level of competence (6) Glasgow RE, Emont S, Miller DC. Assessing delivery of the five As for patient-centred counselling. Health Promotion International 2006;21(3):245-55AIM: to introduce the 5 A�s model to support supervisors as a process when conducting workplace-based assessments The 5 A�s model (6) is used to develop a personal action plan for patients. This model and its processes can be utilised by supervisors when conducting workplace-based assessments. Assess: the supervisor observes the trainee in consultation with the patient. Using one of the workplace-based assessments tools the supervisor makes an assessment of the trainees performance Advice: the supervisor gives advice to the trainee Agree: collaboratively, the supervisor and trainee set goals Assist: the supervisor will assist with suggestions to achieve training goals. This may include removing barriers, problem solving activities, further reading etc. Arrange: specify a time and date to re-assess level of competence (6) Glasgow RE, Emont S, Miller DC. Assessing delivery of the five As for patient-centred counselling. Health Promotion International 2006;21(3):245-55

38. AIM: to provide a brief overview of the different workplace-based assessment tools Workplace-based assessment measures what doctors actually do in practice; evaluating a range of competencies that a trainee uses during day-to-day encounters with patients. The RANZCP workplace-based assessment (WBA) tools are for formative evaluation of trainees within training. The primary purpose of this tool, along with all WBA tools is to promote learning for a trainee by providing structured feedback on performance within an authentic workplace context. The value of WBA tools for trainees, lies in the opportunity it provides for immediate structured feedback on their performance, supporting and enhancing learning. Convincing evidence supports the notion that systematic feedback through workplace-based assessment can change clinical performance. These tools will be particularly invaluable when supervising trainees experiencing difficulties. Case-based Discussion (CbD) CbD is a structured interview designed to explore professional judgment in clinical cases selected and presented by the trainee. The CbD is conducted in the workplace, during a meeting between a supervisor and trainee. Mini-Clinical Evaluation Exercise (mini-CEX) The mini-Clinical Evaluation Exercise is a 15 minute snapshot of doctor/patient interaction. A supervisor observes the trainee and provides feedback about their performance. Over the course of training, the mini-CEX is repeated in multiple settings, looking at various clinical skills with different observers/assessors. Observed Clinical Activity (OCA) Similar in structure to the Observed Clinical Interview (OCI), the Observed Clinical Activity formative assessment (OCA) requires trainees to be observed for the duration of a clinical encounter; trainees will then be marked on a series of competencies and provided with immediate feedback. Professional Presentation The Professional Presentation tool evaluates a trainee during a professional presentation to varying audiences and provides feedback to the trainee about their performance, for example grand round presentations and journal clubs. AIM: to provide a brief overview of the different workplace-based assessment tools Workplace-based assessment measures what doctors actually do in practice; evaluating a range of competencies that a trainee uses during day-to-day encounters with patients. The RANZCP workplace-based assessment (WBA) tools are for formative evaluation of trainees within training. The primary purpose of this tool, along with all WBA tools is to promote learning for a trainee by providing structured feedback on performance within an authentic workplace context. The value of WBA tools for trainees, lies in the opportunity it provides for immediate structured feedback on their performance, supporting and enhancing learning. Convincing evidence supports the notion that systematic feedback through workplace-based assessment can change clinical performance. These tools will be particularly invaluable when supervising trainees experiencing difficulties. Case-based Discussion (CbD) CbD is a structured interview designed to explore professional judgment in clinical cases selected and presented by the trainee. The CbD is conducted in the workplace, during a meeting between a supervisor and trainee. Mini-Clinical Evaluation Exercise (mini-CEX) The mini-Clinical Evaluation Exercise is a 15 minute snapshot of doctor/patient interaction. A supervisor observes the trainee and provides feedback about their performance. Over the course of training, the mini-CEX is repeated in multiple settings, looking at various clinical skills with different observers/assessors. Observed Clinical Activity (OCA) Similar in structure to the Observed Clinical Interview (OCI), the Observed Clinical Activity formative assessment (OCA) requires trainees to be observed for the duration of a clinical encounter; trainees will then be marked on a series of competencies and provided with immediate feedback. Professional Presentation The Professional Presentation tool evaluates a trainee during a professional presentation to varying audiences and provides feedback to the trainee about their performance, for example grand round presentations and journal clubs.

40. AIM: to consider the implications of formalising supervision Time The WBAs have been structured to comfortably fit within the current one hour of one-on-one supervision time. Not all one-on-one supervision times will be used for workplace-based assessments. The number will vary for each trainee. WBAs are also formalising current daily activities such as grand round presentations and journal clubs (professional presentation). Knowledge Training workshops, resources and professional development opportunities will be made available for supervisors to up-skill in teaching and learning and contemporary approaches to all aspects of feedback, evaluation and the assessment of performance. Trainee difficulties WBA tools provide an objective review of the trainees� performance and highlight any issues early in the training and/or rotation. WBA provide objective data that can be used in discussions with the DOT or local training coordinator.AIM: to consider the implications of formalising supervision Time The WBAs have been structured to comfortably fit within the current one hour of one-on-one supervision time. Not all one-on-one supervision times will be used for workplace-based assessments. The number will vary for each trainee. WBAs are also formalising current daily activities such as grand round presentations and journal clubs (professional presentation). Knowledge Training workshops, resources and professional development opportunities will be made available for supervisors to up-skill in teaching and learning and contemporary approaches to all aspects of feedback, evaluation and the assessment of performance. Trainee difficulties WBA tools provide an objective review of the trainees� performance and highlight any issues early in the training and/or rotation. WBA provide objective data that can be used in discussions with the DOT or local training coordinator.

41. AIM: to highlight the significance of feedback to the performance of the trainee Feedback in an assessment for learning context occurs while there is still time to take action. It has more value and authenticity when it is immediate to the task. It functions as a global positioning system, offering descriptive and specific information about the trainee�s observed performance relative to the intended learning outcomes and standard. Both the supervisor and the trainee should use formative assessment results to make decisions about what actions to take to promote further learning. Feedback is formative: this is essential to guiding the learner�s participation in the educational process. Feedback in an assessment for learning context occurs while there is still time to take action. It should offer descriptive information about the work relative to the intended learning goal � competencies Feedback aids learning: It is a process which generates discussion, allows the asking of questions and is an opportunity for clarity on benchmarking of standards Feedback reinforces good practice: it reinforces continued good practice and motivates the trainee Feedback corrects performance: it clarifies the expected standard of performance and encourages the trainee to modify their behaviour Feedback helps to close the gap: it suggests a route of action trainees can take to close the gap between where they are now and where they need to be Feedback should be immediate: most gains are from immediate feedback. Feedback should be specific: effective descriptive feedback focuses on the intended learning, identifies specific strengths and points to areas needing improvement Feedback should be frequent: it is likely that trainees will ask for more frequent feedback in their pursuit of competence. Feedback can be planned and can be casual: part of the WBA methodology is that a time and place is set in advance to provide feedback to the trainee. However, not all feedback needs to be given in this way � current practice should continue. AIM: to highlight the significance of feedback to the performance of the trainee Feedback in an assessment for learning context occurs while there is still time to take action. It has more value and authenticity when it is immediate to the task. It functions as a global positioning system, offering descriptive and specific information about the trainee�s observed performance relative to the intended learning outcomes and standard. Both the supervisor and the trainee should use formative assessment results to make decisions about what actions to take to promote further learning. Feedback is formative: this is essential to guiding the learner�s participation in the educational process. Feedback in an assessment for learning context occurs while there is still time to take action. It should offer descriptive information about the work relative to the intended learning goal � competencies Feedback aids learning: It is a process which generates discussion, allows the asking of questions and is an opportunity for clarity on benchmarking of standards Feedback reinforces good practice: it reinforces continued good practice and motivates the trainee Feedback corrects performance: it clarifies the expected standard of performance and encourages the trainee to modify their behaviour Feedback helps to close the gap: it suggests a route of action trainees can take to close the gap between where they are now and where they need to be Feedback should be immediate: most gains are from immediate feedback. Feedback should be specific: effective descriptive feedback focuses on the intended learning, identifies specific strengths and points to areas needing improvement Feedback should be frequent: it is likely that trainees will ask for more frequent feedback in their pursuit of competence. Feedback can be planned and can be casual: part of the WBA methodology is that a time and place is set in advance to provide feedback to the trainee. However, not all feedback needs to be given in this way � current practice should continue.

42. Entrustment would reflect not only confidence in a trainee being able to competently complete an activity, but also that the trainee would recognise when they need additional supervision Entrustment would reflect not only confidence in a trainee being able to competently complete an activity, but also that the trainee would recognise when they need additional supervision

43. EPAs should be activities that are undertaken regularly in everyday practice The primary indication of the standard of competence of a trainee is the level of supervision deemed necessary for them undertaking particular activities Requires greater scrutiny of trainees in practice but allows for exercising of the expert judgement of supervisors � debate currently underway about the where the line should be drawn: patient safety VS audacity of supervisors in letting trainees practice unsupervised� When unpacked the constitutive elements of these activities are numerous � all EPAs are likely able to be defined in terms of several of the CanMEDS roles EPAs should be activities that are undertaken regularly in everyday practice The primary indication of the standard of competence of a trainee is the level of supervision deemed necessary for them undertaking particular activities Requires greater scrutiny of trainees in practice but allows for exercising of the expert judgement of supervisors � debate currently underway about the where the line should be drawn: patient safety VS audacity of supervisors in letting trainees practice unsupervised� When unpacked the constitutive elements of these activities are numerous � all EPAs are likely able to be defined in terms of several of the CanMEDS roles

44. Result: 4 EPAs were defined with timelinesResult: 4 EPAs were defined with timelines

49. Estimated as maximum possible impact to Supervisors � modelled to account for the maximum possible supervision activities Individual Supervision: includes mini-CEX preparation time EPAs: estimated at 2 per year plus preparation time Admin: training in the mini-CEX, EPAs + 12 others. After this start up it remains as supervision, paperwork, consultation Exam Preparation Exam co-ordination: per site Other Training: in work hours Unknowns include examination preparation time, additional administration costs, transition costs. Time dedicated to supervision is intended to remain at the current minimum of four hours per week for 40 weeks of training annually.� Of this total supervision time, one hour per week will remain allocated to individual supervision of clinical work.� � How much DOT time is envisaged under the�CBFP? The main time cost to Directors of Training (DOTs) will be in the initial training of the new program; DOTs will receive training and may be involved in training their supervisors.� In terms of program changes, it is not anticipated that there will be any additional work for DOTs. This will be carefully monitored during the implementation process. �The formative assessment forms that the supervisor undertakes with the trainee will not come back to the DOT unless there is an issue with the trainee�s progress.� The end of rotation reports will be sent to the DOTs. Estimated as maximum possible impact to Supervisors � modelled to account for the maximum possible supervision activities Individual Supervision: includes mini-CEX preparation time EPAs: estimated at 2 per year plus preparation time Admin: training in the mini-CEX, EPAs + 12 others. After this start up it remains as supervision, paperwork, consultation Exam Preparation Exam co-ordination: per site Other Training: in work hours Unknowns include examination preparation time, additional administration costs, transition costs. Time dedicated to supervision is intended to remain at the current minimum of four hours per week for 40 weeks of training annually.� Of this total supervision time, one hour per week will remain allocated to individual supervision of clinical work.� � How much DOT time is envisaged under the�CBFP? The main time cost to Directors of Training (DOTs) will be in the initial training of the new program; DOTs will receive training and may be involved in training their supervisors.� In terms of program changes, it is not anticipated that there will be any additional work for DOTs. This will be carefully monitored during the implementation process. �The formative assessment forms that the supervisor undertakes with the trainee will not come back to the DOT unless there is an issue with the trainee�s progress.� The end of rotation reports will be sent to the DOTs.

50. Supervisors will continue with the current allocated supervisor time of 4 hours, of this time a minimum of 1 hour individual supervision time. Health services will continue to release supervisors to attend training and other relevant professional development sessions. A cultural change is required to the Supervisors approach to supervision, in particular around the formalisation of supervision and undertaking assessment tasks.Supervisors will continue with the current allocated supervisor time of 4 hours, of this time a minimum of 1 hour individual supervision time. Health services will continue to release supervisors to attend training and other relevant professional development sessions. A cultural change is required to the Supervisors approach to supervision, in particular around the formalisation of supervision and undertaking assessment tasks.

52. Training Directors will be one group of Trainers to train Supervisors. It was suggested that DOTs could reasonably train between 12-20 Supervisors. Again this will vary for each region. It will be necessary to recruit �others� to meet the shortfall. �Others� is yet to be determined but could be from within the Health Services for example SCOTs or individuals from outside of the Health Services with an education and training background. What role should DOTs have????? Training Directors will be one group of Trainers to train Supervisors. It was suggested that DOTs could reasonably train between 12-20 Supervisors. Again this will vary for each region. It will be necessary to recruit �others� to meet the shortfall. �Others� is yet to be determined but could be from within the Health Services for example SCOTs or individuals from outside of the Health Services with an education and training background. What role should DOTs have?????

53. Meeting outcomes: To explain the CBFP and plans To understand issues the health services face throughout the roll out process and beyond To determine collaboration process, both short and long term To address current concerns Meeting outcomes: To explain the CBFP and plans To understand issues the health services face throughout the roll out process and beyond To determine collaboration process, both short and long term To address current concerns