210 likes | 423 Views
Doris V. Bennett BSN, RN, CCRN NUGR550 Evaluation of Practice. E- Portfolio and Simulation. 11/9/11 dvb. Objectives . Compare and contrast E-Portfolio with Simulation Evaluate the benefits of each Assess the limitations of each 11/9/11 dvb. E-Portfolio. What is it?
E N D
Doris V. Bennett BSN, RN, CCRN NUGR550 Evaluation of Practice E- Portfolio and Simulation 11/9/11 dvb
Objectives • Compare and contrast E-Portfolio with Simulation • Evaluate the benefits of each • Assess the limitations of each • 11/9/11 dvb
E-Portfolio • What is it? • How does it evaluate/assess? • Is it reliable or valid? • What is the educational effect? • What is the feasibility/practicality/efficiency of e-portfolio? • How acceptable is it as an evaluation method? • Is it generalizable to multiple learners? 11/9/11/dvb
E-Portfolio Is… • A collection of evidence/documentation that learning has occurred, or there was a change in knowledge, skills, behaviors or attitudes. • An assessment of prior learning • An introspective analysis of connections between learning and performance. • An evaluation of formal and informal learning • A tool that can increase self awareness 11/9/11 dvb
E-Portfolio Evaluates/Assesses By… • Consideration of documented descriptive personal content individually • Skilled assessors who view it holistically • Utilization of well defined rubrics • Often in triangulation with other supporting learning methods • Comparing the evaluations of more than 1 assessor 11/9/11 dvb
E-Portfolio Reliability/Validity… • Depends on the specific criteria to be identified • Depends upon the topics to be covered • Depends on the assessors – inter-rater reliability is a big concern 11/9/11 dvb
E-Portfolio Educational Effect… • Can increase awareness of reflective and work based learning • Can help with self-directed learning • Can have positive effect with faculty interaction and feedback • Requires positive instructor attitude • Must establish reasonable time demands • Must have clear objectives, in line with course outcomes • Allow for time to see improvement 11/9/11 dvb
E-Portfolio Feasibility/Practicality/Efficiency… • Most learners have computers/access • Software/programs must be user friendly • Links within programs improves use of tools • The busier the learner is, the less likely they are to use – requires commitment • Must “meet the learner needs” • Requires organizational support, and instructor support • Maintenance is big issue • Time consuming for both learner/instructor 11/9/11 dvb
E-Portfolio Acceptability… • Can be used for • Personal growth • Professional growth • Educational growth 11/9/11 dvb
E-Portfolio As A Generalizable Evaluation Tool… • Dependent upon goals and objectives • Can be used in a variety of educational/personal/professional levels • Requires skill of the evaluator • Requires commitment from both the learner and the instructor 11/9/11 dvb
Simulation … • What is it? • How does it evaluate/assess? • Is it reliable or valid? • What is the educational effect? • What is the feasibility/practicality/efficiency of simulation? • How acceptable is it as an evaluation method? • Is it generalizable to multiple learners? 11/9/11 dvb
Simulation … • Is suspended disbelief • Mimics reality and real work world • Requires rigorous curriculum development and learning opportunities AFTER • Robust needs assessment completion • Requires debriefing • Can develop insight and self-awareness • Empowering • Best practice for learning • Can be supplemented with reflective journaling 11/9/11 dvb
Simulation Evaluates/Assesses By… • Practicing in a safe environment • Learners have to opportunity to problem solve, develop psychomotor skills • Debriefing allows quality corrective feedback and reflection of performance • Captures specific behaviors and responses • Is generally formative, can be summative 11/9/11 dvb
Simulation Reliability/Validity… • Depends on the development, or use of pre-established scenarios • Depends on the tools used to evaluate the performances • Depends on the skill of the debriefer • Depends upon what the objectives of the simulation was 11/9/11 dvb
Simulation Education Effect… • Significant reports upon improvement in confidence and transference of skills from lab to practice • Can meet the ION requests for collaborative curriculum incorporation • Debriefing is the most important part of sim • Must use EBP standards • Must be relevant and real when designing scenario 11/9/11 dvb
Simulation Feasibility/Practicality/Efficiency… • Costly!! • Time intensive • Resource intensive – including the staff to run • Must have didactic education prior to simulation • Space constraints (depending upon level of fidelity) • Requires preparation 11/9/11 dvb
Simulation Acceptability… • Depends upon the content – whether is adaptable to general population or specific to particular area/group • Most levels of learners from novice student to veteran nurse can receive benefit • Can be organizational, community, academic, or a combination of any/all 11/9/11 dvb
Simulation As Generalizable Evaluation Tool… • Has the potential for use with all levels of education/locations • Depends upon availability of resources and content 11/9/11 dvb
Summary: • Portfolios are individual introspective look at change over time. • Subjective in assessment, however, should be triangulated for more accuracy. • Simulations are practice with reflection of practice, performed with group • Both require skill of the evaluators • Can be used together • Both can be formative and summative, however, primarily, both formative 11/9/11 dvb
References: • Buckley, S., Coleman, J., DAvison, I., Khan, K. S., Zamora, J., Malick, S., et al. (2009). The educational effects of portfolios on undergraduate student learning: a best evidence medical education (BEME) systematic review. BEME guide NO 11. Medical Teacher , 340-355. • Dagley, V., & Berrington, B. (2005). Learning from an evaluation of an electronic portfolio t support general practioners‘ personal development planning, appraisal and revalidation. Education for Primary Care , 567-74. • Eppich, W., Howard, V., Vozenilek, J., & Curran, I. (2011). Simulation-based team training in healthcare. Simulation in Healthcare , S14-S19. • Joosten-ten Brinke, D., Sluijsmans, D,. Jochems, W. (2010). Assessors’ approaches to portfolio assessment in assessment of prior learning procedures. Assessment & Evaluation in Higher Education, 59-74. • Nairn, E., Traynor, V., Williams, G., Chapple, M., & Johnson, S. (2006). Student nurses' knowledge, skills and attitudes towards the use of portfolios in a school of nursing. Issues in Clinical Nursing , 1509- 1520. • Reese, C. E., Jeffries, P. R., & Engum, S. A. (2010). Learning together: using simulations to develop nursing and medical student collaboration. Teaching with Technology/Student Collaboration , 33-37. • Schoening, A. M., Sittner, B. J., & Todd, M. J. (2006). Simulated clinical experience, nursing students' perceptions and the educators' role. Nurse Educator , 253-258. • Stefanski, R., & Rossler, K. L. (2009). Preparing the novice critical care nurse: a community-wide collaboration using the benefits of simulation. Journal of Continuing Education in Nursing , 443-451. • Tochel, C., Haig, A., Hesketh, A., Cadzow, A., Beggs, K., Colthart, I., et al. (2009). The effectiveness of portfolios for post-graduate assessment and education: BEME guide No 12. Medical Teacher . 11/9/11 dvb