Scholarly project: Developing a clinical rubric

1. Scholarly project:Developing a clinical rubric Erin Kibbey Ferris State University

2. Objectives • Describe the project goals and process taken in developing a clinical rubric • Discuss relevant research and survey results influencing rubric development • Describe a plan for future implementation of clinical rubric

3. Project Description • Goals • Create rubrics for measuring clinical competency of critical care interns at Munson • Implement rubrics for use by preceptors working with the interns • Objectives • Obtain references containing information about the use of rubrics and their development • Collaborate with team and staff educators for input • Create an evaluation tool • Present rubrics to preceptors • Trial rubrics on units • Obtain and compile feedback

4. Procedure/Methodology “A carelessly planned project takes three times longer to complete than expected; a carefully planned project takes only twice as long” ~ Golub's Law

5. Procedure/Methodology

6. Procedure/Methodology - Step One • Research Topics • Clinical performance evaluation • Evaluation strategies • Using rubrics as a clinical evaluation method • Rubric development • Clinical competency development • Adult learning theory • Cognitive learning theory

7. Clinical Evaluation Research Observing performance & arriving at judgments Compares data collected about performance to a set of standards • Issues • Consistency • Subjective • Values • Norm-referenced: Compared to others • Criterion-referenced: Compared to pre-determined criteria • Known in advance & used as basis for evaluation (Bonnel, 2008; Gaberson& Oermann, 2010)

8. Best Practices for Clinical Evaluation • Extensive formative evaluation and periodic summative evaluation • Timely & continuous feedback with ways to improve • Based on preset outcomes, clinical objectives, or competencies • Without a guide to the process = No basis for evaluation • NEED a framework for observing performance and arriving at judgments • Preparation and tools to promote consistency in evaluation (Bonnel, 2008; Gaberson& Oermann, 2010)

9. Rating Scales & Rubrics • Creation and use of rating scales or rubrics is supported in a wide range of academic subjects • Economics, writing, speech, dentistry, and chiropractic medicine • (McGoldrick & Peterson, 2013; O’Donnell et al., 2011; Rezaei & Lovorn, 2010; Saxton, Belanger, & Becker, 2012; Xiaohua & Canty, 2012) • Can be used for self-evaluation • (Bonnel, 2008) • Despite the lack of consistent use of rubrics in clinical nursing education, rubrics have many benefits • (Frentsos, 2013)

10. What is a Rubric? Rubrics are “scaled tools with levels of achievement and clearly defined criteria placed in a grid. Rubrics establish clear rules for evaluation and define the criteria for performance” (O’Donnell, Oakley, Haney, O’Neill, & Taylor, 2011, p. 1163)

11. What Does the Research Say? Several Benefits • Support adult learning principles • Provide competency documentation required by regulatory agencies • Can improve quality of care • Allow more discrimination in judging behaviors • Can increase knowledge gain • (Bonnel, 2012; Frentsos, 2013; O’Donnell et al., 2011; Walvoord & Anderson, 2010)

12. Procedure/Methodology – Step Two The first step to rubric creation is reflection (Stevens & Levi, 2005) • Needs assessment survey • Target Population - • Critical care intern preceptors • 32 Question Survey Sent to 55 Preceptors

13. Respondent Demographics Department Years Precepting in Department

14. Respondent Demographics

15. Key Findings/Results 1

16. Key Findings/Results 2

17. Key Findings/Results 3

18. Key Findings/Results 4

19. Key Findings/Results 5

20. Key Findings/Results 6

21. Key Findings/Results 7

22. Key Findings/Results 8

23. Key Finding/Results 9

24. Procedure/Methodology – Step 3 • How is competence defined? • ANA (2010) • Expected level of performance that integrates knowledge, skills, abilities, and judgment • Can be influenced by the nature of the situation • IOM (2002) • Five Domains: Patient centered care, interdisciplinary team, evidence-based practice, quality improvement • QSEN adds to these domains: safety & informatics • (Cronenwett, et al., 2007)

25. Procedure/Methodology – Step 3 • Standards of Practice • QSEN - Quality & Safety Education for Nurses • ANA – American Nurses Association • AACN – American Association Of Critical-Care Nurses • IOM – Institute of Medicine • NCSBN – National Council of State Board of Nursing • Joint Commission • Benner’s Novice to Expert Model (ANA, 2010)

26. Rubric Development • 3 Main parts Scale, Dimensions, Description of dimensions (O’Donnell et al., 2011)

27. Rubric Development (Kirkpatrick & DeWitt, 2012; O’Donnell et al., 2011; Stevens & Levi, 2005)

28. Important Design Considerations • Ease of use • Valid & Reliable • Fair • Flexible & Adaptable • Evaluated (Bargainnier, 2003; O’Donnell et al., 2011; Stevens & Levin, 2005)

29. What’s the Next Step? • Feedback & evaluation from you! • Metarubric • Reliability and validity (Stevens & Levi, 2005; O’Donnell et al., 2011)

30. Questions and Discussion

31. References • American Nurses Association [ANA]. (2010). Nursing scope and standards of practice (2nd ed.). Silver Spring, MD: American Nurses Association. • Bargainnier, S. (2003). Fundamentals of rubrics. Retrieved from http://www.webpages.uidaho.edu/ele/scholars/practices/Evaluating_Projects/Resources/Using_Rubrics.pdf • Bonnel, W. (2012). Clinical performance evaluation. In D. Billings & J. Halstead (Eds.), Teaching in nursing: A guide for faculty (4th ed.). (pp. 485-502). St. Louis, MO: Elsevier Saunders. • Cronenwett, L., Sherwood, G., Barnsteiner J., Disch, J., Johnson, J., Mitchell, P., . . . Warren, J. (2007). Quality and safety education for nurses. Nursing Outlook, 55(3), 122-131. • Frentsos, J. M. (2013). Rubrics role in measuring nursing staff competencies. Journal for Nurses in Professional Development, 29(1), 19-23. • Gaberson, K. & Oermann, M. (2010). Clinical teaching strategies in nursing (3rd ed.). NY: Springer Publishing Company. • Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23-34. • Institute of Medicine [IOM]. (2002). Who will keep the public health: Educating health professionals for the 21st century. Washington, DC: National Academy Press.

32. References continued • McGoldrick, K., & Peterson, B. (2013). Using rubrics in economics. International Review of Economics Education, 12, 33-47. • O’Donnell, J.A., Oakley, M., Haney, S., O’Neill, P.N., & Taylor, D.  (2011). Rubrics 101: A primer for rubric development in dental education.  Journal of Dental Education, 75(9), 1163-1175. • Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15, 18-39. • Saxton, E., Belanger, S., & Becker, W. (2012). The critical thinking analytic rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments. Assessing Writing, 17(4), 251-271. • Stevens, D. D., & Levi, A. J. (2005). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Sterling, VA: Stylus. Retrieved from https://resources.oncourse.iu.edu/access/content/user/fpawan/L540%20_%20CBI/steven-rubrics.pdf • Walvoord, B., & Anderson, V. A. (2010). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass. • Xiaohua, H., & Canty, A. (2012). Empowering student learning through rubric-referenced self-assessment. Journal of Chiropractic Education, 26(1), 24-31.