1 / 43

Promoting Educational Research Capacity through Systematic Evaluation of Curricular Innovations

Promoting Educational Research Capacity through Systematic Evaluation of Curricular Innovations. Janet Landeen, Lynn Martin, Charlotte Noesgaard , & Kirsten Culver, McMaster Donna Carr, Conestoga Nancy Matthew- Maich , Mohawk. Changing Perspectives. Overview of Approaches. Today’s Session.

zavad
Download Presentation

Promoting Educational Research Capacity through Systematic Evaluation of Curricular Innovations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Promoting Educational Research Capacity through Systematic Evaluation of Curricular Innovations Janet Landeen, Lynn Martin, Charlotte Noesgaard, & Kirsten Culver, McMaster Donna Carr, Conestoga Nancy Matthew-Maich, Mohawk

  2. Changing Perspectives

  3. Overview of Approaches

  4. Today’s Session • Setting the Context-Curriculum Renewal • Gathering the Research Team • Identifying the Big Questions • Using Interpretive Description to Organize Approaches • Narrowing the Questions, Finding Resources, Inviting Others to the Dance • Sharing Progress to Date

  5. The Context

  6. Designing the Kaleidoscope Curriculum Experiences of the McMaster Mohawk Conestoga BScN Program

  7. Honouring the many authors of the Kaleidoscope

  8. Kaleidoscope… Kalos…beautiful Eidos…shape Scopeo…to reflect on

  9. Key Areas for Curricular Revision Integrated Knowing/Acting/Being Ways of Knowing Thinking Like a Nurse PBL/PBL Retention of Program Philosophy & Goals

  10. The Person in the Middle • Encounter the person (individual, family, group, community) • Use of narrative to have person come alive

  11. Turn the Kaleidoscope to a different aspect

  12. Clinical Reasoning & Judgment in Nursing (Tanner, 2006, p.208)

  13. Turn the Kaleidoscope to a different aspect

  14. Pedagogical Approach: Ways of Knowing in Nursing • Empiric (scientific, evidence based) • Ethical (moral component-Codes of Ethics) • Personal (consistent with therapeutic use of self) • Aesthetic (art of nursing) • Emancipatory (critical reflection & action based on inequities) (Carper, 1978; Chinn & Kramer, 2008)

  15. Turn the Kaleidoscope to a different aspect

  16. Integrated Approaches to Learning • Pathophysiology & Evidence Informed Decision Making • Scaffolded across multiple courses versus stand-alone courses (AACU, no date)

  17. So What?OrWho Cares?

  18. Gathering the Research Team • Interest in educational reform and research • Identification of key individuals- • Chair of Program Evaluation Committee • Director of Nursing Education Research Unit • Leaders within Program • Individuals looking for career growth opportunities • Graduate Students • Undergraduate Students

  19. Identifying the Big Questions

  20. Program Evaluation vs. Educational Research • Evaluation of all aspects of program • Real life/real time curriculum-as-lived versus curriculum-as-planned • Used to inform local decision-making • In-depth exploration of specific topics • Comparison group, before/after design, or systematic qualitative exploration of experience or process • Uncover broader knowledge relevant to others

  21. CASN Accreditation Standard 3.2.5 Ongoing comprehensive evaluation continually improves program outcomes. • Descriptor: The program is deliberative and responsive in making timely improvements based on data from ongoing rigorous and ethical evaluations from faculty, learners, graduates, employers, and others, through the application of the Key Elements. • Key Element: The program and curriculum are monitored and evaluated to ensure currency and relevance to nursing practice.

  22. Program Evaluation at McMaster Mohawk Conestoga • Comprehensive Program Evaluation Plan • Use of Stufflebeam’s CIPP Model (Context, Input, Process, & Product) to develop Evaluation Matrix • Matrix critiqued for congruence with program philosophy, Stake’s Responsive Evaluation Model, & Aoki’s suggestion of multiple perspectives (Aoki 1991; Stake 2003; Stufflebeam & Shinkfield 2007)

  23. Program Evaluation Situational Issues Institutional Priorities Accreditation Standards Student Expectations Specific Questions

  24. Program of Research Faculty Interest Funding Opportunities Informed by Program Evaluation Specific Research Project

  25. Evaluating the Impact of the Kaleidoscope Curriculum • Have the educational innovations of the Kaleidoscope Curriculum made a difference in the graduates of the BScN Program? • Are there unintended consequences of the curricular changes?

  26. Sub-questions • Do students achieve the same or better outcomes on standard measures of performance? • Do clinical faculty and clinical preceptors notice a difference in graduating (Level 4) students of the Kaleidoscope Curriculum? If so, are those differences consistent with the educational innovations?

  27. Sub-questions cont’d. • Do graduating students integrate knowledge of pathophysiology and evidence informed decision making into their clinical reasoning and judgement? • Do stories of the educational experience that graduates and faculty tell focus on the person (individual, family, group, community)? • Do graduating students utilize ways of knowing in their approaches to nursing practice?

  28. Interpretive Description

  29. Integrative description as organizing approach • When purpose is to gain knowledge that can be directly applied to enhance nursing education. • “requires an integrity of purpose deriving from two sources: (1) an actual practice goal, and (2) an understanding of what we do and don’t know on the basis of the available empirical evidence (from all sources).” (Thorne, 2008, p. 35)

  30. Developing the Approach • Do students achieve the same or better outcomes on standard measures of performance?

  31. Progress to Date

  32. Evaluating the Impact of Pedagogical Innovations: Does the Kaleidoscope Curriculum Make a Difference in the  Clinical Practice of Final Year Undergraduate Nursing Students? Kaleidoscope Research Team Funded by Centre for Leadership in Learning, McMaster University $8,000

  33. Evaluating Clinical Learning Outcomes • Integrative description • Focus groups or individual interviews with 30 Clinical Faculty who taught students pre and post Kaleidoscope • Individuals likely to be sensitive to change in students but least involved in curriculum discussions • Semi-structured interview questions • Graduate student Research Coordinator • Poised to begin recruitment

  34. Understanding the Experiences of Students Who are Enrolled in Joint College/University Collaborative Educational Programs Nursing: J. Landeen (Mac), N. Mathew-Maich (Moh), M. Parzen (Moh), & L. Hagermann (Con) Bachelor of Technology: L. Bolan (Mac) & D. Bender (Moh) Medical Radiation Sciences: M. Faquharson (Mac) & L. Marshall (Moh)

  35. “Dual Identity” Project • McMaster President’s “Forward with Integrity” Fund $5,000, matched by Dean of Engineering • Uses interpretive description methodology • 8 undergraduate students (2 per program & site) paid to develop full proposal, recruit participants, conduct focus groups, analyze results, & present findings • Nursing student researchers receive course credit prior to assuming paid role • Interdisciplinary faculty team mentoring & teaching students

  36. Examples of Other Research Projects • Evaluation of written clinical evaluation forms 2 years pre and 2 years post Kaleidoscope to assess degree of uptake of curriculum renewal-in proposal writing stage (Noesgaard et al) • Exploring the Meaning of Consistency in PBL Programs-study completed (Landeen & Jewiss) • What Makes an Effective Teacher? Listening to the Voice of Our Students- in data analysis (Matthew-Maich et al)

  37. Barriers to conducting educational research • Work load of faculty • Cultural & institutional differences between universities & college partners • Funding sources for educational research • Perceptions of value of educational research versus clinical research

  38. Facilitators to Educational Research • Resurgence of interest in educational research • Organizations such as International Society for Teaching and Learning in Higher Education (ISSOTL) • Accreditation standards • Multiplicity of venues for presenting and publishing Enriching the experiences of our students!

  39. References • Aoki, T. T. (1991). Layered understandings of orientation in social studies program evaluation. In Pinar. W.F. & Irwin, R.L. (Ed.), (2005). Curriculum in a new key: The collected works of Ted. T. Aoki. .(pp.167-185) Mahwah, NJ: Lawrence Erlbaum.. • Association of American Colleges & Universities. (no date) Integrative Learning VALUE Rubric. Retrieved from: http://www.aacu.org/value/rubrics/integrativelearning.cfm • Carper, B. (1978). Fundamental patterns of knowing in nursing. Advances in Nursing Science, 1(1), 13-23. • Chinn, P., & Kramer, M. (2008). Integratedtheory and knowledge development in nursing (7th ed.). St. Louis: Mosby Elsevier.

  40. References cont’d. • Stake, R. (2003). Responsive evaluation. In T. Kellaghan & D.L Stufflebeam (Eds). The international handbook of educational evaluation, . (pp. 63-68). Dordecht: Klower Academic Publishers. • Stufflebeam, D.L. & Shinkfield, A.J. (2007). Evaluation theory, models, and applications. San Francisco: Jossey-Bass • Tanner, C.A. (2006). Thinking like a nurse: A research-based model of clinical judgment in nursing. Journal of Nursing Education,45(6), 204-211 • Thorne, S.E. 2008. Interpretive description. Walnut Creek, CA: Left Coast Press

  41. Thank you landeen@mcmaster.ca

More Related