Introduction & Evaluation Questions

Summary Evaluation of the Spiritual Care Demonstration Project: Staff, Patients, and FamilyGwen C. Uman, R.N., Ph.D.; Harold N. Urman, Ph.D.; and Adrienne Dellinger, MPH

Introduction & Evaluation Questions Archstone Foundation’s Spiritual Care Demonstration Project focused on the improvement of the quality of spiritual care provided by palliative care teams at nine Southern California hospitals. The overarching goal of the project was to examine how and to what extent each hospital fostered, facilitated, and sustained the development of quality spiritual care. Evaluation questions for the Spiritual Care Demonstration Project addressed these quantitative measures: • What are the barriers and challenges to improving spiritual care, and how are they being addressed? • How and to what extent has spiritual care improved? • What are the promising practices and lessons learned?

Methods Supplemental Quarterly Reports: All hospitals were required to submit quarterly reports tracking spiritual care activities and patient demographics, including age, race, ethnicity, religious preference, and diagnosis. In addition, the reports provided the number of chaplain referrals, initial and follow-up visits, and the number of spiritual screenings, histories, and assessments conducted. Patient, Family, Staff Surveys: Vital Research and the key stakeholders drafted three individual surveys for patients, family members, and staff. Each hospital modified the surveys as desired. Patient and family interviews were conducted face-to-face by hospital staff trained by Vital Research on structured interview techniques. Staff surveys were self-administered on paper or online utilizing SurveyMonkey. This summary evaluation presents data on spiritual care activities, as well as staff, patient, and family survey data from the hospitals. Due to IRB processes or other constraints, not all hospitals were able to collect data for each group. The table below depicts data collection efforts for this project.

Methods (cont.)

Barriers There were multiple barriers that prevented consistent data collection efforts or analysis. The most often cited issues by hospitals included the following: • Hiring a board-certified chaplain: Due to funds and/or scarcity of qualified personnel available for the position in palliative care. • Bureaucratic delays: Delayed access to Archstone funding, cumbersome Human Resources processes, and delayed contract approval for components of hospital goals. • Competing priorities at each hospital: These included items such as clinical workload, hospital-wide reorganization, hospital-wide roll-out or upgrades of electronic medical record systems. • Personnel: Difficulty staffing the project due to the hours required for evaluation components, and high staff turnover at the hospitals. • Standardized methodology: Lack of standardization amongst hospitals in terms of surveys utilized, data collection methods employed, and databases to track project outputs created difficulties in comparative analysis across hospitals.

Spiritual Care Activities - Results • Patients were extremely diverse, with 37% indicating Hispanic/Latino ethnicity, and 16% selecting Spanish as their main language. The majority were female (52%) and approximately 35% were over the age of 75. • Catholicism was the most cited religion (34%), followed by Protestantism (28%), and no religious affiliation (17%). Cancer was the most common diagnosis (44%) followed by cardiac problems (11%). • Patients received more chaplain referrals and visits as the project progressed, and over 70% were receiving spiritual screenings from Q4 to Q9. • Across the nine quarters, hospitals offered an average total of 10 trainings with an average of 17 participants (including a variety of disciplines) per training.

Spiritual Care Activities – Results (cont.)

Staff Survey Results Five hospitals administered staff surveys assessing changes in staff knowledge, self-efficacy regarding spiritual care competency, and opinions about spiritual care in their hospital. • As shown below, there were four common survey items across all five hospitals. All scores on the items increased from pre- to post-survey administration. • Hospitals also showed a slight increase in average scores from pre- to post-survey on overall Knowledge (N=4), and overall Self-Efficacy and Attitude (N=5) scores.

Staff Survey Results (cont.)

Patient / Family Survey Results Five hospitals conducted patient and family member interviews assessing both patient and family member experiences with the spiritual care team and hospital. • The average scores for patient experience increased in four out of six areas depicted below. By post-survey, the average patient score ranged from 69% to 92%. • On average, over 75% of patients reported being offered a chaplain visit, accepting the visit, or having a member of the spiritual care team visit on the post-survey. • By post-survey, family members’ assessment of patient experience ratings of Agree and Strongly Agree increased in three of four items common to the five hospitals on a scale of 1 (Strongly Disagree) to 4 (Strongly Agree).

Patient / Family Survey Results (cont.)

Conclusions Spiritual Care Activity Results: • Hospitals demonstrated an increase in screening patients, chaplain referrals, and chaplain visits/revisits to patients, suggesting important increases in opportunities to detect and treat spiritual distress. • New trainings were offered at all hospitals targeting different disciplines, adding to the quality of spiritual care provided to patients. Staff Survey Results: • In the face of widely varied training targets, intensity, and methods, overall knowledge about spiritual care increased slightly. • Among staff in five hospitals, their perceived self-efficacy in providing spiritual care and opinions about the quality of spiritual care increased from pre- to post-survey. • Increases in spiritual care activities and training may have led to better staff knowledge, more positive opinions about their hospital’s spiritual care, and feelings of greater competency in providing appropriate care within their scope of practice. Patient/Family Survey Results: • More spiritual care activities, such as spiritual screenings and chaplain visits, combined with systematic and ongoing training for staff, may have resulted in increased satisfaction among patients and family members.

Lessons Learned Overall, the Spiritual Care Demonstration Project was successful in implementing and improving spiritual care in palliative care settings. In conducting data collection and analysis, the following lessons were learned: • Funding: Hospitals needed additional funding for a number of reasons, including better salaries for staff to reduce turnover, more money to allow dedication of a staff member solely to collect evaluation data, and to prevent reallocation of funds from their own budgets to cover costs towards this project. • Extension: Many hospitals were unable to initiate data collection until nearly Q4 due to the multi-component and large-scale nature of the project. As such, seven of nine hospitals were provided a no-cost extension to allow for completion of their project components and goals. With this in mind, future spiritual care projects should be extended in length and funded in entirety. • Electronic Medical Record: Hospitals with EMRs in place had difficulties modifying the existing systems, but had an advantage over those without EMRs in their ability to track site activity components. Some hospitals implemented EMRs during the project period, but not all systems were comparable.

Recommendations For evaluating future multi-site spiritual care improvement projects, Vital Research suggests the following ways to enhance the validity of assessing project impact: • Measurement: Hospitals should utilize the same instruments and administration methods to make the evaluation stronger and more meaningful. • Adding to the body of knowledge: Additional measurements should be considered, such as measuring spiritual distress including what type of distress, frequency of suffering, and what was done to manage or relieve the distress. • Sustainability: Ensure that data elements directly related to sustainability of spiritual care are incorporated and sufficiently funded to provide evidence of importance and necessity to hospital decision makers.

Acknowledgements Vital Research would like to acknowledge the data collection efforts of all nine hospitals, as well as thank the City of Hope Convening Center for evaluation support. In addition, we thank Archstone Foundation for providing the opportunity to evaluate this meaningful project.

Introduction & Evaluation Questions