April 28 2014 State of the Art Nursing Conference Angie Andersen DNP, ACNP-BC

1. April 28 2014 State of the Art Nursing Conference Angie Andersen DNP, ACNP-BC Examination of the Interrater Reliability of a Palliative Care Assessment Tool in Patients at Hospital Admission: A Pilot Study

2. Faculty Disclosure Angela Andersen has no financial interest or arrangement that would be considered a conflict of interest.

3. Research Team • Angela Andersen, DNP, ACNP-BC, Nurse Practitioner, Palliative Care Department, The Nebraska Medical Center, Omaha, NE. • Mary Parsons, PhD, RN, Associate Professor and Chair DNP Program, Creighton University School of Nursing, Omaha, NE. •  Regina Nailon PhD, RN, Clinical Nurse Researcher, The Nebraska Medical Center Omaha, NE. • Sue Ann Gaster BSN, RN, Staff Nurse, Adult Progressive Care Unit, The Nebraska Medical Center, Omaha, NE. • Rachael Mooberry BSN, RN, Staff Nurse, Oncology-Hematology Specialty Care Unit, The Nebraska Medical Center, Omaha, NE. • Jane Meza, PhD, Professor College of Public Health Biostatistics, University of Nebraska Medical Center, Omaha, NE.

4. Goal of Palliative Care • Improve quality of life for patients and their families facing the problems associated with serious or life-threatening illness, through the prevention and relief of suffering World Health Organization , 2011

5. Interdisciplinary Team • Palliative care teams utilize an interdisciplinary approach in which physicians, nurses, chaplains, social workers, and other allied health professionals provide care Weissman & Meier, 2011

6. Palliative Care Consultation • Treat pain and other symptoms that can cause complications in hospitalized patients • Establish goals for care • Support family members in crisis • Plan for safe transitions from hospital to other settings Weissman & Meier, 2011

7. Levels of Care • Three levels of palliative care: • Primary • Basic skills and competencies required to manage the day-to-day patient care • Secondary • Treating physician refers to a specialist-level palliative care provider for management of complex or difficult problems • Tertiary • Education and research Von Gunten & Lupu, 2004; Weissman & Meier, 2011

8. How We Die • Nearly half of all Americans die in a hospital • 7 out of 10 Americans say they would prefer to die at home • Only 25 % of Americans actually die at home • More than 80% of patients with chronic disease say they want to avoid hospitalization and intensive care when they are dying Centers for Disease Control, 2005; Dartmouth Atlas of Health Care, 2005

9. How We Die • Researchers examined medical records for 840,000 people 66 or older who died in 2000, 2005, and 2009: • Increase use of hospice program in 2009, but more than a quarter of hospice use was for 3 days or less, and • 40% of those late referrals followed a hospitalization with an intensive-care stay • Patients receive aggressive care until time of death and did not receive full benefit of hospice care or program Teno et al., 2013

10. Literature Review • Extensive body of evidence demonstrates difficulties in providing adequate pain and symptom management, as well as inconsistent communication for hospitalized patients with serious or life threatening conditions Tilden et al., 1995; Hanson et al., 1997; Claessens et al., 2000; Lynn et al., 2000; Norton & Talerico, 2000; Norton et al., 2002

11. Literature Review • The aggressive care provided in hospitals during the last year of life accounts for approximately 12% of the U.S. health care budget and 27% of Medicare expenditures Centers Disease Control, 2009 • Left unchecked, it is projected that health care spending will increase 25% by 2030, largely because of the aging population and chronic disease Centers Disease Control, 2009

12. Literature Review • Evidence suggests patients and families with serious or life-threatening illness who received palliative care interventions along with standard care reported: • Improved physical and psychological symptoms • Improved quality of life • Longer median survival time • Improved family caregiver well-being Lautrette,2007 ; Wright et al., 2008; Bakitas et al., 2009; & Temel et al., 2010

13. Early Identification of Needs • Despite the evidence, transition from disease-directed treatment to an emphasis on palliative care often occurs within days of end of life, if at all Hui et al., 2010; Reville et al., 2010; Hi et al., 2012 • Early identification of palliative care needs is critical for clinicians to provide appropriate and timely interventions directed at the specific level of palliative care required by the patient and their family Weissman & Meier, 2011

14. Assessment Instrument • General lack of assessment techniques that would equip providers to identify palliative care needs in hospitalized patients • Although a variety of palliative care assessment instruments have been developed, non have examined reliability or validity to date Bradley & Brasel, 2009; Fins, Miller et al., 1999; Fins, Schwager et al., 2000; Imhof, Kaskie, & Wyatt, 2007

15. Evidence-based Instruments • Investigating the psychometric properties of an instrument is a common standard prior to implementing the tool in clinical practice http://www.jointcommission.org/accreditation/hospitals.aspx • The Centers for Medicare & Medicaid Services and Joint commission on Accreditation of Healthcare Organizations support the use of evidence-based instruments and practices by clinicians caring for hospitalized patientshttp://www.jointcommission.org/accreditation/hospitals.aspx

16. CAPC Criteria • The Center to Advance Palliative Care (CAPC) developed a set of criteria to identify patients at hospital admission that would be appropriate to receive further palliative care assessment and interventions Weissman & Meier, 2011

17. CAPC Criteria • No study has examined the reliability and other psychometric properties of the CAPC criteria • Reliability is a prerequisite for any kind of validity, and is the degree to which measurement error is absent from data Polit & Beck, 2008

18. Interrater Reliability • Interrater reliability is a specific type of reliability referring to the amount of agreement between different raters. • In the case of the CAPC criteria, interrater reliability is useful to measure whether two (or more) raters independently come to an exact or nearly exact agreement when scoring a patient Polit & Beck, 2008

19. Interrater Reliability • Although exact agreement of independent raters is ideal, a small difference in rating is of minor clinical relevance Polit & Beck, 2012 • Conversely, if the difference in assessment and scoring between raters increases, it is very likely this will have implications for clinical practice Polit & Beck, 2012 • For example, whereas one nurse considers a patient at risk for having unmet palliative care needs and provides intervention to address the specific level of palliative care required by patient and their family, another nurse may regard the patient not at risk and will not consider any further interventions

20. Purpose The purposes of this pilot study were twofold: • To establish the interrater reliability of CAPC criteria that identify and trigger primary palliative care assessment at hospital admission in adult patients admitted to oncology and progressive care units who received palliative care services during their hospitalization; and • To describe the CAPC criteria identified most frequently in study patients who met CAPC criteria at hospital admission

21. Methods • Design • Retrospective, descriptive, exploratory • Setting and Sample • Midwestern academic medical center • Randomized, purposive sample • Using a sample of patients known to have received palliative care services strengthened the study design and enhanced the study team’s ability to determine the interrater reliability of the CAPC criteria • A sample size of 100 was adequate to determine the instrument’s reliability using the kappa statistic, along with a 95% confidence interval. • Inclusion Criteria • 19 years or older • Admitted to the adult oncology or progressive care units between January 1 and December 31, 2011 who received palliative care services during their hospitalization

22. Instrument • The CAPC criteria comprise an instrument for use at hospital admission to identify patients whose conditions warrant a primary palliative care assessment • National consensus panel developed criteria from research findings, national standards, and expert opinion Weissman & Meier, 2011

23. Instrument • The CAPC criteria has primary and secondary criteria to facilitate identification of patients appropriate for primary palliative care assessment • The starting point for assessing any given patient using the primary and secondary criteria is the identification of patients with potentially life-limiting or life-threatening conditions Weissman & Meier, 2011

24. Instrument • Primary Criteria • Includes the 5 most important indicators identified by the consensus panel • These criteria are global indicators that represent the minimum that nurses and clinicians should use to screen patients at risk for unmet palliative care needs at hospital admission Weissman & Meier, 2011

25. Primary Criteria

26. Instrument • Secondary Criteria • More specific indicators of higher likelihood of unmet palliative care needs and are designed to be used as supplemental criteria in hospitals with more comprehensive palliative care services available

27. Secondary Criteria

28. Procedure • Nurse investigators independently reviewed the medical record of each patient for evidence of CAPC criteria present within 48 hours of patients hospital admission • The nurse investigators determined the presence of one or more CAPC criteria that identified the need for primary palliative care assessment • The principal investigator reviewed all 100 medical records and each co-investigator reviewed 50

29. Data Analysis • Inter-rater reliability was examined with the kappa statistic, along with a 95% confidence interval • A test for whether kappa is different from zero was also calculated • If the p-value < 0.05, we concluded that the kappa value was significantly different from zero

30. Oncology Unit Interrater Reliability Kappa Value /Agreement: 1.000 = perfect 0.99 -0.81 = almost perfect 0.80-0.61 = substantial 0.60 or less = poor • 1Unable To Determine

31. Oncology Unit Interrater Reliability Kappa Value /Agreement: 1.000 = perfect 0.99 -0.81 = almost perfect 0.80-0.61 = substantial 0.60 or less = poor • 1Unable To Determine

32. Progressive Care Unit Interrater Reliability Kappa Value /Agreement: 1.000 = perfect 0.99 -0.81 = almost perfect 0.80-0.61 = substantial 0.60 or less = poor • 1 Unable To Determine

33. Progressive Care Unit Interrater Reliability Kappa Value /Agreement: 1.000 = perfect 0.99 -0.81 = almost perfect 0.80-0.61 = substantial 0.60 or less = poor • 1Unable To Determine

34. Results • Study sample revealed most frequently identified CAPC Criteria: • Life-limiting condition • Surprise question

35. Discussion • Nurse investigators had perfect to substantial agreement for the majority of the CAPC criteria • Perfect to substantial agreement provides confidence in nurses’ abilities to administer and score the CAPC instrument for the study population

36. Discussion • Prior to making inferences about interrater reliability of CAPC criteria, it is important to note the limitations of the CAPC instrument • Poor level of agreement for four criteria: • Limited social support • Functional dependency • In the last 3 months decline in function • In the last 3 months decline in weight • Lack of operational definitions

37. Discussion • In clinical practice, it is common that a team of interdisciplinary clinicians provide care to patients and their families • Nurses on the team have an essential role in identifying unmet needs of patients and coordinating services • Therefore it is essential that any instrument used to assess patients for palliative care needs has findings that are repeatable between nurses

38. Study Limitations • Retrospective study design may have contributed to the nurse investigators’ inability to determine the presence or absence of each criterion • Quality of documentation • Investigators knowledge and experience • Generalizability of study findings: • Patients who received palliative care • Definition of “hospital admission”

39. Clinical Implications • Identification of palliative care needs is necessary for nurses and other clinicians to be able to provide interventions directed at the specific level of palliative care required by the patient and their family • Establishing interrater reliability of the CAPC criteria is a necessary first step in determining the utility of having registered nurses conduct the screening at hospital admission

40. Research Implications • Beginning of a process that will contribute to the availability of data that describe the characteristics of hospitalized patients appropriate for further palliative care assessment and intervention • Future examination of the interrater reliability of CAPC criteria: • Other patient populations • Formal education of nurse investigators to increase understanding of what each criterion is intended to measure • Describe clinically relevant differences between nurses’ disagreements

41. Conclusions • Nurses play an essential role in identifying hospitalized patients who are at risk for having unmet palliative care needs • Establishing interrater reliability of the CAPC criteria is essential to determining the utility of having RN’s conduct the screening at hospital admission of patients who are at risk for unmet palliative care needs

42. Conclusions • Based on the study findings, it is realistic and suitable for nurses to administer and score the CAPC criteria at hospital admission • Implementation of an established instrument will provide the structure and process needed to ensure consistent and timely identification of patients at risk for having unmet palliative care needs

43. Conclusions • This study is a first attempt at establishing psychometric properties of the CAPC criteria to identify and trigger further palliative care assessment at hospital admission

44. Questions?