Current Controversies in Nurse Staffing Research

1. Current Controversies in Nurse Staffing Research Barbara A. Mark PhD, RN, FAAN Sarah Frances Russell Distinguished Professor School of Nursing The University of North Carolina at Chapel Hill

3. Theoretical Issues • Research Design Issues • Data Issues • Interpretation Issues and Strength of Evidence

4. The Need for Better Theory • To explain why empirical results occur • To predict conditions under which observed relationships are most and least likely to hold true • To develop theoretically-based, targeted interventions aimed at improving quality of care

5. Better Theory Requires • Precise conceptual and operational definitions of major constructs • Clear explication of proposed causal relationships • Identification of critical mediators and moderators of the proposed relationships

6. Conceptual Definitions of Nurse Staffing • What does “nurse staffing” mean? • Is it the NUMBER of RNs who are providing care? • Is it the number of HOURS of care delivered to patients by RNs?

7. Is it the PROPORTION of care delivered to patients that is delivered by RNs? • Is it WHAT RNs do? • Is it the COMPETENCE of the RNs who are providing care? • Is it the QUALITY of the actual care delivered by RNs?

8. Common Operational Definitions of “Nurse Staffing” • RN hours of care • RN hours as a percent of all hours • Percent of total staff who are RNs • FTEs per 1000 patient days • FTEs per 1000 case mix adjusted patient days

9. These Arise from Different Conceptual Approaches • Capacity: actual or potential ability to perform • Availability: ready for use • Actuality: actual existence; reality

10. Nurse hours reflect average staffing across a 24 hour period, not fluctuations in census, scheduling patterns in different shifts and period of the year • Interpretation of an “FTE” • Overlooks the issue of staffing “adequacy” … Do we need more under-performing nurses (i.e., “warm bodies”)?

11. Example: Unit with All RN Staff • Staffing is operationally defined as “proportion of total staff who are RNs” • Assumption is that patients will receive care from RNs. However… • Poor management, inadequate support services, poor medication/delivery and supply systems, etc…. • Results in patients actually receiving few hours of care …. albeit all from RNs

13. The primary question • What is the theoretical rationale for expecting that variations in nurse staffing will cause variations in quality of care? • WHY more RNs, better outcomes? • Fagin’s surveillance function • Silber’s failure to rescue • Cure, care, coordination (Mauksch, 1966!)

14. “Nurse Sensitive” Outcomes • Issues raised about staffing conceptualization and operationalization also apply to selection of nurse sensitive measures • What does it really mean when we say a selected variable is “nurse sensitive?” • How do we demonstrate that an outcome is “nurse sensitive”?

15. Are there outcomes of hospital care that are not nurse sensitive? • Might there be a continuum of nurses’ contributions to patient outcomes? • Does nurse staffing relate differently to adverse outcomes (mortality, complications, etc.) than more positive outcomes (self-care ability, illness-related self-efficacy, disease management knowledge and skill)?

16. Opening The Black Box: Mediation Y X M Addresses the WHY and HOW of a hypothesized relationship

17. Possible Mediators of the Staffing-Quality Relationship • Professional practice environment • Communication quality • Nurse-physician collaboration

18. 2) 30% variance Outcome Staffing 3) 9% variance 1) 32% variance 3) 19% variance Communication Quality

19. Opening the Black Box: Moderation X1 X2 Z X1X2 The interaction terms represents a joint relationship between the two independent variables and accounts for additional variance in the outcome beyond that explained by either variable alone. Addresses the issue of “under what conditions” does a relationship exist?

20. Possible Moderators • Managed care penetration • Safety climate • Urban/rural location

21. Moderating Effect of HMO Penetration Marginal effect of one-unit increase in RN FTEs/1000 inpatient days on mortality ratio (Mark, Harless & McCue, 2005)

22. Research Design Issues • Unit of analysis • Cross-sectional vs. longitudinal designs • Risk adjustment

23. Unit of Analysis • “Levels” in organizations • Hospital (i.e., organizational) • Work/Nursing unit • Individual • Implies a conceptual hierarchy in which one level is “nested” in higher levels • Conceptual approach should drive selection of unit of analysis (or multi-level approach) • Little agreement regarding which level is the most appropriate

24. Challenges • Hospital level analyses assume homogeneity of effects across work/nursing units • Adjusting standard errors: HLM-type models to deal with clustered data • Evaluating appropriateness of aggregating data from one level to another

25. Sampling issues are important, particularly if data are collected from individuals and aggregated to the hospital level • Concerns about data reliability, validity and representativeness when very small numbers of individuals provide data

26. Multi-Level Analyses • Few studies examine multi-level models • Mark, Salyer & Wan (2003) • McGillis-Hall et al., (2003) • Specification of correct multi-level model: • extremely important • conceptually demanding • statistically challenging (sometimes!)

27. Advantages Efficient, relatively easy to do Disadvantages Difficult to establish causality Takes a “one-time” slice of organizational action that may not be representative of critical processes that can best be observed over time Advantages Examines change over time Allows stronger causal inference than cross-sectional designs Disadvantages Tricky issue of lag structure and temporal ordering Extremely costly and time-consuming Research Designs Cross SectionalLongitudinal

28. Unaddressed Questions • How long does it take for changes in nurse staffing to have an impact on quality? Are they immediate or delayed? • What is the functional form of that trajectory? • Are some outcome measures more likely to be affected by short-term vs. long-term staffing changes? • Are there certain thresholds of nurse staffing that can be identified?

29. Risk Adjustment • Risk: “the possibility of suffering harm or loss” • Risk of what? death, complications, functional status, resources used (costs, LOS) • Severity from a medical perspective vs. severity from a nursing perspective (i.e., intensity of nursing care or acuity)

30. Common Risk Adjustment Strategies • Medicare Case Mix Index • developed as a measure of resource use, not for risk • used for Medicare patients • all patient CMI • Nursing Intensity Weights (developed in New York; updated in 2004) • DRG based • measure nursing resource utilization

31. APR-DRGs • AHRQ’s Clinical Classifications Software and comorbidity software • Medstat (or other proprietary) disease staging algorithms • Disease specific risk models (i.e. CABG, AMI) • Study specific risk models

32. Cautions with any risk model • Generally contain substantial measurement error • Depend on coded data from standard administrative and claims databases • Subject to inaccurate and incomplete coding

33. Systematic coding differences across institutions • Unmeasured aspects of risk known to the provider but not captured in the risk model • Complication or comorbidity?

34. Data Issues • Research question and unit of analysis drive data issues (although sometimes it is true that data availability drives the research!) • Different databases and data elements are differentially available depending upon unit of analysis and on whether primary or secondary data are used • In general, hospital level data are obtained through secondary data; unit level data are obtained through primary data collection

35. Ideal Data Characteristics • Reliable: should, as accurately as possible, reflect actual staffing and be free of systematic measurement error • Valid: should allow for direct measurement of staffing at the level theoretically appropriate for the study • Should be possible to disaggregate to the appropriate unit of analysis • Unfortunately, there is no database that meets all these requirements

36. Hospital Level Staffing Data • American Hospital Association • Total facility (i.e., hospital and long-term care unit) • Does not differentiate RNs working in in-patient and out-patient facilities • Does not differentiate RNs in direct care from those in support/supervisory roles • Does not report UAPs • Does not differentiate paid vs. productive time

37. CMS Provider of Services File • RNs and LPNs only • State databases • not available in all states • little consistency in definitions across states • allocation of nurses to relevant subunits may be extremely difficult

38. Unit Level Nurse Staffing Data • Frequently relies on primary data collection, which is extremely costly and time intensive • CalNOC, VANOD, MILNOD, NDNQI • California Office of State Health Planning and Development (OSHPD) has arguably the best data on nurse staffing • Service level, not unit level, however • Has agency hours • Reports productive hours

39. Staffing Data from Primary Data Collection • Still would like the data to meet the ideal characteristics addressed earlier • Now also have an additional human element (the data collector) • In multi-site studies, definitional problems reign supreme • For example, how is an FTE measured?

41. Consistency across sites is also problematic • Requires an enormous amount of follow up and obsessive attention to detail to “get it right.”

43. Strength of Evidence Issues • Measurement error • Omitted variable bias • The functional form of the relationship • Endogeneity • Selection bias

44. Measurement Error • Nurse staffing estimates based on data that do not measure staffing directly • Rely on assumptions in terms of allocating staffing estimates to the appropriate acute care unit • Administrative hours method • Adjusted patient days method • Revenue proportion method

45. Omitted variable bias • Improperly specified model • Most studies do not include other care providers (i.e., MDs), who clearly interact with RNs in provision of patient care • Strong conceptual basis for study, valid and reliable data, valid and reliable risk adjustment model all reduce likelihood of omitted variable bias

46. Functional Form • Many studies assume linearity in relationship between nurse staffing and outcomes • Curvilinear relationships • Examining marginal effects is important in terms of policy implications

47. Endogeneity • Correlation between an independent variable and the error term of the model • Example: a hospital experiences an unexpected increase in deaths or complications

48. Hospital responds by increasing the number of nurses • Difficult to estimate the true effect of nurse staffing on outcomes, since, in this case, the change in outcomes resulted in a change in nurse staffing

49. Possible Remedy • Instrumental variable approach • Theoretically difficult, since selection of the instrument must • Actually cause variation in the independent variable, but • Affect the outcome variable ONLY through its influence on the independent variable

50. Selection bias • Occurs when selection/participation is contingent upon the value of the dependent variable • Generally not dealt with, in either cross-sectional or longitudinal studies of nurse staffing and quality of care • Two-stage Heckman models; conceptually challenging • Propensity scores: Aiken, Smith & Lake (1994) matched 39 magnet hospitals along 12 control variables with 195 non-magnet hospitals