Hospital Ownership and Performance: An Integrative Research Review

1. Preliminary: work in progress Comments welcome Hospital Ownership and Performance: An Integrative Research Review Research-in-Progress Seminar Stanford, May 11, 2005 Yu-Chu Shen Naval Postgraduate School and NBER Karen Eggleston, Joseph Lau, Christopher Schmid Tufts University Funded by grant #050953 under the Robert Wood Johnson Foundation’s Changes in Health Care Financing and Organization (HCFO) Initiative

2. Presentation Outline • Research objective • Brief theory background • State of the empirical literature • Scope of our integrative review • Analytical methods to synthesize literature • Results • Discussions

3. Mixed Ownership Is an Abiding Feature of Healthcare Delivery in the US Source: Eggleston (2004), based on Rorem (1930); Hayes (1954); American Hospital Association Hospital Statistics (various years).

4. Research Objective • Does ownership affect hospital performance (quality, finance, or provision of uncompensated care)? • Competing theories with contrasting predictions • Hundreds of empirical studies to date with conflicting findings • policymakers have little clear evidence • economics of ownership and behavior imperfectly understood

5. Empirical Predictions Consistent With Some Ownership Theories

6. Mixed Empirical Evidence • Studies differ widely in analytic methods • Mixed and inconclusive evidence on whether ownership differs and the magnitude of differences in quality, cost, and social benefits • We use meta analytical methods to combine quantitative evidence from different studies

7. Scope of the Integrative Review • Synthesize the main findings of the empirical literature between 1990 and July 2004 on hospital ownership and performance (published or unpublished) • Examine multivariate empirical studies of US acute general short stay hospitals; • Examine studies that compare differences between for-profits and nonprofits, between nonprofits and government, or both.

8. Scope of the Integrative Review • Focus on four broad categories of performance measures: • financial performance (efficiency, cost, revenue, profit) • quality / patient outcomes • uncompensated care or community benefits • Staffing • Presenting only findings from financial performance measures

9. Literature Selection Process • 1434 potentially relevant studies from 1990 to 2004 were identified and screened for retrieval through • search engines (EconLit, MedLine, Proquest, ABI) • contacting all corresponding authors of initially included studies

10. Defining Study Population

11. Inclusion and Study Design Criteria

12. Outcome Criteria and Other Exclusions

13. Number of Studies By Category of Hospital Performance

14. Detailed Financial Performance Categories

15. Detailed Financial Performance Categories

16. How much work is a systematic review? • Allen and Olkin (1999) analyze 37 meta-analyses • Average hours were 1138 per study • Based on their formula, it implied 1,044 hours for our review

17. Analytical Methods • A typical study estimates the impact of ownership on performance as follows: • The coefficients β1 and β2 capture the effect on Y of for-profit and public ownership, respectively, relative to nonprofit ownership

18. Defining Effect Size of Ownership Studies (1) • The goal of our integrative analysis is to answer the following questions: • What is the magnitude of the relationship between ownership and performance—what is the effect size? • How precise or reliable is this estimated effect size? • How do differences in analytic methods and other study features affect the estimates of effect size?

19. Defining Effect Size of Ownership Studies (2) • Problems with using β1 and β2 directly from studies • Heterogeneous dependent variables • Effect size can be measured in actual dollars or in percentage.

20. Partial Correlation Coefficient As Effect Size • Partial correlation coefficient, r, measures the correlation between a given ownership and Y controlling for the effect of X • It can be derived from commonly reported statistics: r= • It’s unit free, so comparable across a heterogeneous set of studies • Unlike t-statistics, magnitude of r does not depend on sample size

21. Estimating Confidence Intervals Around the Effect Size • Adjusted effect size = • Variance(Zr)= • 95% Confidence interval of the adjusted effect size=

22. Combining Effect Sizes Across Studies (1) • A common way to combine study results is to compute a weighted average effect size: • The weight that minimizes the variance of this measure is the inverse of the effect size variance from each study:

23. Issues In Combining Effect Size For Hospital Ownership Literature • Research questions are not homogeneous. • Studies vary widely in analytical methods. Need to categorize the methods in some ways. • Overlapping hospitals and data sources • Unlike randomized clinical trials with independent samples, there are fewer than 5000 general acute hospitals in the US. Many studies analyze almost the entire population of hospitals • Furthermore, most studies use one of two common data sources.

24. Research Questions Vary • Fixed- or random-effects models? • When the combined studies are a homogeneous set designed to answer the same question in the same population, a fixed-effects model is appropriate. • When heterogeneity is detected, random-effects models are used, which assume that there is no single truth, but a distribution of such truths.

25. Categorizing Analytical Methods • Three types of methodology rigor • Type 3: if a study meets both of the following conditions: (a) uses panel estimation or explicitly accounts for potential selection problem (b) includes two of the following three sets of controls: patient level, hospital level, market level • Type 2: if meets EITHER (a) or (b) • Type 1: if meets NEITHER (a) nor (b)

26. Overlapping Sample and Data Sources • Partial correlation coefficients are valid effect size measures when observations are correlated. • Meta regression has been suggested as a possible imperfect solution by including dummies of the common data sources. • No satisfactory solutions to date.

27. Meta Regression • The regression approach allows us to examine whether differences in effect sizes across studies can be explained by analytical methods, region studied, years covered, or other study features. • The dependent variable is the effect size from each study. • The explanatory variables are the empirical features of each study (differ across financial measures) • The model is necessarily parsimonious due to sample size issues.

28. Integrative Review of Hospital Overall Efficiency • Overall efficiency is usually defined as least cost production or least amount of input for a given level of output. • Two common ways to estimate overall efficiency: • Stochastic frontier approach • Data envelopment analysis (DEA) • Both are controversial (e.g. Newhouse 1994) • 10 studies contain N-F comparison • 7 studies contain N-G comparison

29. Efficiency: Summary of Effect Size By Methods (N-F)

30. Efficiency: Summary of Effect Size By Decade (N-F)

31. Efficiency: Summary of Effect Size By Covered Region (N-F)

32. Efficiency: Potential Publication Bias?N-F Comparison

33. Integrative Review of Hospital Cost: N-F Differences • Studies assume different functional forms for the cost model: • Log (total cost): • 6 studies/10 observations • Log (average cost per admission): • 5 studies/11 observations • Average cost per admission/discharge: • 3 studies/6 observations • Others: • 4 studies/4 observations

34. Cost: Summary of N-F Effect Size By Cost Definition Other Cost Def Log(Total Cost) Log(Avg Cost) Avg Cost

35. Cost: Summary of N-F Effect Size By Decades Data from 1980s Data from 1990s

36. Cost: Summary of N-F Effect Size By Covered Years 1 year of data Multiple year of data

37. Cost: Summary of N-F Effect Size By Method Types Method Type 1 Method Type 2 Method Type 3

38. Cost: Potential Publication Bias?N-F Comparison

39. Integrative Review of Hospital Revenue: N-F Differences • Studies assume different functional forms for the revenue model: • Log (average revenue): • 3 studies/4 observations • Average cost per admission): • 3 studies/3 observations • Returns on assets: • 5 studies/5 observations • Others: • 1 study/1 observations

40. Revenue: Summary of N-F Effect Size By Revenue Definition Log(average revenue) Average revenue Returns on assets

41. Revenue: Summary of N-F Effect Size By Covered Region

42. Revenue: Summary of N-F Effect Size By Method Types

43. Revenue: Potential Publication Bias?N-F Comparison

44. Integrative Review of Profit Margin: N-F Differences • Profit margins are usually defined in the form of (revenue-cost)/revenue.

45. Profit Margin: Summary of N-F Effect Size By Covered Region CA FL Urban National VA

46. Profit Margin: Summary of N-F Effect Size By Method Types

47. Profit margin: Potential Publication Bias? N-F Comparison

48. What Do We Learn? (1) • Evidence is pretty conclusive regarding revenue and profit margins • For-Profits tend to earn more revenue (per admission) and have higher profit margins • There is little evidence of any difference in cost between FP and NP hospitals • Evidence is inconclusive regarding efficiency. • Although almost all individual studies report significant findings, collectively their results are not consistent.

49. What Do We Learn? (2) • Functional forms and analytical methods matter • Weaker methods and functional forms tend to predict larger differences between nonprofits and for-profits • National samples tend to produce more conservative estimates of effect size • No evidence of publication bias