Topics in Health Economics – class 3

1. Topics in Health Economics – class 3 Matilde P. Machado matilde.machado@uc3m.es

2. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Title: Estimating the Quality of Care in Hospitals using Instrumental Variables • Typically the literature on Hospital Quality has used hospital-specific output measures such as mortality rates to compare hospital performance. • Potential Problem with these measures is that they suffer from ENDOGENEITY which can bias estimates in the wrong direction i.e. the best hospitals may show worse mortality data simply because they treat the most difficult patients.

3. Gautam Gowrisankaran and Robert Town (JHE, 1999) • In an equation such as: • It could be that the best hospitals receive the most difficult cases i.e. attract a sicker population of patients and have higher mortality rates. Therefore bi has two components (quality and severity) • Researchers try to solve this selection bias by including case-mix variables, i.e. variables that capture the severity of the patients received at each hospital (DRGs, main diagnosis, secondary diagnosis, demographic variables (age, gender, comorbidities))

4. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Still the problem remains if there is unobserved severity that is not captured by X but is correlated with the H dummies, in which case the best hospitals may look worse or not much better than the rest of hospitals (u’s are not equally distributed across the H’s, the H’s are endogenous): • To avoid this bias, Gowrisankaran and Town (1999): • Suggest to use IV that are correlated with H dummies but not to these unobservables. • This paper shows evidence of remaining selection even after controlling for observed patient characteristics.

5. Gautam Gowrisankaran and Robert Town (JHE, 1999) • The authors use mortality due to pneumonia from Medicare patients’ data (patients older than 65 years of age) because: • Pneumonia is a common disease among this age group • It may lead to death, it is the 4th leading cause of death among the elderly. In their sample in-hospital mortality due to pneumonia in 1989 was 17.9%. • The outcome depends in a great deal on the hospital’s procedures, therefore, an important part of the variance in outcomes can be explained by differences in hospital quality. • Note: It is important to notice that the paper can estimate only the quality of hospitals in pneumonia treatment (which may or may not be correlated with other aspects of hospital quality) • Other researchers have used pneumonia mortality as a benchmark for hospital quality (Keeler et al. JAMA , 1990, and McGarvey and Harper, 1993). • When diagnosed with pneumonia the patient/doctor have time to choose the “right” hospital, they do not need to hurry to the closest one (as with heart attacks). This allows to test whether selection is important. • Medicare patients pay the same out-of-pocket expenditures at every hospital

6. Gautam Gowrisankaran and Robert Town (JHE, 1999) • The Model of mortality and hospital choice • The Choice Equation: Once the patient becomes ill with pneumonia he (his physician) chooses the hospital based on: the perceived quality of treatment, the patient’s severity of illness, the cost to the patient, and the distance to the hospital.

7. Gautam Gowrisankaran and Robert Town (JHE, 1999) • The mortality Equation: They model it as a hazard function that gives the probability of a patient dying in a given day conditional on being alive at the beginning of the day. The hazard depends on the observed and unobserved severity of the patient’s illness, the number of days since admission and the quality of treatment received at the hospital chosen. Linear probability model bi are interpreted as hospital quality. Parameters of interest

8. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Notice that sit will affect the decision of which hospital to go to, more severe patients may choose higher quality hospitals, i.e. sit may be part of the error term uit ,which implies that the error terms in both equations are correlated, implying that ci is correlated with the error term in the mortality equation. • Deaths after discharge are not observed. • Important!: Assume sit(residual health status after controlling for all the Xs) and eit are identically and independently distributed in the population and orthogonal to the regressors xi and dt.

9. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Instruments: • There are J-1 endogenous variables (c1,….cJ-1), we need at least as many instruments. There is a reference hospital, example for J=2: • Distance of each patient to hospitals as IVs for ci. Comments: • A good IV must be correlated with ci and not with sit or eit. • A patient’s distance to hospital j is negatively correlated to the choice of that hospital so should be a good candidate. • A patient distance to a given hospital should not be correlated with the error term sit+eit under the assumption of equal distribution of sit • However, more severe patients may be willing to travel larger distances to be treated at a particular hospital. So distance of a given patient to the chosen hospital is correlated with severity.

10. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Imagine same distribution of severity in town A as in B. For simplicity, assume each town has a hospital in downtown i.e. symmetric distance of patients to A and B (dA and dB is independent of severity distribution) • Most severe patients (shown in red) are in the same proportion in both towns. • Hospital A is of higher quality – so all patients in region A go to A and all severe patients (in red) in region B go to A. So the average distance of patients that choose hospital A is higher than the average distance of patients that chose B. So distance to chosen hospital is correlated with severity (invalid instrument) a simple code A B

11. Gautam Gowrisankaran and Robert Town (JHE, 1999) • But is the assumption that severity is equally distributed a plausible one? Presumably more sick patients may prefer to leave closer to a good hospital. If this is so then their instrument is not valid. Patients in red are severe while patients in green are less severe. In this case, severe patients locate close to the best hospital, say A. So distance to A would be negatively correlated with severity, those with less distance to A are the most severe. The opposite for B. Distance to A and B would not be valid instruments B A Note: One instance in which this is so, is nursing homes which tend to locate near high quality hospitals. For these reason they discard patients admitted from nursing homes.

12. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Instruments (cont.): • Use 2J instruments: • dij≡ distance of patient i to each j hospital • exp(-fdij) ≡ a non-linear function of distance. • The reason is that most likely in the choice equation the effect of distance is non-linear so they rely on more instruments (any function of distance should do) but they rather estimate f. • They estimate the following equation for each hospital separately: maximum likelihood would do. T

13. Gautam Gowrisankaran and Robert Town (JHE, 1999) • For f<1 the function exp(-fd) looks as follows: distance For each variable cij they obtain an estimate of fj. They set f equal to the average value of 0.25

14. Gautam Gowrisankaran and Robert Town (JHE, 1999) • They estimate the mortality equation with a linear probability models. Linear probability models: • There is heteroskedasticity, can be solved via GLS. • Predicted values lie outside the [0,1] range, cannot be solved. • The coefficients have nice interpretations, i.e. bj is the incremental probability of death on any day at hospital j. • Technical:

15. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Then the variance of the error term is given by: • Then if the error term is hit=sit+eit the variance of the error term is var(hit)=hit(1-hit).

16. Gautam Gowrisankaran and Robert Town (JHE, 1999) Two step procedure: • IV regression (not controlling for heteroskedasticity)  consistent (but not efficient) estimates. • Construct for each observation based on the IV estimates from step 1 • Because the var-cov matrix is diagonal with elements pre-multiply the mortality equation by .Notice they use a slightly different formula to correct for (rare) cases where the estimated residuals are negative or greater than 1: apply IV again to the transformed variables. Estimates will be consistent and asymptotically efficient (it is not necessary to transform the instruments).

17. Gautam Gowrisankaran and Robert Town (JHE, 1999) • Endogeneity Test (Hausman,1978) on the c’s: Compares IV-GLS and GLS. H0: No endogeneity (IV-GLS≈GLS). Need a consistent and efficient estimator under the null and a consistent estimator under the alternative. • Overidentification test (Bowden and Turkington, 1984), the idea is that the Z and the h’s should not be correlated: J-1 number of endogeneous variables Full vector of estimates S2 is the estimated variance of the residuals in the second stage.

18. Gautam Gowrisankaran and Robert Town (JHE, 1999) Data Selection • They keep patients ≥65 years old that entered into a hospital in one of 4 of the California counties with a diagnosis of pneumonia (they specify carefully their selection based on the diagnosis) • They eliminate all patients that were transferred from nursing homes and other medical facilities since these facilities hold the most severe cases and are usually close to the highest quality hospital so that the assumption that sit is equally distributed in the population would be violated. • Remove small hospitals from the sample because the FE for these hospitals would be very imprecise. • Drop patients where no matching with income data is possible due to “unknown race” for example. • Final data has 178,972 observations

19. Gautam Gowrisankaran and Robert Town (JHE, 1999) Results: • Their main result is that the Hausman test rejects (at 1%) the null hypothesis of exogeneity. So c’s are endogenous, i.e. there is selection. • Moreover they find that the spread of IV coefficients of hospital quality is larger than the spread of the GLS coefficients. Because the severity is positively correlated with quality (i.e. negatively correlated with the beta’s since lower betas means high quality) this is evidence of this correlation. Why? For the low true betas (high quality hospitals) the non-iv estimate is higher because these hospitals receive more serious cases i.e. it underestimates the true quality, and the opposite for the low quality hospitals. Therefore the non-iv estimates of beta have less variance than the unbiased estimates.

20. Gautam Gowrisankaran and Robert Town (JHE, 1999) Estimated betas Unbiased estimates 45º line Take beta=0 as the benchmark. So beta below 0 is better than the benchmark hospital. The more negative the better (less mortality). - -GLS estimates Low quality GLS estimates High quality GLS estimates True betas Overestimates the quality here Underestimates the quality here

21. Gautam Gowrisankaran and Robert Town (JHE, 1999) Results (cont.): • They also find that the correlation between the GLS and IV was negligible which implies that the GLS completely misses the correct hospital ranking. • The standard deviations of the IV estimates are high which implies that the quality of most hospitals is not statistically significantly different from the quality of the average hospital. • What explains hospital quality? Why are some hospitals better than others: • The authors cannot include in their mortality regression hospital characteristics (in-variant with time) because these are colinear with hospital dummies. • What is possible is once we obtain FE estimates, to regress them against a bunch of hospital characteristics to see what characteristics better explain the differences in quality. {type of organization (public, not-for-profit, for-profit), teaching hospital dummy, dummy for operating long-term-facility, geriatric care unit, size (usually measured as number of beds) interacted with hospital type, average length of stay for medicare pneumonia patients and occupancy rates.

23. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • What are report cards? Report cards constitute a public disclosure of information on the performance of physicians or hospitals. • Interesting web-page (report cards for several states): http://www.consumerhealthratings.com/index.php?action=showSubCats&cat_id=301 • Example for NY 2004 (hospitals and surgeons) for coronary artery bypass graft (CABG):http://www.health.state.ny.us/statistics/diseases/cardiovascular/docs/pci_2005-2007.pdf • Net effects of report cards are unclear: • positive because they decrease asymmetries of information and may give incentives to increase quality • Possibly negative: give physicians and hospitals incentives to decline the most difficult cases in order to avoid worsening their mortality score.

24. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • What may be the problems with report cards: • Mortality rates must be adjusted for patients’ risk factors (“risk adjustment”) otherwise providers treating more serious cases would fare worse. • But analysts can only adjust for characteristics in databases, providers always have better information  capacity of selecting patients and case-mix unable of capturing it. • Even if risk adjustment was correct there may still be incentives to select patients, why? Suppose the matrix of mortality by hospital quality and patients’ risks is the following: Patients’s severity Risks Quality hospitals

25. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Mortality 10 Regression for the Low quality hospitals 5 Regression for the high quality hospital 2 1 Patients’ risk ={0,1} 0 1 If we could observe hospital quality we would estimate two different regressions. Key: The difference in outcomes (between High and Low quality hospitals) is higher for high risk patients! Low quality providers look more similar to High quality for less sick patients.

26. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • i.e. if we observe the quality of the hospital, we could run two regressions: • If we do not observe the quality of the hospital and we run a single regression

27. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Regression without selection (equal proportion of high and low risks in both hospitals); c=1.5; a=6 Mortality 10 Regression with selection (higher proportion of high risks at the best hospital) c=1.8; a=4.2 5 2 1 1 Patients’ risk ={0,1} With selection the constant (i.e. the mortality of the low risk patients, R=0) is identified by the low quality hospital and slope (the mortality of the high risk patients, R=1) by the high quality hospital. The residuals from both the low quality and the high quality hospital are now smaller and therefore the adjusted mortalities more similar among the two type of hospital.

28. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” But what does that imply? Suppose the high quality hospital has a higher proportion of high risks then a = 4.2, and the adjusted mortalities: The Low quality fares better when it has a lower proportion of high risk patients

29. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • Finally, even if risk adjustment was correct (on average) but noisy and if providers were risk averse then again the providers may shun away high risks patients if the variance of outcomes is higher for the high risks. Variance for the low risks EULow risk EUHigh risk Variance for the high risks Risk Adjusted Mortality mortality

30. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • Mandatory CABG (coronary artery bypass graft) report cards were introduced in the beginning of the 90’s in NY and Pennsylvania. • They use Medicare data from 1987- 1994 and estimate the short-run effects of the introduction of report cards in New York and Pennsylvania. • In particular they are interested in the effects of report cards on: • The matching of patients to providers – may improve if sicker patients have more to gain from being treated at higher quality hospitals and report cards gives correct information about high quality hospitals. • The incidence and quantity of CABG surgeries – The incidence of surgeries may have shifted from sicker to healthier patients and the quantity of surgeries may have gone up or down. If sicker patients have more to gain from the surgeries then social welfare may decrease. • The incidence and quantity of substitute treatments – substitute treatments may have increased for sicker patients, however if hospitals avoid patients altogether then we would observe a decrease in these as well. • Health care expenditures and patients health outcomes -

31. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • Methodology: they use a differences-in-differences approach where they compare the trends before and after the report cards for the NY and Penn states with the trends for control states (where there was no introduction of report cards). • Findings: • Report cards improve the matching of patients to hospitals + • CABG surgery increased and changed the incidence from sicker to healthier patients ± ? • Higher health costs – • Deterioration of health outcomes, specially amongst the most ill patients – • Conclusion: report cards were welfare-reducing

32. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • Identifying assumptions (in order to be able to conduct the differences-in-differences approach): • Assumption 1: The adoption of report cards must be exogenous i.e. uncorrelated with state level unobservables in trends in costs, health outcomes, and treatments. • Assumption 2: AMI patients (acute myocardial infarction) patients are a relevant at-risk population for CABG surgery and its composition does not change by the introduction of report cards (means that people do not have more or less hearth attacks because of the introduction of report cards). • In December 1990 NY released report cards on raw and risk-adjusted hospitals’ mortality on patients receiving CABG surgery • In 1992 NY released surgeon specific mortality • November 1992, Pennsylvania published hospital+surgeon report cards on risk adjusted CABG mortality. • NY and Penn are treatment states, the rest are control states

33. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” 1st: Show evidence that report cards did not alter the distribution of AMI patients (that was assumption 2). Note that SEVERITY is measured by the expenditures in the year previous to admission. Nationwide increase in intensity of treatment for people with heart problems, not different at treatment states, consistent with assumption 2.

34. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” 2nd: Trends in the people that undergo CABG surgery look very different Smaller expenditures of people undergoing CABG surgery in treatment states consistent with a shift towards healthier patients in these states.

35. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • Empirical Model • They study cohorts of AMI patients – all those that suffered at least one heart attack, they may or may not go through CABG. • They study cohorts of CABG patients - who may or may not have had an AMI. • The idea here is that (at least in the short run) the CABG report cards did NOT affect the AMI population (this is an emergency event) but in contrast may affect the composition of those receiving CABG because it is an elective procedure. • They estimate two models • Hospital is the unit of analysis • Patient as the unit of analysis

36. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” • Empirical Model – in the hospital-level analysis, the effects of report cards are measured by comparing the evolution of treatment states with the evolution of control states. • Matching – check whether report cards led to greater homogeneity (in health status) of patients within the hospital. • Incidence – check whether the health status of patients going through CABG changed due to report cards. If the health status has changed there may be two explanations: • 1. report cards had to do with it • 2. the underlying health of the population has changed – to dismiss this possibility they do the same exercise with AMI patients (which by assumption are not affected in the short run by report cards).

37. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” p is the difference-in-difference estimate coefficient of the effect of report cards on the severity of CABG patients. If p<0 then report cards caused a shift in incidence of the surgeries to an average healthier patient. Hospital Level analysis :

38. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” A negative p could be caused by an effect of report cards on a better health status of all patients (in risk of CABG surgery). To discard this explanation they examine the effects of report cards on AMI patients (there should be no effect since these patients are not subject to selection). Re-estimate the model using the mean severity of AMI patients as the dependent variable. Compare the estimated p’s of both models. Also, instead of using mean severity as dependent variable they use the coefficient of variation (s.d./ mean) before treatment within each hospital. Improved matching of patients would cause the coefficient of variation to decline in NY and Penn relative to other states, again a negative p. Finally, with improved sorting we should observe more ill patients to be at higher quality hospitals. Since quality is not observed they use teaching status as a quality proxy and re-estimate the model with an interaction term ZTEACHlst×Lst if the coefficient is positive it means report cards lead more sick patients to be treated at teaching hospitals.

39. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” 3rd: hospital level regressions Sample with selection Sample without selection Difficult to interpret because the increase in cv is also due to decrease mean severity Better matching Decline in illness severity => shift in incidence towards healthier patients

40. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” 3rd: hospital level regressions (cont.) they also find that the more serious cases go more to the teaching hospitals after the report cards. Constant severity at teaching hospitals Increase severity of patients at teaching hospitals: evidence of better matching

41. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Patient Level Analysis – similar regressions: To test for a quantity effect of report cards on CABG surgery: A positive p means that report cards increase the probability of an AMI patient having a CABG surgery. To test for the effect on substitute treatments (PCTA and cath) replace C by these treatments and re-estimate (2).

42. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Patient Level Analysis (cont.): To assess the effects of report cards on sick versus healthier patients they include an interaction term:

43. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Patient level regressions: 1) increase in CABG quantity; 2) that increase was based on healthier patients; 3) decrease in PCTA; 4) delay in treatment (1) (2) (3) The second part of table 4 has PTCA within one day of admission and cath

44. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Patient Level Analysis (cont.): For health outcomes and health expenditures follow similar procedure. If report cards increasethe incidence of adverse health outcomes and increasecosts  decrease welfare If report cards decrease the incidence of adverse health outcomes and decrease costs increase welfare Otherwise they can compute the cost-effectiveness of report cards

45. (1) (2) (1) Increase in costs because now the average patient is more likely to undergo CABG; Also more costs to the severely ill although these were less likely to undergo treatment, which means they became even sicker and therefore eventually demanded more expensive treatment. Other results show health status declined with RC, specially among the sicker (2).

46. Dranove et al. (JPE, 2003) – “Is more information better, the effects of report cards” Patient level regressions (cont): • increase in expenditures for all patients (surprisingly since the more severe were now less likely to receive CABG and less likely to receive PTCA now, the problem is that they will have more complications and that is costly too) • worse health outcomes one year after admission for sicker patients episodes of AMI and heart failure and slightly better outcomes for healthier patients (due to higher CABG procedures). Overall they conclude that the welfare was reduced. • They claim that report cards should focus on the population at risk, say the AMI population and not only report information on those that actually received the procedure • They claim that sorting i.e. the apparent better matching of patients to providers is positive • The short run analysis cannot say anything about the long run impact on providers’ quality