1 / 51

Basic Econometrics of Outcomes Research

Basic Econometrics of Outcomes Research. Jim Lightwood May 8 , 2012 UCSF Epi 211 Performance Measurement. Outline of Lecture. Cost Analysis Basics of cost analysis in US health care Dealing with Uncertainty: When and How Variance Estimation Concepts Sensitivity Analysis

lloyd
Download Presentation

Basic Econometrics of Outcomes Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Basic Econometrics of Outcomes Research Jim Lightwood May 8, 2012 UCSF Epi 211 Performance Measurement Econometrics of Outcome Measurement Epi 211

  2. Outline of Lecture • Cost Analysis • Basics of cost analysis in US health care • Dealing with Uncertainty: When and How • Variance Estimation Concepts • Sensitivity Analysis • Statistical Methods • Summary statistics • Regression Methods • Predictive Data Mining • Simulation models • Questions and Discussion (hopefully, last 20 minutes) Econometrics of Outcome Measurement Epi 211

  3. Important Issues • Description, versus prediction of change due to intervention • Describe situation with no intervention • Predict what will change if you do intervene • Point estimation versus interval estimation • Is point estimate of parameter (mean, median) enough? • Do you need interval estimates (confidence interval of mean or distribution, inter-quartile range)? Econometrics of Outcome Measurement Epi 211

  4. Cost (Price), Real Output, and Expenditure • Need to keep three distinct concepts in mind • Cost per unit (Price): value of an unit of a real good for service • the cost of treating one person for stroke) • Real Input: independently measurable unit real production that goes into providing a unit of service • measurable resources used as inputs for treatment following stroke, attributable to the stroke • Real Expenditure per unit= Cost per unit * Real Input per unit • Total Real Expenditure: Sum of Real Expenditure per unit over all units attributable to disease or risk factor Econometrics of Outcome Measurement Epi 211

  5. Total Costs • Total costs • Fixed Costs (FC) • Fixed over period of analysis • Overhead costs • Variable Costs (VC) • Flow costs of goods and services as function of flow of units produced • Total Costs (FC + VC) • Sum of FC and VC • Must not decrease as units produced increases Econometrics of Outcome Measurement Epi 211

  6. Marginal and Average Costs • Marginal Total Cost • Change in total cost of producing one more unit of service • Average Variable cost • (Variable Cost/Units of service produced) • Average Total Cost • (Total Cost/Units of service produced) Econometrics of Outcome Measurement Epi 211

  7. Market Economics Rationale for Using Marginal Cost = Price • In competitive market, over long run • Firm will produce until marginal cost = average total cost • Market price will be equal to marginal cost • Use market price to measure cost of unit of resource • How useful is this advice for US healthcare? Econometrics of Outcome Measurement Epi 211

  8. Curses of High Fixed Costs • Increase FC to 10,000 • Suppose your hospital is in catchment area with 200 cases per year • Can MC pricing work? • If there is joint production • (fixed costs are allocated between departments) • Allocation of fixed costs of, say hospital to services, is arbitrary • Focus on long run average total costs Econometrics of Outcome Measurement Epi 211

  9. Difference Between Charges and Costs • Charges are usually the only publically available costs data from providers • Costs of any kind are usually proprietary, and function of institution specific, arbitrary, algorithms • Charges are NOT usually the actual transaction prices • Analogous to ‘sticker price’ at car dealers, starting point for negotiation • What to do? Econometrics of Outcome Measurement Epi 211

  10. Deriving Costs from Charges • Usual approach is to derive average long run cost from charges • Critical role of cost-to-charge ratio • Usual algorithm for cost-to-charge ratio • Assume health care cost = health care revenue for institution • Count all funds that come in • Subtract incidental services, profits and retained earnings to get total revenues for patient care • Count observed outlays for health care inputs • Divide total revenue for patient care by observed outlays • May be available on institution, service, unit level • Cost-to-charge ratios for California hospitals recently around 50%; Maryland: 87% Econometrics of Outcome Measurement Epi 211

  11. Problems with estimated costs • Estimates of value of total expenditure using variable costs depend on institutional factors that may vary widely • Some variation due to differences in institution specific cost accounting • Estimates using cost derived from cost to charge ratio depend on institutional and regional health care market factors • Comparative studies show that there is significant heterogeneity in estimates of cost of diagnosis, or disease tx • Stroke is the only case I’ve seen where different estimates seem to be randomly distributed around a single overall point estimates Econometrics of Outcome Measurement Epi 211

  12. Practical Tips on Cost • Some surveys and data sets provide estimates of average cost • Medical Expenditure Panel Survey • California OSHPD (institutional level for hospitals through financial reports) • Need to read algorithms for cost estimates carefully • Derived from charges or from institutional cost accounting system? • Do analysis in terms of relative charges • But these may not be comparable when comparing institutions from different health care markets Econometrics of Outcome Measurement Epi 211

  13. Dealing with Uncertainty: When and How Econometrics of Outcome Measurement Epi 211

  14. Dealing with Uncertainty: When and How • For social decision making (country, world) • Focus on point estimate of the mean or median cost • Extreme economic cost-benefit analysis position is that ONLY point estimate of the mean is important • For planning for individual organization • Standard error, or inter-quartile range • For organizational risk management, financial planning over short to medium term, individual patient level • Standard deviation, or range Econometrics of Outcome Measurement Epi 211

  15. Point and Interval Estimates in a Regression Context • Point estimate of conditional mean • Regression line • Interval estimate for regression line • Confidence interval for regression line • Interval estimate for individual observations • ‘Prediction’ or ‘forecast’ interval for individual observations Econometrics of Outcome Measurement Epi 211

  16. Relevant Parts of Regression Output Econometrics of Outcome Measurement Epi 211

  17. More Usual Cost Distribution Econometrics of Outcome Measurement Epi 211

  18. Natural Log of More Usual Cost Distribution Econometrics of Outcome Measurement Epi 211

  19. Pros and Cons of Transformations • Pros • If you can demonstrate cost difference in transformed data with ‘nice’ distribution, there will be cost difference on original scale, but perhaps only with VERY large samples • Cons • Government agencies and CFOs do not pay bills in natural logarithm dollars, or square root dollars Econometrics of Outcome Measurement Epi 211

  20. Pros and Cons of Transformations (cont.) • A Very Important CON of Transformations Econometrics of Outcome Measurement Epi 211

  21. Pros and Cons of Transformations (cont.) • A Very Important PRO of Transformations Econometrics of Outcome Measurement Epi 211

  22. Pros and Cons of Transformations (cont.) • A Very Important PRO of Transformations Econometrics of Outcome Measurement Epi 211

  23. Sensitivity Analysis: What Variation is Relevant? Econometrics of Outcome Measurement Epi 211

  24. Sensitivity Analysis: What Variation is Relevant? Econometrics of Outcome Measurement Epi 211

  25. Usual Assumption on Marginal and Average Costs for Social Costs Econometrics of Outcome Measurement Epi 211

  26. Generalization of Treatment Groups: Breast Cancer Econometrics of Outcome Measurement Epi 211

  27. Generalization of Treatment Groups: Importance of estimating treatment effect, RCT, or natural experiment Econometrics of Outcome Measurement Epi 211

  28. Generalization of Treatment Groups: Importance of estimating treatment effect, observational data Econometrics of Outcome Measurement Epi 211

  29. Estimation with predetermined groupings • Continuous variables (cost, length of stay, etc.) • Use summary statistics • Mean, standard deviation, standard error • Median, inter-quartile range, standard error of median • Graphical methods (box plots, histograms, etc.) • Look for • Normal distribution • If not normal, try monotonic transformations to • Normal distribution • Symmetric distribution, if normality not achievable • For hypothesis tests • Normality or large sample: t-tests, trimmed t-tests • Normality not achievable: rank sum or median tests Econometrics of Outcome Measurement Epi 211

  30. Estimation with predetermined groupings • Continuous variables with censoring (survival, time to relapse, etc.) • Use Kaplan-Meier analysis • Estimate RR, OR, depending on study design • Look for • Assumptions on independent censoring, selection to treatment group, met • If assumptions not met, sensitivity analysis • For hypothesis tests • Logrank test Econometrics of Outcome Measurement Epi 211

  31. Estimation with predetermined groupings • Categorical variables with censoring (response, recovery, etc.) • Use 2x2 tables • Estimate relative risk, odd ratio (depending on study design) • Look for • Assumptions on selection to treatment met • If assumptions not met, do sensitivity analysis • Hypothesis tests: chi-square, or Fisher exact for independence Econometrics of Outcome Measurement Epi 211

  32. Regression methods • Estimation for • Large number of observations • Social cost or outcome analysis • Most important to estimate the conditional mean • Uncertainty, variance estimation less important • If results of a simple technique not sensitive to changes in distribution of outcomes that will occur after any intervention, • Then start with simple techniques: Ordinary Least Squares, Instrumental regression, etc. Econometrics of Outcome Measurement Epi 211

  33. Example 1 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211

  34. Example 1 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211

  35. Example 2 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211

  36. Example 2 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211

  37. Two-part estimation • Part one: Logistic (or Probit) regression to estimate probability of incurring positive cost • Part two: for those who have positive cost, regression estimate of log cost Econometrics of Outcome Measurement Epi 211

  38. How to deal with this con of transformations • A Very Important Con of Transformations Econometrics of Outcome Measurement Epi 211

  39. Lognormal Cost Data Econometrics of Outcome Measurement Epi 211

  40. Lognormal Cost Data Econometrics of Outcome Measurement Epi 211

  41. Linear Regression on Raw Cost Data Econometrics of Outcome Measurement Epi 211

  42. Linear Regression on Log of Cost Data Econometrics of Outcome Measurement Epi 211

  43. What happens when we try to predict mean cost using ln(cost) regression • Calculate predicted mean of cost from regression of log cost • = 3.155 + 0.09241*mean(age) • = 7.28 • Predicted cost = exp(7.28) = 1608.02? Econometrics of Outcome Measurement Epi 211

  44. Smearing Estimator • Take the residuals from the estimated regression • Take the exponential of the residuals • The smearing factor is the mean of the exponential of the residuals Econometrics of Outcome Measurement Epi 211

  45. What happens when we try to predict mean cost using ln(cost) regression • Calculate predicted mean of cost from regression of log cost • = 3.155 + 0.09241*mean(age) • = 7.28 • Predicted cost = exp(7.28) = 1608.02 • SF*predicted cost =1.66*1608.02 = 2669.05? Econometrics of Outcome Measurement Epi 211

  46. What happens when we try to predict mean cost using ln(cost) regression • Trial solution • Stratify by age where you seen change in distribution of cost • Note that variance of cost increases at around age 55 • Calculate ln(cost) and SFstratified by age at 55 • Result is much closer approximation • Mean observed cost <= 55: 2187.04 > 55: 9383.63 Econometrics of Outcome Measurement Epi 211

  47. Exploratory Analysis: Classification and Regression Trees Econometrics of Outcome Measurement Epi 211

  48. Regression Results on LnCost Econometrics of Outcome Measurement Epi 211

  49. Simulation Estimates Econometrics of Outcome Measurement Epi 211

  50. Simulation Estimates Econometrics of Outcome Measurement Epi 211

More Related