510 likes | 734 Views
Basic Econometrics of Outcomes Research. Jim Lightwood May 8 , 2012 UCSF Epi 211 Performance Measurement. Outline of Lecture. Cost Analysis Basics of cost analysis in US health care Dealing with Uncertainty: When and How Variance Estimation Concepts Sensitivity Analysis
E N D
Basic Econometrics of Outcomes Research Jim Lightwood May 8, 2012 UCSF Epi 211 Performance Measurement Econometrics of Outcome Measurement Epi 211
Outline of Lecture • Cost Analysis • Basics of cost analysis in US health care • Dealing with Uncertainty: When and How • Variance Estimation Concepts • Sensitivity Analysis • Statistical Methods • Summary statistics • Regression Methods • Predictive Data Mining • Simulation models • Questions and Discussion (hopefully, last 20 minutes) Econometrics of Outcome Measurement Epi 211
Important Issues • Description, versus prediction of change due to intervention • Describe situation with no intervention • Predict what will change if you do intervene • Point estimation versus interval estimation • Is point estimate of parameter (mean, median) enough? • Do you need interval estimates (confidence interval of mean or distribution, inter-quartile range)? Econometrics of Outcome Measurement Epi 211
Cost (Price), Real Output, and Expenditure • Need to keep three distinct concepts in mind • Cost per unit (Price): value of an unit of a real good for service • the cost of treating one person for stroke) • Real Input: independently measurable unit real production that goes into providing a unit of service • measurable resources used as inputs for treatment following stroke, attributable to the stroke • Real Expenditure per unit= Cost per unit * Real Input per unit • Total Real Expenditure: Sum of Real Expenditure per unit over all units attributable to disease or risk factor Econometrics of Outcome Measurement Epi 211
Total Costs • Total costs • Fixed Costs (FC) • Fixed over period of analysis • Overhead costs • Variable Costs (VC) • Flow costs of goods and services as function of flow of units produced • Total Costs (FC + VC) • Sum of FC and VC • Must not decrease as units produced increases Econometrics of Outcome Measurement Epi 211
Marginal and Average Costs • Marginal Total Cost • Change in total cost of producing one more unit of service • Average Variable cost • (Variable Cost/Units of service produced) • Average Total Cost • (Total Cost/Units of service produced) Econometrics of Outcome Measurement Epi 211
Market Economics Rationale for Using Marginal Cost = Price • In competitive market, over long run • Firm will produce until marginal cost = average total cost • Market price will be equal to marginal cost • Use market price to measure cost of unit of resource • How useful is this advice for US healthcare? Econometrics of Outcome Measurement Epi 211
Curses of High Fixed Costs • Increase FC to 10,000 • Suppose your hospital is in catchment area with 200 cases per year • Can MC pricing work? • If there is joint production • (fixed costs are allocated between departments) • Allocation of fixed costs of, say hospital to services, is arbitrary • Focus on long run average total costs Econometrics of Outcome Measurement Epi 211
Difference Between Charges and Costs • Charges are usually the only publically available costs data from providers • Costs of any kind are usually proprietary, and function of institution specific, arbitrary, algorithms • Charges are NOT usually the actual transaction prices • Analogous to ‘sticker price’ at car dealers, starting point for negotiation • What to do? Econometrics of Outcome Measurement Epi 211
Deriving Costs from Charges • Usual approach is to derive average long run cost from charges • Critical role of cost-to-charge ratio • Usual algorithm for cost-to-charge ratio • Assume health care cost = health care revenue for institution • Count all funds that come in • Subtract incidental services, profits and retained earnings to get total revenues for patient care • Count observed outlays for health care inputs • Divide total revenue for patient care by observed outlays • May be available on institution, service, unit level • Cost-to-charge ratios for California hospitals recently around 50%; Maryland: 87% Econometrics of Outcome Measurement Epi 211
Problems with estimated costs • Estimates of value of total expenditure using variable costs depend on institutional factors that may vary widely • Some variation due to differences in institution specific cost accounting • Estimates using cost derived from cost to charge ratio depend on institutional and regional health care market factors • Comparative studies show that there is significant heterogeneity in estimates of cost of diagnosis, or disease tx • Stroke is the only case I’ve seen where different estimates seem to be randomly distributed around a single overall point estimates Econometrics of Outcome Measurement Epi 211
Practical Tips on Cost • Some surveys and data sets provide estimates of average cost • Medical Expenditure Panel Survey • California OSHPD (institutional level for hospitals through financial reports) • Need to read algorithms for cost estimates carefully • Derived from charges or from institutional cost accounting system? • Do analysis in terms of relative charges • But these may not be comparable when comparing institutions from different health care markets Econometrics of Outcome Measurement Epi 211
Dealing with Uncertainty: When and How Econometrics of Outcome Measurement Epi 211
Dealing with Uncertainty: When and How • For social decision making (country, world) • Focus on point estimate of the mean or median cost • Extreme economic cost-benefit analysis position is that ONLY point estimate of the mean is important • For planning for individual organization • Standard error, or inter-quartile range • For organizational risk management, financial planning over short to medium term, individual patient level • Standard deviation, or range Econometrics of Outcome Measurement Epi 211
Point and Interval Estimates in a Regression Context • Point estimate of conditional mean • Regression line • Interval estimate for regression line • Confidence interval for regression line • Interval estimate for individual observations • ‘Prediction’ or ‘forecast’ interval for individual observations Econometrics of Outcome Measurement Epi 211
Relevant Parts of Regression Output Econometrics of Outcome Measurement Epi 211
More Usual Cost Distribution Econometrics of Outcome Measurement Epi 211
Natural Log of More Usual Cost Distribution Econometrics of Outcome Measurement Epi 211
Pros and Cons of Transformations • Pros • If you can demonstrate cost difference in transformed data with ‘nice’ distribution, there will be cost difference on original scale, but perhaps only with VERY large samples • Cons • Government agencies and CFOs do not pay bills in natural logarithm dollars, or square root dollars Econometrics of Outcome Measurement Epi 211
Pros and Cons of Transformations (cont.) • A Very Important CON of Transformations Econometrics of Outcome Measurement Epi 211
Pros and Cons of Transformations (cont.) • A Very Important PRO of Transformations Econometrics of Outcome Measurement Epi 211
Pros and Cons of Transformations (cont.) • A Very Important PRO of Transformations Econometrics of Outcome Measurement Epi 211
Sensitivity Analysis: What Variation is Relevant? Econometrics of Outcome Measurement Epi 211
Sensitivity Analysis: What Variation is Relevant? Econometrics of Outcome Measurement Epi 211
Usual Assumption on Marginal and Average Costs for Social Costs Econometrics of Outcome Measurement Epi 211
Generalization of Treatment Groups: Breast Cancer Econometrics of Outcome Measurement Epi 211
Generalization of Treatment Groups: Importance of estimating treatment effect, RCT, or natural experiment Econometrics of Outcome Measurement Epi 211
Generalization of Treatment Groups: Importance of estimating treatment effect, observational data Econometrics of Outcome Measurement Epi 211
Estimation with predetermined groupings • Continuous variables (cost, length of stay, etc.) • Use summary statistics • Mean, standard deviation, standard error • Median, inter-quartile range, standard error of median • Graphical methods (box plots, histograms, etc.) • Look for • Normal distribution • If not normal, try monotonic transformations to • Normal distribution • Symmetric distribution, if normality not achievable • For hypothesis tests • Normality or large sample: t-tests, trimmed t-tests • Normality not achievable: rank sum or median tests Econometrics of Outcome Measurement Epi 211
Estimation with predetermined groupings • Continuous variables with censoring (survival, time to relapse, etc.) • Use Kaplan-Meier analysis • Estimate RR, OR, depending on study design • Look for • Assumptions on independent censoring, selection to treatment group, met • If assumptions not met, sensitivity analysis • For hypothesis tests • Logrank test Econometrics of Outcome Measurement Epi 211
Estimation with predetermined groupings • Categorical variables with censoring (response, recovery, etc.) • Use 2x2 tables • Estimate relative risk, odd ratio (depending on study design) • Look for • Assumptions on selection to treatment met • If assumptions not met, do sensitivity analysis • Hypothesis tests: chi-square, or Fisher exact for independence Econometrics of Outcome Measurement Epi 211
Regression methods • Estimation for • Large number of observations • Social cost or outcome analysis • Most important to estimate the conditional mean • Uncertainty, variance estimation less important • If results of a simple technique not sensitive to changes in distribution of outcomes that will occur after any intervention, • Then start with simple techniques: Ordinary Least Squares, Instrumental regression, etc. Econometrics of Outcome Measurement Epi 211
Example 1 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211
Example 1 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211
Example 2 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211
Example 2 for Simple Regression Analysis Econometrics of Outcome Measurement Epi 211
Two-part estimation • Part one: Logistic (or Probit) regression to estimate probability of incurring positive cost • Part two: for those who have positive cost, regression estimate of log cost Econometrics of Outcome Measurement Epi 211
How to deal with this con of transformations • A Very Important Con of Transformations Econometrics of Outcome Measurement Epi 211
Lognormal Cost Data Econometrics of Outcome Measurement Epi 211
Lognormal Cost Data Econometrics of Outcome Measurement Epi 211
Linear Regression on Raw Cost Data Econometrics of Outcome Measurement Epi 211
Linear Regression on Log of Cost Data Econometrics of Outcome Measurement Epi 211
What happens when we try to predict mean cost using ln(cost) regression • Calculate predicted mean of cost from regression of log cost • = 3.155 + 0.09241*mean(age) • = 7.28 • Predicted cost = exp(7.28) = 1608.02? Econometrics of Outcome Measurement Epi 211
Smearing Estimator • Take the residuals from the estimated regression • Take the exponential of the residuals • The smearing factor is the mean of the exponential of the residuals Econometrics of Outcome Measurement Epi 211
What happens when we try to predict mean cost using ln(cost) regression • Calculate predicted mean of cost from regression of log cost • = 3.155 + 0.09241*mean(age) • = 7.28 • Predicted cost = exp(7.28) = 1608.02 • SF*predicted cost =1.66*1608.02 = 2669.05? Econometrics of Outcome Measurement Epi 211
What happens when we try to predict mean cost using ln(cost) regression • Trial solution • Stratify by age where you seen change in distribution of cost • Note that variance of cost increases at around age 55 • Calculate ln(cost) and SFstratified by age at 55 • Result is much closer approximation • Mean observed cost <= 55: 2187.04 > 55: 9383.63 Econometrics of Outcome Measurement Epi 211
Exploratory Analysis: Classification and Regression Trees Econometrics of Outcome Measurement Epi 211
Regression Results on LnCost Econometrics of Outcome Measurement Epi 211
Simulation Estimates Econometrics of Outcome Measurement Epi 211
Simulation Estimates Econometrics of Outcome Measurement Epi 211