Modeling to Address Health Policy Questions

1. What, Why and How:Modeling to Address Health Policy Questions Deborah Chollet Senior Fellow, Mathematica Policy Research The Robert Wood Johnson Foundation’s State Coverage Initiatives Program Washington, DC November 10, 2004

2. What is a Model? • A structured way to think about a problem • A way to measure responses and outcomes • A way to compare alternative options

3. Why Model Policy? • Understand the impacts of policy change and sensitivity to program options • Program enrollment and cost • Uninsured population • Understand the sensitivity of estimates to environmental factors • Private insurance premium growth • Changes in employment • Demographic change

4. Types of Models

5. Determinate Models • Estimate aggregate response to a program change • Support sensitivity analysis only by subgroups identified in the model • “Broad brush” approach, not suited to “fine-grained” analysis of complex systems and interactions • Offer a measure of precision of estimates, or a confidence interval

6. Spreadsheet Model • Mimics the operation of a program or system • Incorporates average behavior by population subgroup • Relies on assumptions borrowed from populations that may differ in unmeasured ways • Outputs subgroup estimates only as defined in the model • Relatively inexpensive and fast to assemble

7. Example: Simple Spreadsheet Model of the Dirigo Program Calculate subgroup offer and take up from available evidence Number of persons by subgroup Calculate financing specific to subgroups: • State funds • Federal match • Employer contribution • Individual premiums Sum across subgroups Sensitivity analyses

8. Microsimulation Model • Mimics the operation of a program or system • Operates on a large database (e.g., CPS) and outputs the same database with variables of interest calculated • Incorporates partial responses to many variables at the level of the individual to calculate final response • Supports relatively flexible analysis of subgroups and sensitivity to assumptions • Often reveals results of complex logical relationships you might otherwise have overlooked

9. Example: Simple MicrosimulationModel of the Dirigo Program Population output data file Constrain take up by eligibility rules Population input data file Subgroup and sensitivity analyses; refine actuarial estimates as needed Calculate stochastic estimate of take up Create insurance families Develop initial actuarial cost estimates Apply parameters/relationship estimates to calculate individual take up probability

10. Choosing the Right Model • How precisely can you identify what you need to know? • How much detail do you need to know? • How much time and budget do you have? • What data are available to inform the model?

11. Designing the Model • What are the key policy questions? How many people will enroll? What woodwork effects? Change in program cost? • What measures address the policy questions? Insurance family membership and income, actuarial factors (family type, family size, age, gender, location) • What subgroups or other responses are of special interest? What program features might be changed? Eligibility groups (parents, childless adults by income), impacts on linked programs

12. Structural Issues • What program features and system relationships are important? • Eligibility rules, outreach activities, income disregards • Funding caps or links to available funds • What environmental variables should the model include? What sensitivity analyses do you want? • Aggregate cost growth • Changes in the industry, firm size, or wage structure of employment

13. Data Issues • Do available data include the population of interest to you? • Are adequate estimates of behavioral response already available from the literature? • Can available data be “enhanced” to improve sample size and precision?

14. Enhancing State Data for Modeling • Merge population samples (e.g., 3 CPS years) • Sample overlap • Data are observations of real residents, but trends may be lost • “Balance” a national or regional sample • National/regional data are “raked” (re-weighted) to state totals • Data are synthetic, but reflect the most recent time period on key (control) variables

15. When Do You Need an Actuary? • You do not need an actuary to estimate enrollment • You do need an actuary to estimate per capita cost when: • Enrollee demographics are likely to change from past experience • The benefit design is new • The benefit design must be targeted to a cost cap

16. What Does an Actuary Need? • Estimated eligibles and enrollment, by • Individual age and gender • Family type and size • Geographic location • Benefit design option

17. Key Lessons for Modeling Policy • Be clear and selective about what the model absolutely has to do • Be pragmatic about structure • If you choose microsimulation: • Modules are easier to build, debug, and update • Full integration can be costly and unnecessary • When possible, estimate model performance and benchmark

18. More Lessons • Build into the model what you need out of it • Variables for subgroup analysis • Parameters for sensitivity analyses • Be aware of tautologies and power • Are the results showing you only the input assumptions? • Do just a few observations drive the result?