Jeffrey S. Napierala Prof. Glenn D. Deane Department of Sociology, SUNY Albany

Demystifying the Regression Coefficient: Rethinking a Complex Tool for Use in Policy Research Jeffrey S. Napierala Prof. Glenn D. Deane Department of Sociology, SUNY Albany Prof. Donald J. Hernandez Department of Sociology, Hunter College & CUNY Graduate Center Research was supported by a Grant from the Annie E. Casey Foundation Please do not cite or distribute this research without consent from the Authors

Outline • The origins of this method • An example using children's reading performance • A “hybrid” approach • Standard Errors • Results • Conclusions

Origins • The audience for policy research often has a diverse background in statistics. • To accommodate those without any proficiency, the presentation must be simple and transparent. • To satisfy those with advanced proficiency, current methods must be used. • We want to have our cake and eat it too…

Origins • At one end of the spectrum: Sample means/proportions are simple and easy to understand …but may not clearly translate into effective policies… • Towards the other end: A standard regression approach has obvious advantages, but results can be difficult to explain to general audiences.

Origins Hernandez, Donald J. Double Jeopardy: How Third-Grade Reading Skills and Poverty Influence High School Graduation. Baltimore: Annie E. Casey Foundation

An Example… • Policy Question: How much would income transfers increase the percentage of (3rd Grade) students reading at the proficient level? • Data: National Longitudinal Survey of Youth (NLSY) • 4,060 children in 2369 families followed across about 30 years • Dependent variable: Peabody Individual Achievement (PIAT) Reading Comprehension Test. Continuous variable ranging from 1-99. • Methods: Weighted, linear GEE models with a correction for clustering within families.

An Example: Model Specification • Two models to highlight approach: • 1. A “Base” model with just income and income squared predicting PIAT Reading Comprehension. • 2. A “Full” model with controls for the 1) non-linear effect of mother’s education, 2) health insurance coverage, 3) whether a child attended head start or preschool, 4) the “quality” of their neighborhood, 5) race, 6) sex, and 7) year of interview.

A “hybrid” Approach • Let’s utilize the power of a regression, but keep the presentation as simple and flexible as possible. • Create a statistical model of the outcome. • “Simulate” new outcomes using the relevant parameters of the model. • Meaningfully summarize the distributions before and after the “simulation” using simple statistics/tabulations. • Compare the summary statistics/tabulations.

A “hybrid” Approach: Standard Errors • Using the common formula for the standard error of a proportion (with and without a Design Effect multiplier of 1.388). • A Monte-Carlo approach to incorporate error from the sample regression and sample proportion. • First, the effect(s) of covariates are added into the original score (the raw DV) by sampling from a normal distribution with a mean and s.d. from the regression. • Rates (of reading proficiency) are computed then additional sampling error is introduced. • After 5000 iterations, the S.E. of the distribution is computed from all the rates.

A “hybrid” Approach: Standard Errors • Also, to compare rates from the same group of children (before and after “simulations”) a “paired proportions” t-test is used (Altman 1997). Source: Altman, Douglas G. 1997. Practical Statistics for Medical Research. London: Chapman & Hall.

Results…

Conclusions • The “hybrid” approach has a few notable advantages over other methods… • The independent effect of income on reading proficiency is much (much) less than might be expected from looking at bivariate or univariate results. • We expect that about 4% (2.5-5.9; 95% C.I.) more kids in poverty would read proficiently if their families were given additional income to move them out of povery.

Thank You! • Email: jnapierala@albany.edu

Jeffrey S. Napierala Prof. Glenn D. Deane Department of Sociology, SUNY Albany