On The Use of Instrumental Variables And Symmetric-prediction for Estimating Impacts Of Mediators

1. On The Use of Instrumental Variables And Symmetric-prediction for Estimating Impacts Of Mediators Laura Peck & Shawn Moulton Welfare Research & Evaluation Conference Washington, DC | May 30, 2013

2. Outline • Background: Motivation, Challenges & Responses • Classes of Questions, with example • Comparison of IV & ASPES • Estimation procedure • Research questions • Assumptions • Interpretation • Applications

3. Background • Policy questions concerning endogenous factors • Simply = take-up, participation (Class 1 questions) • Complexly = dosage, quality, components/features, mediators (Class 2 questions) • Analytic challenges • Commonly used methods • Multivariate Regression, Structural Equation Modeling, Propensity Score methods, Principal Stratification • Focus = Instrumental Variables (IV) estimation, Analysis of Symmetrically-predicted Endogenous Subgroups (ASPES)

4. Illustrative Example: SHM • Class 1 questions: What is the impact of an intervention on those who actually take-up the treatment? (TOT) • What is the impact of SHM on couples’ average report of relationship happiness for participants who attended any marriage education group sessions? • Class 2 questions: What is the impact of a mediating variable? • How does the impact of SHM on couples’ relationship happiness vary by variation in the number of hours of participation the program? • How does the impact of SHM vary by type and number of workshops/sessions attended?

5. IV & ASPES Estimation Procedure • Two stage procedures • Instrumental variables: Stage 1: Mi= γ0 + γ1Ti+ ηi Stage 2: Yi= β0 + β1i + εi • Analysis of symmetrically-predicted subgroups: Stage 1(prediction subsample):Mi = β0 + Xi θ + ε1i Stage 2 (analysis subsample): Yi= α + βi + δTi + γTii + ε2i

6. IV & ASPES Research Questions • Instrumental variables: • Class 1 questions: influence of some post-randomization experience (participation) on outcomes • ITT  TOT = correcting for no-shows, cross-overs • Analysis of symmetrically-predicted subgroups: • Class 2 questions: influence of some post-randomization experience (mediator) on impacts • Those things that baseline traits can productively predict, including: dosage, quality, programmatic mediators, early outcomes/personal/program milestones

7. Summary Comparison

8. IV & ASPES Assumptions • Both IV & ASPES: • SUTVA • Randomization • Instrumental variables: • Exclusion Restriction (instrument is sole pathway) • Strong Instrument (noncontroversial with experiment) • Analysis of symmetrically-predicted subgroups: • Homogeneity (impact is same, on average, for predicted and actual) [see next slide]

9. ASPES Homogeneity Assumption

10. ASPES Homogeneity Assumption

11. Summary Comparison (cont.)

12. Interpretation & Applications • Instrumental variables: • Causal Local Average Treatment Effect, by assumption • ITT  TOT conversions, when exclusion restriction holds • Analysis of symmetrically-predicted subgroups: • Causal Treatment Effect of • Causal Treatment Effect of M, by assumption • Causal analysis of predicted mediators (risk)  analysis of actual mediators

13. Summary Comparison (cont.)

14. Contact: Laura Peck Principal Scientist Laura_Peck@abtassoc.com (301) 347-5537