Multiple Imputation Using ICE: Simulation Study on Binary Response

1. Multiple imputation using ICE: A simulation study on a binary response Jochen Hardt Kai Görgen 6th German Stata Meeting, Berlin June, 27th 2008 Göteborg University University of Mainz, Bernsteincenter for Computational Neuroscience, Berlin

2. Almost all sociological / medical data have missings - typically in the range of .5 to 5 % in a variable Many statistical procedures can only use cases without missings • What we already know about missing substitution: • With a small amount of missings everything is easy • Large samples are easy

3. Overview • Missingness at random • A very simple example • - Analysis of complete cases • - Imputation of means • - Singular regression imputation • - Multiple imputation: hotdeck • - Multiple imputation: chained equations • A not so simple example • Multiple imputation by chained equations in real data

4. Background I There is a distinction in the literature about data being missing completely at random (MCAR), missing at random (MAR) or being missing not at random (Rubin, 1996). MCAR means that the pattern of missings is totally at random, not depending on any variable in or not in the analysis. MAR is an intuitively somewhat misleading label, because it allows strong dependencies in the pattern of missings. If, for example, in a set of variables all data for men are missing and for women are non-missing, the dataset is still MAR as long as gender is included as a variable. The formal definition is that missings are at random given all information available in the dataset.

5. Background II MCAR usually does not apply to data in social sciences MAR seems quite plausible for many datasets. But the definition has the disadvantage that it can never be tested on any given dataset – always it is possible that some unobserved variables - at least partitially - cause the pattern of missing. MNAR means that there is such an unknown process in the data that creates the missings. E.g. for socially undesirable behaviour, such as lying, stealing or betraying, it is plausible to assume that missing values rather reflect higher than lower levels of such behaviour, but an exact modelling of the answering process is mostly not possible. One of the most prominent question for MNAR is the one about income, which has high rates of missings, usually in the range of 20 % - 50 %.

6. A very simple example: reg Y X, both standard distributed continuous variables Y = 1*X + 1*error, n = 50, i = 3%, 8%, 13%…. 68% of X are set missing, for each I: 200 replications were made y x

7. The old solution: take only the cases without missings. Estimate for beta ± sd Standard deviation for beta Works ok but waste of information, particularly in multivariate analyses Percent missings in x

8. The 2nd solution: mean substitution Quite stable estimate, stronger increase in sd than in complete case analysis

9. The 3rd solution: substitution by regression Overestimation of the effect when response is included

10. Hotdeck Imputation s Typo: 1 of course

11. Number 4: Multiple imputation - hotdeck Considerably more variance due to imputation, break-down at about 50 % missings (m = 5, 4 variables)

12. Multiple Imputation by Chained Equations: ICE s

13. Multiple Imputation • a random subset of the data is drawn • A value for each missing of var X1 is estimated via (linear, logistic, ordered, etc) regression • The closest observed values to that estimate are chosen and replace the missings • The program switches to X2 • …….. • Cycled over ten times • Finish when m datasets are created

14. Multiple Imputation: Analysis • in each dataset a (regression) analysis is performed • Results are combined due to Rubins rule(a) parameters (b) variances within between total

15. No 5 finally: Multiple Imputation on Chained Equations - ICE Stable estimates with small variances (m = 5, 4 variables)

16. Analysis of complete cases: not bad when only few variables • Imputation of means not bad for continous variables don‘t impute the mode take the mean for categorical variables, too no inflation of ß‘s when no replacement in response • Regression imputation don‘t include response into model • Multiple imputation: Hotdeck Stata‘s version is not recomendable • Multiple Imputation by Chained Equations very good • Let‘s have a look onto a not so simple example Preliminary summary on the very simple example

18. One binary response (Suicide attempts) is predicted by 20 continous variables plus 5 discrete Variables: Var ß sd X1: maternal love .74 .19 Response: Lifetime suicide attempt 0 = no (83 %) 1 = yes (17 %) N = 505

19. Percent missing in x ICE estimate for beta, 4 variables in the model, CMAR

20. Percent missing in x ICE estimate for beta, 4 variables in the model , CMAR

21. Percent missing in x ICE estimate for beta, 4 variables in the model , CMAR

22. Percent missing in x ICE estimate for beta, 11 variables in the model , CMAR

23. Percent missing in x ICE estimate for beta, 25 variables in the model , CMAR

24. Percent missing in x The same done with MICE in R estimate for beta, 11 variables in the model , CMAR

25. Percent missing in x single regression substitution estimate for beta , CMAR 10 variables in the model (response excluded)

26. Percent missing in x mean substitution imputation estimate for beta , CMAR

27. Percent missing in x ICE estimate for beta, 11 variables in the model, NMAR

28. Percent missing in x Single regression imputation 10 variables in the model,NMAR

29. Percent missing in x All non-linear effects are downward biased by any method. The example shows an interaction coefficient estimated with ICE, 11 variables in the model, CMAR

30. Summary-In large samples we can substitute considerable higher proportions of missings than in small ones.- Multiple imputation with ICE performs well in all situations (as far as we examined) • Having more variables in the imputation model leads to better estimates, i.e.smaller sd’s. • With binary responses, ICE may report extreme sd’s when the number of variables grows high, or the number of cases low. Then we have gone too far. • Single regression imputation performs quite well under certain conditions • Non-linear effects get lost with all methods