On the Clock: Event History Modeling in the Study of Leadership Tenure Ryan Kennedy

1. On the Clock: Event History Modeling in the Study of Leadership Tenure Ryan Kennedy The Ohio State University, Department of Political Science kennedy.310@polisci.osu.edu http://polisci.osu.edu/grads/kennedy How do we test whether results are robust for unobserved heterogeneity across different countries and regimes in event history? Example: “Selectorate Theory” Abstract • Developed by Bueno de Mesquita, Morrow, Siverson and Smith (1999; 2002; 2003 – hereafter BMSS). • Posits two variables that influence leadership tenure: • Selectorate size – the portion of the population from which a winning coalition can be drawn. Measured by whether the country has an elected legislature (2), an appointed legislature (1) or no legislature (0). Larger selectorate size is hypothesized to make leadership tenure more stable. • Winning coalition size – the portion of the selectorate whose support is necessary to hold office. The final score ranges from 0 to 4, with 4 indicating the largest necessary winning coalition size. Larger winning coalitions are hypothesized to produce more unstable leaders. While the study of leadership survival across countries and political systems has become a central part of comparative politics, international political economy, and international relations, surprisingly little discussion is dedicated to the challenges in empirically modeling this phenomenon. Models of leadership tenure have unobserved heterogeneity on two levels: (1) the leader's country and (2) the nominal regime category in which the leader serves. Unfortunately, the tools drawn from biostatistics, while appropriate for a large range of data and very efficient, also carry some assumptions that will not always hold in social science data. This paper suggests the use of several other widely available tools, which are not common practice in event history, to test these assumptions. Scholars in the social sciences can use these tools to verify their results or raise important questions about their variables and measurement. The potential shortcomings of current practice and the utility of these robustness checks are demonstrated through a re-analysis of “selectorate theory” as an explanation of leadership tenure. Table 1: Previous Studies of Leadership Survival Table 2: Replication of BMSS Results. Cox proportional hazard model most often used in leadership tenure studies. Replication (w/o controls) consistent with BMSS’ (1999) original findings. Country-Level Heterogeneity Assumes consistent coefficient across regime types. • Introducing controls into the equation produces mixed results. • Chart 1 demonstrates that there is substantial reason to suspect country-level heterogeneity. • Table 3 demonstrates one of the weaknesses of just using shared frailty. In model 2, the shared frailty term does not affect the substantive interpretation and actually makes the coefficient of selectorate size slightly stronger. • The other models provide reason for doubting the conclusions of the shared frailty model. In every case where robust clustering is used, the standard errors for selectorate size increase substantially (and the p-values fall well outside of commonly accepted levels). • In the models which operationalize fragility on the country level, selectorate size loses strength on its coefficient. This is not the case when stratification is utilized. Unfortunately, clear guidelines on testing for coefficient bias are not available in the event history context. • All of this suggests that something else may underlie the relationship between selectorate size and leadership tenure. • Two variables are closely associated with the absence of an elected legislature: military regimes and times of extreme politics (see Geddes, 1999; Geddes, 2002; Stepan, 1988; Marshall and Jaggers, 2002). • When these variables are controlled for in Table 4, selectorate size changes direction and loses statistical significance. These relationships are robust to clustered standard errors and fixed effects. Note: Coefficients are reported with p-values in parentheses (1-tailed). Shared Frailty Most efficient, but does not tell us if country-level effects may be driving results. • Unobserved heterogeneity caused by the country in which a leader serves is usually modeled as a random variable on the shared characteristic (Box-Steffensmeier and Jones, 2004; Hougaard, 2000). The standard Cox equation is modified to • hij (t)=h0(t)exp(β’xij+ψ’wj) • where wj are the subgroup frailties, assumed to be an independent sample from a distribution with mean 0 and variance 1. • Researchers need to understand that there are several limitations to this strategy: • A test can be done on the random covariate to determine the presence of unobserved heterogeneity, but this does not indicate the source of that heterogeneity – the significance of individual groupings is not reported. • Shared frailty models assume that the random covariate is uncorrelated with the other covariates. Unobserved heterogeneity that might impact the covariates is not captured. Since country-level heterogeneity is almost always an alternative hypothesis in comparative politics, this assumption should be explicitly tested (Hougaard, 2000, p. 116). No control for unobserved heterogeneity!!! Table 3: Effects of Selectorate Variables Accounting for Unit Heterogeneity. Shared frailty generally consistent with replication. Stratification Regime Type Heterogeneity • Stratification has been the primary method for modeling differences across regime types (see Table 1). This involves absorbing the institutional differences into the baseline hazards by modifying the Cox model to • hij(t)=h0j(t)exp(B’xij) • where h0j(t) is the baseline hazard of a leader in regime type j losing office. • Like the shared frailty model, stratification carries a couple of assumptions that should be tested: • While stratification allows the baseline hazard to vary, it assumes that B’ is the same on both strata. • The effect of regime type is often of substantive interest, but that effect is not estimated in stratification studies. More explicit methods for modeling unobserved heterogeneity show different results! Why? Look a little more closely at the data. • Similarly, BMSS (1999) assume that the only regime types that matter are captured in their independent variables. As can be seen in Table 5, this is not the case. • Elections are associated minimal costs for leadership removal and term limits. Chart 2 shows a much more rapid increase in the baseline hazard for leaders in electoral regimes. • When included as a covariate, the presence of elections to remove leaders makes them over 200% more likely to be removed from office in a particular year. The selectorate theory variables also lose their effect. • When stratifying by election, the selectorate theory variables lose statistical significance, but this does not show the independent effect of elections. It also assumes that the coefficients are consistent across regime types. • Models 11 and 12 suggest that the assumption of consistent coefficients is not reasonable. Both covariates have very different directions and standard errors under these two regime types. Table 4: Effect of Selectorate Variables Accounting for Military Regimes and Extreme Politics. “Selectorate size” appears to be a proxy for military regimes and times of extreme politics! Testing the Assumptions (Getting More Information) Military regime and extreme politics consistent for robust estimation and fixed effects. • There are several methods that can yield more information on country-level effects: • Using robust clustered standard errors – assume that standard errors are independent among leaders of different countries, but are correlated for leaders of the same country. • Fixed effects estimation – useful in conjunction with robust errors when bias in the coefficient is suspected because of country-level heterogeneity. Three methods: • Stratification on country. • Operationalizing fragility as a running tally of leadership turnovers. • Country dummy variables. • Similarly, there are several methods for dealing with the effect of regime type: • Stratification on regime type. • Including relevant regime distinction as a covariate. • Separate models for regime types (or interaction with inconsistent covariates). • MOST IMPORTANTLY: There is no one cure-all for heterogeneity. Utilizing several of these additional tools can help researchers test the robustness of their findings and identify latent patterns in the data. Conclusions • While shared frailty and stratification remain the best practices where they can be applied, they carry assumptions that should be explicitly tested. Doing so will sometimes yield important information for the researcher. • Among the basic tests that should be used to test the robustness of shared frailty specifications are robust clustered standard errors to correct heteroscedasticity and some form of fixed effects to check for coefficient bias. • To test the robustness of regime type stratifications, researchers should separate out models by regime to make sure coefficients are relatively consistent across strata. If they are not, the researcher should consider interacting regime type with covariates or running separate models. In addition, if regime type is of substantive interest, it should be included as a covariate. • In addition to testing the robustness of findings, these methods can be utilized to yield more information for researchers and may suggest some useful alternative hypotheses (or at least force researchers to take another look at their data). • BMSS’ empirical results are surprisingly weak when alternative specifications are tried. While they argue that their measures are just preliminary, it is difficult to apply their theory to any substantive problems without a clear method for operationalizing their concepts. • It is important to use and present multiple methods for dealing with country-level and regime type heterogeneity in comparative politics applications of event history modeling. Doing so helps prevent bias and misleading hypothesis tests. It also provides additional information on the structure of the data and the relationship between variables. Note: Coefficients are reported with p-values in parentheses (1-tailed). Table 5: Effect of Selectorate Variables Accounting for Elections. Selectorate theory covariates appear to be a proxy for elections! Coefficient inconsistent across regime types! Note: Coefficients are reported with p-values in parentheses (1-tailed). Prepared for presentation at the 2007 Summer Methods Workshop, State College, PA. The author would like to thank Jan Box-Steffensmeier, Luke Keele, Bill Miller, Anand Sokhey, and all the participants at the OSU methodology poster session for their help and encouragement. All errors are the author’s.