Game Theory and Empirics: What Can Go Wrong

1. Game Theory and Empirics: What Can Go Wrong

2. Using Formal Theory to Identify and Characterize Biases • James Fearon, “Signaling versus the Balance of Power and Interests” • Kenneth Schultz, “Looking for Audience Costs” • Alastair Smith, “To Intervene or Not to Intervene: A Biased Decision”

3. “Signaling versus the Balance of Power and Interests” • Question: How do observable indicators of the balance of interests and capabilities influence the success of immediate extended deterrence? • Intuition: The more the balance of interests and the balance of capabilities favor the defender, the more likely the defender’s deterrent threat will succeed. • Findings: • Some indicators of defender interests (alliance, contiguity) predict immediate deterrence failure. • Past deterrent threat also predicts failure. • Inconsistent evidence on balance of capabilities.

4. The Theoretical Model

5. The Huth Deterrence Data • Population: 58 cases of extended immediate deterrence • A challenger has made a threat against a protégé • A defender has made a deterrent threat to protect the protégé • Data set records • Success or failure: Did challenger back down in the face of the defender’s threat? • Fight or not fight: If challenger did not back down, did defender make good on its threat?

6. Relationship of Data to Model Immediate deterrence

7. Possible Selection Effects • The decision to challenge is conditioned on • the challenger’s type • unobserved by analysts and the defender • the challenger’s prior beliefs about the defender’s type, esp. aD • Observed, at best, with error by analysts • common knowledge to participants • all other payoffs, including the stakes for the challenger, aC • observed with error by analysts • common knowledge to participants

8. Balance of Interests Hypothesis: Observable indicators of the defender’s interest in the protégé (which should increase aD)will predict general deterrence success but immediate deterrence failure. Rationale: • The challenger will only threaten a high-value protégé if it is highly motivated—i.e., aC is high. • As aD increases, probability of challenge goes down, but probability that challenger will act given a threat and mobilization goes up.

10. A Note on Bias Is the estimated effect of alliance ties on deterrence outcomes biased? • It is an accurate measure of the empirical association between these variables. • It is biased estimate of the effect of alliance ties on the defender’s payoffs. • Table 6 seems to suggest that alliance ties decrease the credibility of the defender’s threat by decreasing aD or wD. • With the model, we see that, if alliance ties increase those parameters, the challenger’s self-selection on aC can reverse the sign in the observed relationship conditional on a challenge. • The bias arises because aC is an omitted variable that (conditional on a threat) is positively correlated with alliance ties but negatively correlated with deterrence success.

11. Past Deterrent Threats Hypothesis: A past deterrence crisis between the two states decreases the probability of immediate deterrence success. Rationale: • If the defender made a previous deterrent threat or was previously a challenger, its type distribution going into the current crisis will have higher mean than if there was no prior crisis. • As before, the more resolved the defender is initially expected to be, the more resolved the challenger must be in order to make a threat.

12. The Balance of Capabilities Hypothesis: The more the ex ante balance of capabilities favors the defender (increasing wD, decreasing wC), the more likely it is that a deterrent threat will succeed. Rationale: • Weak challengers only threaten strong defenders when they have a high value for the issue at stake. This should predict immediate deterrence failure. • But: Weak challengers threaten on matters that they initially believe the defender does not care about. This means that a deterrent threat should lead to updating and immediate deterrence success.

13. Evidence • Deterrence success more likely when • short-run and immediate balance of capabilities favors defender • major power defender vs. minor power challenger • defender possesses nuclear weapons (post 1945) • Informal evidence that challenges in the face of large power asymmetries are “limited probes” on issues thought to be peripheral for stronger power. • What we would like to see: Evidence that observable indicators of defender interests are negatively correlated with defender power advantage

14. Summary • The fact that challengers are selecting into crises based on variables that we cannot directly observe (esp., aC, aD) creates selection effects that complicate making inferences about • the effects of observable variables on game payoffs • whether our framework for understanding these kinds of interactions is correct • This paper uses intuition from a formal model to think about the likely direction of the biases induced.

15. “Looking for Audience Costs” • Questions: • Do state leaders incur domestic “audience costs” for backing down after making a threat? • Do democratic leaders incur higher costs than non-democratic leaders? • Possible testing strategies • Indirect: Look for a correlation between variables thought to increase audience costs, e.g., democracy, and outcomes associated with higher audience costs, e.g., acquiescence by targets (Schultz 1999). • Direct: Look for whether leaders who backed down in a crisis face a higher risk of removal than those who did not. • Can you make inferences about the distribution of audience costs (mean, covariates) by examining the sample of costs that are actually incurred?

16. The Theoretical Model C ac, at known Challenge Status Quo T Concede Resist (0,1) C Back Down Stand Firm (1,0) T Back Down Stand Firm (– ac ,1) (– wc , – wt ) (1, – at )

17. Equilibrium Strategies CH/SF CD/BD Target Concedes CH/BD CD/BD Target Concedes SQ/BD CD/BD Status Quo bt CH/SF RS/BD Target Backs Down CH/BD RS/BD Challenger Backs Down SQ/BD RS/BD Status Quo Challenger’s audience costs observed wt kt CH/SF RS/SF War CH/BD RS/SF Challenger Backs Down SQ/BD RS/SF Status Quo Target’s audience costs observed kc bc wc

18. Strategic Censoring • bt is always decreasing in ac  probability of resistance decreases with the audience costs generated by the challenge • kc is nondecreasing in ac  probability that the challenger stands firm is nondecreasing in the audience costs it generated Therefore, increase in ac lowers the probability that ac is actually incurred.

19. Increasing the Challenger’s Costs bt Challenger’s audience costs observed wt kt Target’s audience costs observed kc bc wc

20. Increasing the Challenger’s Costs bt Challenger’s audience costs observed wt kt Target’s audience costs observed kc bc wc

21. Strategic Censoring • kc is nonincreasing in at probability that the challenger stands firm is nonincreasing in the audience costs generated by the target • kt is increasing at  probability that the target stands firm is increasing in the audience costs it generated. Therefore, increase in at lowers the probability that at is actually incurred.

22. Increasing the Target’s Costs bt Challenger’s audience costs observed wt kt Target’s audience costs observed kc bc wc

23. Increasing the Target’s Costs bt Challenger’s audience costs observed wt kt Target’s audience costs observed kc bc wc

24. A Monte Carlo Demonstration Assume audience costs are generated as follows: Assume war costs are generated as follows:

25. A Monte Carlo Demonstration • Assume ac = at = 0, bc = bt = 1, s = 1 • audience costs same for targets and challengers • ln a for democracies ~ N(1,1) • ln a for non-democracies ~ N(0,1) • Generate 100 data sets of simulated data • each set contains 50,000 dyad observations • each state has 30 percent chance of being democratic • for each observation, generate shocks and determine equilibrium outcome from the model • Compare summary statistics of actual and incurred audience costs (logged).

27. Summary • Efforts to find direct evidence of audience costs hampered by • partial observability: we can only measure them when they are incurred • strategic selection: the probability that they are incurred is a function of their value • These problems tend to • lower the mean of observed audience costs • attenuate the effects of covariates on audience costs • What is to be done?

28. Questions: Do states condition their behavior in crises on the expectation that allies will join in the event of war? How reliable are alliance? Fact: Previous research finds that states come to aid of allies in war only about 25 percent of the time. If crisis behavior is conditioned on expectations of alliance reliability, then our estimates of reliability and its covariates can be biased. “To Intervene or Not to Intervene: A Biased Decision”

29. The Theoretical Model A Attack Status Quo B SQ Resist Acquiesce C (an ally of B) ACQ Honor Alliance Stay Out INT BI-WAR

30. Modeling Reliability • C’s decision to intervene: • Can we estimate bc from observational data on whether C intervenes conditional on a war between A and B?

31. Sources of Selection Bias • B’s decision to resist is conditioned on C’s probability of honoring the alliance • B is more likely to resist if C is more likely to intervene • If B has information about C that we cannot control for, then this will bias upward our estimate of C’s reliability • A’s decision to attack is conditioned on C’s probability of honoring the alliance • A is less likely to attack if C is more likely to intervene • If A has information about C that we cannot control for, then this will bias downward our estimate of C’s reliability

32. How B’s Decision Introduces Bias • B’s decision to resist: • B’s decision to resist is conditioned on C’s reliability if Note: Does this make sense?

33. How B’s Decision Introduces Bias • If B resists, • If then • Hence, in the sample of cases in which B has resisted, ec is not mean zero nor uncorrelated with Xc.

34. A Test for Bias • B does not condition on C if • Smith shows that, if these conditions are met, then the distribution of Xc is expected to be the same for all outcomes of the game. • Hence, we can determine the distribution of reliability in the sample of cases that became wars, and test the null hypothesis that the distribution of reliability is the same in the sample of SQ and ACQ outcomes.

35. Data • Observations on 1772 dyads with outcomes • SQ (no force used by either side): 975 • ACQ (force by one side only): 355 • War: 366 (Bigwar: 156) • For each war, every ally of every participant in the war is an observation • Dependent variable: honor alliance or not • Independent variables: age of alliance, type of alliance, regime type, capabilities of C, expected utility indicators

37. Alliance reliability:

38. Summary • Countries do condition behavior on expected reliability of allies. • Average predicted probability of intervention at least .07 higher in cases in which B resists than in cases in which it does not. • Evidence of bias, but what direction? • What is to be done?