Causal inference in the social sciences

1. Causal inference in the social sciences Why mechanisms matter and how to find out about them by Attilia Ruzzene Erasmus Institute for Philosophy and Economics, Rotterdam attilia.ruzzene@gmail.com

2. About me: • Phd dissertation: “Causality in economics. A methodological inquiry” • (defended Feb 2010, department of economics Cognetti de Martiis, Torino) • Currently, Phd student at Erasmus Institute for Philosophy and Economics (EIPE, Rotterdam): “Tracing the process: why and how mechanisms matter in the social sciences” • Today presentation: • Brief summary of my past research • Insights to pursue further • Pending problems in need of a solution Overview

3. General motivation to my (first) project: • Relevance of causal knowledge to the variety of purposes economists (and social scientists as well) pursue: • (To differ extent) explanation, prediction, intervention. • Difficulties in having that knowledge reliable • J.S. Mill (1843), on the difficulties of applying its methods for causal inference to a science that is not experimental • Marshall (1870), on the complexity of causal forces as to their number, (lack of) regularity, way of combining (chemical rather than mechanical) • Hence, the questions: • How do economists attain causal information (the methods in use); • What type of causal information this is (do different methods provide causal knowledge that is diverse?); • What purposes it is intended to serve (what type of causal knowledge serves what type of purposes). “Causality in economics. A methodological inquiry”

4. Economists use a variety of methods to draw causal inferences: • Statistical methods (SEM, Bayes Nets); • Experiments (thought ex., natural ex., laboratory ex.); • Case-based methods (process tracing). • I selected two cases –instances- of scientific practice and analysed them in detail to see: • How the methods work in the specific context; • What assumptions they rely on; • What causal information they provide; • What the specific causal information is (thought) useful for. The methods in use

5. Case study research: Regional Advantage. Culture and competition in Silicon Valley and Route 128 by A. Saxenian 1994; • Techniques of causal analysis: small-n method of comparison and process tracing; • Type of evidence: correlation and mechanistic; • Research purpose: explaining socio-economic phenomenon (asymmetry in regional development). • Structural econometric model: Marshallian causal functions by J. Heckman 2000, 2001, 2008; • Technique of causal analysis: controlled (ceteris paribus) variation in thought experiments; • Type of evidence: causal effects • Research purpose: measuring the impact of economic and social policies on outcomes of interest (intervention). The sample

6. Mechanisms play a crucial role in the case-study research: they help causal inference. • In particular, the mechanism gives structure (and order) to the causal backbone (the web of causal relations) around which the historical narrative is articulated. • If the mechanism was not detected the narrative would not be possibly recounted. • Thus, process tracing –and the mechanism it detects- are necessary in the circumstances, though not sufficient by themselves, for the causal inference. • The small-n method plays a complementary and equally necessary role. • Mechanisms play no role in Heckman’s proposal for evaluating intervention. However, they should. • MCF for causal effects are irrelevant in the range of circumstances in which interventions affects the causal structure in intended and unintended ways . • These alterations need to be taken into account if one wants the model to evaluate the impact of intervention. • I believe that what is needed in these circumstances is a model for causal mechanisms. Some conclusions on the role of mechanisms

7. Mechanisms matter in the social sciences. They help us attain epistemic and non epistemic goals by: • Helping causal inference ; • Facilitating extrapolation; • Evaluating interventions. • This formulation is too vague: are mechanisms necessary and sufficient for 1, 2 and 3? • No. Whether and how they matter depends on the context. There is no general demand for mechanisms (Kincaid 1997). • Sensible proposal. By itself, however, it does not tell us when and how mechanisms matter. It simply relocates the methodological investigation and places it closer to the actual scientific practice. • What are the contextual features that make the search for mechanisms desirable? • Daniel Steel argues that it depends on the available information. In the social sciences it happens sometimes that we can more easily access the mechanisms and their components than the causal relationships at the aggregate level (Steel 2008). Mechanisms matter; but, how?

8. What types of clues suggest that the search for mechanisms is desirable in the circumstances? • General clues: field of investigation, research purposes, subject of study, applicable methods, background knowledge. • More specifically, consider Heckman’s MCF for measuring intervention. (If my conclusions are correct) causal effects are irrelevant in the circumstances in which: • The causal structure is affected by the intervention; • The intervention operates by jiggling distant rather than proximate causes of the effect of interest. • These are contextual clues. As such, they • do not require one to have full knowledge of the causal structure in place; • point out the irrelevance of a model for causal effects and the need of a model for mechanisms. • The need for clues is consistent with the idea that in the social sciences reliable knowledge of mechanisms is not easy to obtain... Contextual clues

9. Mechanisms help scientists attaining epistemic and non epistemic purposes under conditions: • They help causal inference only if they solve the problem of confounders (spurious causal relations); • They help extrapolation only if they bear external validity besides being internally valid; • They help assessing interventions only if they are relevant to the policy implemented. The conditions above can be fulfilled only if mechanisms are detected correctly • This is a major issue for the social sciences because social mechanisms are not there ready to be picked up: • They are not directly observable; • They consist of social practices that have to be interpreted (Steel 2004, 2008); • They are bound to uncertainty; • They are context-dependent; • Various mechanisms can operate simultaneously. • Therefore, scientists need first to find out methods that give reliable evidence of causal mechanisms if they are to use those mechanisms as valid evidence for causal and policy hypotheses. Mechanisms matter only if valid

10. Process tracing is the method for detecting mechanisms in the social sciences. • Whether process tracing provides valid evidence for mechanisms and how (at what conditions) is related with how we define mechanisms in the first place; • If we understand social mechanisms as regular sequences of events (Little 1991, Bunge 2004, Mayntz 2004), process tracing should test the stability in the association of events; • but then it would fall prey of the same problems that plague statistical inferences (spurious correlation). • If we understand social mechanisms as sets of social practices (Steel 2004 2008), process tracing is to provide evidence for the existence of those practices. What we need to establish is then how social practices are to be interpreted unambiguously. The troubles of process tracing

11. To sum up: • Social scientists make extensive use of mechanisms and mechanistic language, • And philosophers and methodologists promote the use of mechanisms for epistemic and non epistemic purposes. • However, how mechanisms actually matter and to what extent seem to depend on the context. • Moreover, they can help solve the aforementioned problems if correctly detected. Hence, the importance of • Identifying the relevant contextual features: Clues that tell the researcher that the search for mechanisms is fruitful in the circumstances; • Identifying the conditions under which process tracing secures the validity of the mechanisms it detects. To conclude