evidence and inter-level inference

1. evidence and inter-level inference Patrick McGivern University of Wollongong

2. evidence and inter-level inference Inter-level inference involves inferring a conclusion on one level based on evidence drawn from another. i.e., ‘individual’ level conclusion from ‘group’ level data There are well-known problems with inter-level inference, and various techniques for recognizing and dealing with these. Prominent among these is multi-level analysis. Proponents of MLA argue that it provides insight that can’t be found in any ‘uni-level’ theory.

3. questions • What’s so great about multi-level analysis? (Is it so great?) • What sort of levels are involved in MLA? • How do levels in MLA relate to levels more generally?

4. inter-level fallacies Let’s begin by looking at the problems of inter-level inference in more detail. Group level Ecological fallacy Fallacy of aggregation Individual level

5. ecological fallacy The ecological fallacy is widely discussed in epidemiology, social science, etc. Robinson’s (1950) paper “Ecological Correlations and the Behavior of Individuals” presents a detailed analysis of the ‘fallacy’. Basic idea: group-level ‘ecological’ correlations (i.e., between racial composition and rates of illiteracy) can be quite different from individual-level correlations (so ecological-level data is a poor substitute for individual-level data)

6. what’s the fallacy? Robinson’s most developed example draws on 1930 census data concerning race and literacy. Analyzing the data at the regional level (dividing the US into nine geographic regions), the correlation between portion of the population who are black and the rate of illiteracy in that population is 0.946 We might be tempted to infer that at the individual level, the correlation between being black and being illiterate is also 0.946 (roughly) Diagram from Robinson (1950). The categories ‘white’ and ‘negro’ are those used in the 1930 census. But this is quite wrong: at the individual level the correlation is much lower, 0.203

7. what kind of fallacy? The ecological fallacy is often described as a problem of reasoning by analogy and/or a problem of bias. I.e., the group-level property ‘percent illiterate’ is analogous to the individual-level property ‘illiterate’, and hence we (wrongly) draw an inference from the former to the latter. Talk of analogy/bias might be appropriate in some cases, but here it obscures the reasons we might be drawn to the fallacious inference in the first place. Perhaps it’s more helpful to see the inference as a form of Inference to the Best Explanation – but one that fails (?)

8. IBE Group-level correlation Evidence E i.e., region-level census data Individual-level correlation I2 Individual-level correlation I1 Our evidence indicates a group-level correlation. For independent reasons, we expect group-level correlations to have individual-level explanations. We infer the best (simplest, etc.) one of these – but sometimes this inference fails (?)

9. IBE In fact, Robinson’s point is stronger than this. He shows that the inference from group-correlation to individual-correlation would only work if a particular homogeneity condition held across regions. In the race-literacy case, we have good reasons for thinking that this condition doesn’thold. Hence the evidence for the group-level correlation is actually evidence against the corresponding individual-level correlation.

10. ecological correlations The relation between ecological and individual correlations which is discussed in this paper provides a definite answer as to whether ecological correlations can validly be used as substitutes for individual correlations. They cannot. While it is theoretically possible for the two to be equal, the conditions under which this can happen are far removed from those ordinarily encountered in data. From a practical standpoint, therefore, the only reasonable assumption is that an ecological correlation is almost certainly not equal to its corresponding individual correlation. (Robinson 1950)

11. responses • Individualism. Establishing individual-level correlations requires individual-level data. • Multi-levelism. Establishing individual-level correlations requires a mixture of individual and group level data. The immediate effect of Robinson’s paper was to encourage individualism, but more recently multi-levelism has become increasingly dominant in discussions.

12. multiple levels Multi-level analysis comes in many forms, but the basic idea is that systems are analyzed/modeled on many levels at the same time. For example, Subramanian et al (2009) reanalyze Robinson’s census data and argue that only considering the individual level evidence gives an incomplete picture: to understand variation in literacy, our models need to include both individual and group factors. (In this case, the relevant groups correspond to states.)

13. individualist worry Groups are arbitrary, whereas individuals are not. Hence any explanation involving group-level factors will be somewhat artificial compared to an individual level one. ML Response: the choice of levels shouldn’t be made arbitrarily – instead it should be based on evidence. But this general requirement should apply as much to an individual-level account as to a multi-level one (so, in the absence of any particular evidence, focusing on the individual level is as arbitrary a choice as focusing on geographic regions.)

14. evidence for a level On familiar philosophical accounts of levels, levels correspond to very broad groupings of entities and properties, usually associated with distinct ‘branches of science’. atomic level  physics (?) molecular level  chemistry (?) cellular level  biology (?) The evidence for these as levels consists of (i) the distinctness of the different branches of science and (ii) the discreteness and appropriateness of the suggested domains.

15. evidence for a level In those generic cases, the evidence for a level is distinct from the evidence for a particular theory at a level. In the case of multi-level analysis, the choice of levels is an integral part of the analysis, and the evidence for the ML account is also evidence for the choice of levels within that account. So the relevant levels are empirically determined case by case rather than postulated based on generic principles.

16. inter-level/mixed-level inference Rather than talking about inter-level inference, perhaps we should talk about mixed-level inference: we have evidence that is described on a mixture of levels, and this supports a particular conclusion (perhaps on a different level entirely). Unlike standard accounts of levels in philosophy, the concern is not with the relationship between theories at different levels – at least not if theories involve distinct explanatory frameworks. Instead, the concern is with the appropriate kind of evidence for establishing the presence of a phenomenon.

17. thanks.