Quantitative versus Qualitative Is a Distraction: Variations on a Theme by Brewer & Hunter (2006)

1. Quantitative versus Qualitative Is a Distraction: Variations on a Theme by Brewer & Hunter (2006) Methods @ Plymouth, 2007 W. Paul Vogt

2. The Theme • Focusing on the distinction between quantitative and qualitative, and on methods of mixing them, “ignores a wider range of methodological problems and opportunities to solve them.”

3. The Variations (Overture) • To think in terms of quant and qual designs is a category mistake. • The q-q distinction diverts attention from other kinds of multi-method research. • It leads to stereotyping and tribalism among researchers. • It encourages us to accept our weaknesses. • It ignores the relation between indicators and concepts. • It is based on confusion about the nature of thinking. • It diverts attention from the nature of the phenomena being studied. • It underemphasizes the extent to which researchers routinely alternate between categorical and continuous variables. • It distracts us from other ways of thinking and handling data, particularly graphic ones. • Mixed methods, by treating the q-q distinction as though it were the most important one, may have the paradoxical effect of reinforcing categories better abandoned or deemphasized.

4. 1. Thinking in terms of quant and qual designs is a category mistake. All designs can accommodate quant and/or qual data. This is an empirical claim. It is easy to find examples of quant or qual data collected using each of the main design types: document analysis, secondary analysis, naturalistic observation, surveys, interviews, experiments, participant observation.

5. 2. The quant-qual distinction deflects attention from other multi-method work. • Document analysis and interviews (both often verbal) • Field observations coded numerically and laboratory experiments coded numerically • Surveys and focus groups

6. 3. It leads to stereotyping and tribalism among researchers. • Correlations and tendencies are confused with logical entailment. • Tashakkori & Teddlie (1998) can illustrate. • Researchers using quantitative “methods” make time-free and context-free generalizations. The number of exceptions is huge, e.g., sociologists who use quant data to study social context, or social historians who use quant data to show change over time. • Researchers using qualitative “methods” do not believe it is possible to discuss causes. Again, the number of counter examples is huge. Causal analysis that does not rely on quant data is venerably old. • Gibbons’ Decline and Fall • Max Weber’s Protestant Ethic • Durkheim’s Division of Labor

7. 4. It encourages us to accept our weaknesses; or provides an excuse for not correcting them • Social psychologists have shown how easy it is for biased side-taking to emerge in social situations. • Ignorance of, say, grounded theory or of regression analysis can be a mark of cultural status among researchers.

8. 5. The important relation of indicators to concepts is not addressed • Is the indicator a symptom? a cause? a component? a predictor? • Law and morality a la Durkheim • Democracy =? fair elections, separation of religion and state, free speech • Principal components analysis and factor analysis (e.g., health and intelligence)

9. 6. It leads to confusion about the nature of thought. • Numbers are words. • One of the most basic of language distinctions, in all languages, is the distinction between singular and plural, one and many, which is fundamentally a quantitative distinction. • Even the most verbal-intensive research contains relations of quantity—such as more or less, mild or strong, and so on. • Even the most number-intensive research contains qualitative distinctions: cause, influence, predict, present/absent, and so on. • Rank order relationships are the somewhat unrecognized meeting ground of qualitative and quantitative data. • Researchers using quantitative data often split continua into categories, such as high, medium, or low on the self-efficacy scale.

10. 7. It diverts attention from the nature of the phenomena being studied. • Height of Mendel’s peas (cat or cont?). • Could evolution could accommodate continuous variation or did it require discontinuous variation? • The battle mattered because it concerned the nature of the phenomena; it had to be resolved on substantive grounds. The issue was the nature of reality, not how to code it.

11. 8. Researchers often alternate between categorical and continuous variables • Falklands War and First Gulf War and suicide rates in the U.K. and U.S. • recessions and the ability to afford university tuition in the U.S. • Quantifying subjective judgments: Pain • Gleason scoring of tumors. Here we have the subjective judgments of physicians about objects. • interrater reliability—inter-subjective. • Quantified subjective judgments are used to make a categorical decision about treatment. • To talk about whether this kind of analysis and decision making is quant or qual is irrelevant. It must inevitably be both.

12. 9. It distracts from other ways of thinking about data, particularly graphic. • Venn diagrams are essential for thinking about overlapping categories. • Change over time in quantitative data is often hard to describe without line graphs. • SEM without graphic means of depicting the measurement and causal models. • Causal models are almost inevitably graphic—not quant, not qual, but graph.

13. 10. Mixed methods may not help • Mixed methods may have the paradoxical effect of reinforcing the categories they were meant to bridge—categories that in many contexts are better abandoned.

14. Conclusions • The characteristics of one’s evidence and how one codes it will constrain one’s analysis strategies—at some point. This point should come relatively later, not earlier. It should be one of the later, not one of the earlier, branches in the decision tree. • What should come earlier?

15. Prior Questions 1 • How can I best gather the kind of evidence I need to answer my research question? (Design) • Who or what do I sample or select for study using that design? (Sampling) • What are the ethical implications of those design and sampling choices? (Ethics) • Only then: Should I code my data using words, numbers, pictures, or all three?

16. Prior Questions 2 Questions about research questions that have priority over the quant-qual distinction include: • Does the problem involve making causal inferences? • Is it necessary to generalize from the cases studied (sample) to a broader group (population)? • Does the problem include the study change over time? • Is it necessary to interact with subjects/participants? • Must one find one’s own data sources and/or generate one’s own data? • Is it necessary to use more than one design?