Unreflective Practice? Case Study and the Problem of Theoretical Inference

1. Martyn Hammersley The Open University Higher Education Close Up 5, University of Lancaster, July 2010 Unreflective Practice? Case Study and the Problem of Theoretical Inference

2. What is Theory? Theory in relation to Practice: ‘OK in theory!’ Irrelevance, guidance, or transformation? Theory versus Fact: ‘Only a theory!’ Speculation as good or bad? Any facts? Theory as Paradigm: No escape from assumptions, and therefore from ideology? Theory by contrast with Description: ‘Explanatory theory’. What causes what? What kinds of theory are relevant to the field of Higher Education Studies?

3. A Caution on Theory and Practice Sophisticated human institutions […] are impossible to manage without a set of ideas that are sufficiently complex and internally consistent to be intellectually credible, but simple enough to provide a workable basis for day-to-day decision-making. Such guiding philosophies are most compelling when they provide clear answers. […] Here, I suspect, is where the greatest challenge for the future lies. For, while the simplified pre-crisis conventional wisdom appeared to provide a complete set of answers resting on a unified intellectual system and methodology, really good economic thinking must provide multiple partial insights, based on varied analytical approaches. Let us hope that practical men and women will learn that lesson. [Mystery Reference]

4. An Ambivalence towards Theory: Schön on Reflective Practice • Existential knowing rather than traditional forms of detached scientific theory. • Theories-in-practice or -use rather than espoused theories. • But double-loop rather than single-loop learning: this gives the phrase ‘practical theorising’ a new sense, it is no longer reactive, a response to experienced problems, but pro-active in questioning assumptions to anticipate or even ‘invent’ problems. Perfectionism?

5. Research as Reflective Practice and the Role of Theory • Does research produce theory? If so, of what kind(s)? • Does the reflective practice of research need to draw on theory? If so, what sorts of theory? Evidence-based research practice? • Methodological theory versus phronesis?

6. Some Neglected Contributions of Methodology • Identifying different kinds of inquiry: academic research, practical research, and inquiry-subordinated-to-another activity. • Recognising limits to the capabilities of research, both intrinsic and temporal. Research cannot serve the function of practical inquiry or practical theorising. (So, Max Weber was right, and Marx, Dewey, Lewin, and Flyvbjerg wrong! In this respect.)

7. The Research Task and its Limits • Intrinsic limits: the task is to produce factual knowledge, not practical recommendations; even less to change the world or for that matter to preserve the status quo. • Temporal limits: given the need for research findings to reach a high threshold of likely validity, and that we must rely upon the current state of knowledge in the field, by no means all significant factual questions are open to worthwhile investigation at any particular time.

8. A Distinction Indicating an Intrinsic Limit to the Capabilities of Research The traditional but increasingly neglected distinction (note, not dichotomy), between: • Factual assumptions, relating to how the world is; and value assumptions, concerned with what is good or bad, right or wrong, etc. ‘Ought’ cannot be derived from ‘is’ alone. • Does research involve value assumptions? Yes, but it only relies upon epistemic values, other values serve as part of value relevance frameworks used solely for working purposes.

9. Stanley Fish’s injunctions • Don’t try to do anyone else’s job. • Don’t allow anyone else to do your job. (Fish, S. 2008 Save the World on Your Own Time, New York, Oxford University Press.)

10. Research is a Demanding Task • The reduction of research to a meaningless ritual, aimed at gaining external funds, producing publications, or career-building. • Its re-interpretation as speculative theorizing: the construction of plausible, appealing ideas. • The obligation, increasingly neglected, to test ideas and only to report findings when they have been well-established. • Also, a common failure to think clearly in methodological terms about what we are doing.

11. Problems with the Methodological Terms We Use • Rather than forming part of well-structured typologies, the methodological terms used to identify different approaches and methods gain their meaning through what they are contrasted with on specific occasions. • They are the product of complex historical relations that can only be understood, at best, by means of genealogical investigation. • They are not good tools to think with.

12. Case Study: A Case in Point What does ‘case study’ mean? • Are case studies always qualitative? • Are case studies concerned with identifying unique characteristics or with producing a theory about a particular category of cases? • Is case study aimed solely at developing causal theories, or also at testing them? • With what other research strategies does ‘case study’ contrast, and on what dimensions?

13. A Challenging Problem • Can case studies be idiographic: concerned solely with documenting some aspect of a particular case? Some perhaps can, but most could not be justified in these terms. • If case studies are to produce general conclusions, what demands does this place upon researchers?

14. Two Distinct Forms of Inference that Produce General Conclusions • Empirical generalisation: from the features discovered in some sample to those of a larger, finite population to which members of the sample belong. • Theoretical inference: from cases studied to all the cases (an infinite number) assumed to fall within the scope of the theory concerned; in other words, to all members of a theoretical category.

15. Empirical Generalisation • What is the population? • Why is it important? • What are the units or cases? • How were the case(s) selected? • What evidence can be provided for using evidence from the case(s) to draw conclusions about the population? • Note that we cannot generalise in this way about causal relations.

16. Theoretical Inference • Here the goal is some set of statements about what types of factor tend to cause a particular type of outcome, and why, i.e. to produce an explanatory theory. • What is the nature of the causal relations assumed here? This varies across qualitative surveys, grounded theorising, analytic induction, and qualitative comparative analysis. • What strategies are available for developing and testing explanatory theories through case study?

17. Strategies for Theoretical Inference • Process-tracing: identifying relevant putative causal processes within cases. But any conclusions from this are hypothetical on their own. • Comparative analysis: systematic comparison of cases in such as way as to develop and test causal hypotheses. But will all the cases required be available? These strategies are complementary, but do they suffice?

18. Conclusion • I have doubts about the possibility of theoretical inference, and therefore about the goal of producing explanatory theories; though we can produce explanations of what occurred in particular cases. • But my main point is that we need more reflection about the nature of the research task, about its limits and the demands it places upon us, about what distinctions need to be respected, and about our current capabilities.

19. References On theory: Hammersley, M. (1995) 'Theory and evidence in qualitative research', Quality and Quantity, 29, pp55-66. Hirst, P. H. (1983) ‘Educational theory’, in Hirst, P. H. (ed.) Educational Theory and its Foundation Disciplines, London, Routledge and Kegan Paul. Mystery reference (Adair Turner or Hector Sants? In the ‘Journal of Turkish Weekly’): http://www.turkishweekly.net/news/104607/-opinion-the-uses-and-abuses-of-economic-ideology.html On Schön, see: http://www.infed.org/thinkers/et-schon.htm On types of research: Hammersley, M. (2002) ‘Varieties of social research: a typology’, in Educational Research, Policymaking and Practice, London, Paul Chapman. On case study: Gerring, J. (2006) Case Study Research: Principles and Practices, Cambridge, Cambridge University Press. Gomm, R. et al (eds.) (2000) Case Study Method, London, Sage. Hammersley, M. (1992) ‘So, what are case studies?’, in What’s Wrong with Ethnography? London, Routledge. On the qualitative survey: Jansen, H. (2010). ‘The Logic of Qualitative Survey Research and its Position in the Field of Social Research Methods’. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 11(2), Art. 11, http://nbn-resolving.de/urn:nbn:de:0114-fqs1002110. On Grounded Theorising, see Dey, I. (1999) Grounding Grounded Theory, San Diego CA, Academic Press. On Analytic Induction, see Hammersley, M. (2008) Questioning Qualitative Inquiry, London, Sage, ch4. On Qualitative Comparative Analysis, see: http://www.u.arizona.edu/~cragin/fsQCA/ On process tracing, see George, A. and Bennett (2005) Case Studies and Theory Development in the Social Sciences, Cambridge MS, MIT Press.