The Point of Educational Research: Truth, Impact, and Stories

1. Martyn Hammersley The Open University, UK MIC International Research Methods Summer School, Limerick, Ireland, June 2011 What’s the Point of Educational Research: Discovering the Truth, Telling Stories, or Having an Impact?

2. Some cautious preliminaries ‘What’s the point’ can refer to: The intended product(s) of educational inquiry; The motives individuals have for pursuing it; The social function(s) it might serve; The rationales for it researchers give to various audiences. We can also draw these distinctions when asking what the point is of any particular study. A slightly different slant: To whom and for what should researchers be accountable?

3. The attack on educational research in the 1990s in the UK • Much educational research 'is […] at best no more than an irrelevance and a distraction'; 'considerable sums of public money are being pumped into research of dubious quality and little value' (Chris Woodhead 1998 – he was then Chief Inspector of Schools, England and Wales) • Charles Clarke, as Parliamentary Under-Secretary of State for education, stated that his aim was to ‘resurrect educational research in order to raise standards’ (Clarke 1998).

4. The crisis in the United States • In the US, the No Child Left Behind Act (2001) and the Education Sciences Reform Act (2002) more or less enshrined the randomised controlled trial as the gold standard for educational research. • ‘Born out of a “methodological fundamentalism” that returns to a much discredited model of empirical inquiry […] such regulatory activities raise fundamental […] issues for scholarship and freedom of speech in the academy’ (Denzin et al 2006:770)

5. The evidence-based practice movement in education Source in evidence-based medicine. Key injunctions: • Effective policies and practices can only be identified by research designed to discover ‘what works’, i.e. the ‘effectiveness’ of policy/practice. • This type of research should be the primary task of educational researchers. • Teachers must access research findings and modify their practice accordingly.

6. Implications for research • Educational enquiry should be designed to serve the the work of policymakers and practitioners; though it is often insisted there is still a place for ‘blue skies’ research. • Randomised controlled trials are the gold standard. Subsequent liberalisation: quantitative methods generally are primary, but with a subordinate role for qualitative research, e.g. in documenting learner perspectives. 3. Systematic syntheses of results from multiple studies are required, designed to provide reliable evidence. ‘Traditional’ reviews of research literatures are of little value.

7. The current situation • In the UK the crisis has subsided, though perhaps not gone away. This item was on the radio 10 days ago: ‘I’m very sceptical of the findings of all education research. It tells you more about the ideological prejudices of the researcher than about the issue being researched’ (Chris Woodhead, now chairman of a company running private schools, Today Programme, 15.6.11) • And there is, of course, a general threat to social science research from substantial cuts in Government spending.

8. The problem is not just external • Public attacks on educational research reflect, in part, debates that have been recurrent within the research community. • These were initially prompted by the conflict between quantitative and qualitative approaches. • But in the past couple of decades there has also been significant conflict among different qualitative approaches. • And there are certainly important methodological problems to address.

9. Conflicting views: • Research must ‘demonstrate conclusively that if teachers change their practice from x to y there will be significant and enduring improvement’ (Hargreaves 2007:9) • It should ‘“sap power”, […] engage in struggle, to reveal and undermine what is most invisible and insidious in prevailing practices’ (Ball 2007:117) • ‘I want to […] propose frivolity as a way of [research] discomposing the language of policy and thereby […]unsettling its totalising ambitions’ (MacLure 2006:1) • My own position: the only legitimate operational goal for research, and all it should be accountable for, is the production of value-relevant knowledge.

10. Complexity and polemic • It’s important to recognise that there are several distinct issues underpinning these disagreements. • Nor are these issues by any means straightforward or easy to resolve. • Yet much discussion proceeds through reliance upon stereotypes, and use of boo-words like ‘positivism’, ‘post-positivism’, ‘neo-realism’, and ‘postmodernism’.

11. Two axes of dispute: • What is the goal or intended product of educational research? Knowledge, or something else instead/as well? As noted earlier, it is important to distinguish between the motives for doing research, or for selecting a particular research topic, and what the intended product of inquiry is. • Can educational research produce knowledge? What is meant by that term? What kinds of knowledge, if any, should educational research aim at?

12. First axis: Goal Is the operational goal of educational research: • To produce educationally relevant knowledge that may or may not be used as a resource by policymakers, practitioners and others; or • To have an impact on policy or practice: • To improve policy and practice (Hargreaves); • To resist and transform these in line with some alternative ideal to that which is dominant (Ball?); • To unsettle conventional wisdom and all claims to knowledge, including that of social science, so as to open the way for change whose character cannot be anticipated (MacLure).

13. Second axis: Facts or fictions? Does research produce: • Truths: Descriptions, explanations and theories that accurately represent the state of the world (including the perspectives, intentions, and actions of people belonging to ‘other cultures’); OR • Fictions: As-if accounts – in prose, poetry, or visual form – to be judged in aesthetic, ethical, and/or political terms.

14. So, what is the point of educational research? I’ll examine some of issues I’ve mentioned by addressing the three answers to this question listed in my title: • To discover the truth • To tell a story • To have an impact I will concentrate mostly on the first of these, since this is the traditional view, and in my view needs defending since, subject to some qualifications, it is the correct answer to the question!

15. What’s wrong with trying to discover the truth? Nothing at all, in my view, but there are: • Problems with the concept of ‘discovery’: Is it possible to uncover and reveal ‘what is there’? No, because we always have to make sense of what is there, to construct an account of the world. Does this mean our accounts are fictions? No! • Problems with the concept of ‘truth’: It is always the truth about something: in other words a true answer to a question, not ‘The Truth’, in terms of either exhaustiveness or absolute validity. Knowledge ≠ reproduction of reality.

16. These complexities have important implications for the pursuit of knowledge For each study we must ask: • What are its starting assumptions about what is already known, and about what is worth knowing and why? • What does this investigation promise to contribute? • What exactly is the question addressed, or to be addressed? • What assumptions frame what counts as an answer?

17. Banal issues that everyone is aware of? • In my view, these issues are not given anything like the attention they require. • Part of the reason for this may be that researchers are preoccupied with other goals than the production of knowledge. • I will focus here on the importance of starting assumptions and framing assumptions.

18. Starting assumptions or findings? An example • ‘This book [Degrees of Choice] shows how the welcome expansion in higher education has also deepened social stratification, generating new inequalities. While gender inequalities have reduced, those of social class remain and are now reinforced by racial inequalities’ (Reay et al 2005, quotation from blurb on the back of the book; bold added) • Yet, as the opening sentence of the blurb indicates, the actual research focus is much more specific: it is a study of social class differences in how applicants decide to which universities to apply.

19. Varying status of claims made • That the expansion of higher education in the UK is ‘welcome’ [Starting assumption?]; • That social stratification has ‘deepened’ [Not clear even what this means]; • That there are ‘new’ inequalities [Presumably these are class inequalities in type of university attended]; • That gender inequalities have reduced [Status unclear, since very little evidence re this provided]; • That social class inequalities in access to higher education remain [Starting assumption?]; • That these are being reinforced by racial inequalities [Empirical finding?].

20. The role of framing assumptions: a joke When Willie Sutton was in prison, a priest who was trying to reform him asked him why he robbed banks. ‘Well’, Sutton replies, ‘that’s where the money is’. (Garfinkel 1981:21)

21. Learning from the joke The framing assumption behind the priest’s question was: Why does Sutton rob banks [rather than earning money legitimately]? The bank robber’s answer operates within a different frame: Why does he rob banks [rather than robbing other sorts of establishment]?

22. Degrees of Choice revisited The central research question was: • Why do more working class (than middle class) applicants apply for non-elite universities? Or, to put it another way: • What factors prevent equivalent numbers of working class (compared with) middle class applicants from applying to elite universities? A framing assumption that seems to be adopted in this study is: any reasons working class applicants have for applying to non-elite universities are socially caused, whereas a decision on their part to apply for elite universities would not need explaining.

23. Clarifying the focus So, in thinking about any particular study, one already carried out or a new project proposed, we need: • To distinguish between the starting assumptions and the questions addressed; and • To be clear about the frame within which any answer to those questions is being formulated.

24. Types of question • Descriptive (What happened? What features?) • Explanatory (Why did it happen? Why those features?) • Evaluative or prescriptive (Good or bad? What is to be done?) [Personally, I don’t believe that research has any authority in offering answers to this type of question.] Which type of question is being addressed has implications for what must be done in order to answer it. Type of question determines type of intended product, and type of finding.

25. Telling a good story? • In an important sense, this is an essential part of writing a research report. • But, what do we mean by ‘story’? • And what would be a good, and what a bad, story in the context of educational research? What is required is an argument that is well-designed to answer the research question(s). • The art of storytelling? Yes, but, I suggest, ‘art’ in the sense of ‘arts and crafts’ rather than modern art.

26. Having an impact? • All of us want people to read our research, and for them to learn something from it. • But ‘having an impact’ or ‘making a difference’ go beyond this, and beyond the idea that producing knowledge is the only operational goal of research. These extensions of goal set up illegitimate forms of accountability for research. • Furthermore, a key question is: what kind of impact or difference? Impact in itself is of no value. • The nonsense of ‘actionable knowledge’: rests on the false idea that there is only a single set of practical or policymaking conclusions that can be derived from any set of factual conclusions.

27. So, what is the point? • To answer factual (not normative) questions that are of educational relevance. • We must aim at answering these in ways that should be convincing to anyone with an interest in the topic, whatever their background assumptions or ideological preferences. • We need to retain a sense of realism about what questions educational research can and cannot answer • We should neither exaggerate nor downplay the likely validity of our findings: the twin dangers of speculation and crude empiricism must be avoided.

28. Conclusion Producing educationally relevant factual knowledge is the distinctive, and should be the only operational, goal of educational research. It cannot legitimately produce practical evaluations or recommendations for policy or practice Researchers must resist criticisms that producing relevant factual knowledge is not sufficient to warrant funding; or that it is not possible. They must insist on the importance of educational research, and recognise the difficulty of doing it well.

29. References Ball, S. J. (2007) ‘Intellectuals or technicians?’, in Hammersley, M. Educational Research and Evidence-based Practice, London, Sage. First published in 1995. Denzin, N. and Giardina, M. (eds) (2006) Qualitative Inquiry and the Conservative Challenge, Walnut Creek CA, Left Coast Press. Garfinkel, A. (1981) Forms of Explanation, New Haven CT, Yale University Press. Hammersley, M. (2000) ‘The relevance of qualitative research’, Oxford Review of Education, 26, 3-4, pp393-405. Hammersley, M. (2002) Educational Research, Policymaking and Practice, Paul Chapman/Sage. Hammersley, M. (2008) Questioning Qualitative Inquiry, London, Sage. Hargreaves, D. H. (2007) ‘Teaching as a research-based profession’, in Hammersley, M. Educational Research and Evidence-based Practice, London, Sage. First published in 1996. MacLure, M. (2006) ‘Entertaining doubts: on frivolity as resistance’, in Satterthwaite, J. et al Discourse, Resistance and Identity Formation, Stoke-on-Trent, Trentham Books. Reay, D., David, M., Ball, S. (2005) Degrees of Choice, Stoke-on-Trent, Trentham Books. Slavin, R. E. (2002) ‘Evidence-based education policies: transforming educational practice and research’, Educational Researcher, 31, 7, pp15-21. Woodhead, C. (1998) Foreword to Tooley, J. and Darby, D. Educational Research: a critique, London, Ofsted. Available at: http://www.ofsted.gov.uk/Ofsted-home/Publications-and-research/Browse-all-by/Education/Leadership/Governance/Educational-research-a-critique-the-Tooley-report

30. Some work of mine that develops these arguments: The Politics of Social Research, London, Sage. Taking Sides in Research: Essays on partisanship and bias, London, Routledge. ‘Can and should educational research be educative?’, Oxford Review of Education, 29, 1, 2003, pp3-25. ‘Action research: a contradiction in terms?’, Oxford Review of Education, 30, 2, pp165-81, 2004. ‘Reflexivity for what? A response to Gewirtz and Cribb on the role of values in the sociology of education’, British Journal of Sociology of Education, 29, 5, 2008, pp549-58. Questioning Qualitative Inquiry, London, Sage. Methodology, who needs it?, London, Sage.