What can social science contribute to educational policy?

1. What can social science contribute to educational policy? Evidence-Based Policies and Indicator Systems July 2006 London Judy Sebba University of Sussexj.c.sebba@sussex.ac.uk

2. We assume that: • we understand policy-making sufficiently to influence the process • policy-makers are receptive to using evidence • robust, reliable and valid evidence is available • the highest quality research has greatest influence • research reports are accessible and draw out implications for policy

3. Key influences in whether research gets into policy • ministers’ own experiences • charismatic individuals/ studies • timing • role of the press • think tanks - creating a culture of research in tension with culture of ideas (Rich, 2005) • quality of research • quality of reporting

4. The case of MMR: how much further damage? • Wakefield published Lancet paper (1997) claiming link between MMR vaccine & autism/ bowel disease following tests in 12 children • Many subsequently decided not to have MMR • Wakefield paid (by lawyers) to find evidence to support claim of damage by vaccine • Wakefield also filed patent applications for new, alternative single measles vaccine & potential treatments for boweldisease • Horton (editor Lancet) later issued a statement acknowledging potential conflict of interest & study had "fatal flaws" - should never have been published • SR (Demicheli et al, 2005) concluded causal link between vaccine & autistic colitis not proven

5. What can we learn from this? • Fallibility of systems of Quality Assurance - peer review & editing are isolated, untrained, unrecognised (RAE, etc), unpaid roles • Power of one guru (with media) or one study needs to be challenged - e.g. reviews • Role of press (& other mediators) spicing up, manipulation, accuracy, etc • Pressures to be fraudulent, exaggerate, publication bias - (Cooper et al, 1997) • Vested interests - e.g. MMR - SR identifies

6. ‘What’ works is not enough: the example of higher education • Meta-analysis of 58 studies of the relationship between research and teaching in universities demonstrated no relationship (Hattie, J. & Marsh, H.W. 1996 Review of Educational Research) • Students perceived clear benefits from staff research including staff enthusiasm and institutional reputation (Jenkins, A. et al, 1998 Studies in higher education)

7. Formative assessment: a case of research-informed policy & practice • Evidence is robust - large effect sizes replicated • Practical examples in different contexts - see OECD report • Research has been successfully disseminated - mediators were teachers?, academics, LEAs • Policy-timing became right - fits ‘personalised learning’ agenda

8. Lords Hansard text for 20 Feb 2001 (210220-45): Lord Morris of Manchester moved Amendment Lord Morris stated: In 1997, a review of research on inclusion conducted by Judy Sebba for Barnardo's found: Pupils with identified difficulties or disabilities appear to benefit educationally from schools developing inclusive education by making significant gains in reading, language, study skills and living skills. Pupils who do not have identified difficulties or disabilities appear to attain as good or better standards ........ 

9. ‘Too much testing is torture for girls’(Daily Mail November 28, 2002) The Eppi systematic review from which this finding was extracted says: • relationship between self esteem & achievement increased after introduction of the tests • teaching style affected by tests • repeated practice tests reinforce low self image of lower achieving students • students (particularly girls) show high level of test anxiety

10. Improving the utility of research through the research process By being transparent/subject to public scrutiny, systematic reviewing is giving users greater access to research: • identifying the relevant questions • exposing ethics in carrying out research • debating interpretations of findings • providing mediation to increase access Hence, both the generation & use of knowledge are being considered

11. Role of think tanks in research mediation in US (adapted from Rich, 2005) Think Tanks: independent, nonprofit research organizations. • Fourfold increase in their numbers since 1970 • Majority of new think tanks are ideological • Staff more likely to have media or political experience than policy or research degree • Prioritise publicity & marketing

12. What do mediators do?(Fielding et al, 2005, p.103) For transferring practice successfully, Brown and Duguit (2002) recommend appointing an ‘organisational translator’ - someone ‘who can frame the interests of one community in terms of another community’s perspective’. Such people are, unsurprisingly, rare. ...A significant feature of ....research such as that of Rudduck et al (2000) and ....Rudd et al (2001) points to the importance of both internal and external brokers or coordinators.

13. Maximising use of research • Build use of research into infrastructure of policy-making • ‘educate’ policy-makers to develop realistic expectations & ensure on-going involvement of research specialists • bring researchers, users and policy makers together to develop dialogue throughout • set up direct contact between ministers and researchers but researchersneed briefing • seek to establish teamwork between researchers, policy-makers and press

14. Receptivity to research: the public, the professionals & the policy-makers As a society we have been challenged by issues such as BSE and genetically modified food which highlight the need for a scientifically literate population Pilkington, M. & Stuart, M. (2001) Science for active citizenship: the challenge for lifelong learning Journal of Access and Credit Studies

15. Paradigm wars will not bring about social justice or justify public expenditure The goal of an emancipatory (social) science calls for us to abandon sterile word-games and concentrate on the business in hand, which is how to develop the most reliable and democratic ways of knowing, both in order to bridge the gap between ourselves and others, and to ensure that those who intervene in other people’s lives do so with the most benefit and the least harm. (Oakley, A. 2000, Experiments in Knowing, p.3)

16. References • Brown, J. S. & Duguid, P., (2002) ‘Organizing Knowledge’ in Little, S., Quintas, P. and Ray, T. (eds) Managing Knowledge. London, Sage • Cooper, H., DeNeve, K.& Charlton, K. (1997) Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2, 447-452 • Fielding et al (2005) Factors influencing the Transfer of Good Practice. DfES • Rich, A. (2005) Think tanks, public policy and the politics of expertise. Cambridge: Cambridge University Press • Rudd, P et al (2001) Beacon Schools: Further External Evaluation of the Initiative Slough: NFER • Rudduck, J. et al (2000) Schools Learning From Other Schools: Co-operation in a Climate of Competition Research Papers in Education 15 (3) pp 259-274