What counts as ‘evidence’? The complexities of providing evidence to inform public policy

2. What counts as ‘evidence’? The complexities of providing evidence to inform public policy Sally Shortall, School of Sociology, Social Policy and Social Work, Queen’s University Belfast s.shortall@qub.ac.uk

3. Overview • What is evidence-based policy? • The complexities of evidence-based policy • Complexities of ‘evidence’ • Difficulties that arise: examples of evidence provision that did not quite work, and examples of good practice • Moving forward: • What do those involved in policy need to think about? • What do academics/ evidence providers need to think about?

4. What is evidence-based policy? • Policy and practice informed by the best available evidence • Central part of the Blair administrations • Also a drive from within academia – funding bodies require that you can demonstrate ‘impact’ of research • In theory, evidence-based policy sounds logical • Who could be against evidence-based policy? • But when we start trying to do it, it gets messy very quickly

5. The complexities of the policy-making process • The theory: a rational and value-free process that progresses through logical stages • The importance of values, ideology and political beliefs • Ministers strive for re-election • How favourable policies are with the public are key • Finite resources – some policies will be prioritised • Expertise, experience and judgement of civil servants

6. The complexities of the policy-making process • Civil servants and movement between policy areas • Competence rather than subject specialism • Membership of Assembly Committees change over time • Assembly Committee Clerks; again not subject specialists and movement between committees

7. The complexities of the policy-making process • Views of pressing policy priorities inherited • Accepted ideas of policy priorities become embedded in organisations over time, and not necessarily based on evidence • Easier to develop policies that are favourable with stakeholder groups • Lobby and interest groups try to influence the policy process • The reality: a complex process with many actors with different agendas. More realistic to speak of evidence-informed than evidence-based policy

8. The complexity of evidence • Many sources of evidence; academic research, government statistics, stakeholder views, government evaluations of existing policies etc. • Evidence is disputed • Evidence changes over time • The social environment in which evidence is used changes over time • Disputed evidence claims and the demand for public or ‘stakeholder’ engagement in the policy process. Experiential evidence

9. Difficulties that arise • The advent of think-tanks, research advisory panels, academic advisers, commissioned research. • Often relationships are tense – why? Some examples. • Where do you start? There is a glut of evidence. How do you assess the value of one against the other? Between research evidence and stakeholder evidence? • Academics don’t necessarily do research to inform policy • Academics don’t necessarily understand the policy process • Communication skills and an open process

10. Moving forward From a policy perspective: • What do you want evidence for? To identify future policy priorities? Better delivery of an existing policy? • What type of evidence do you want? Review of existing evidence? New research? Variety of types of evidence? • Be very clear about your time frame. Do you want a short ‘rapid response’? • Be very clear about how you want your evidence presented. It is best to provide a template.

11. Moving forward: From a policy perspective: • Majority of civil servants not subject specialists, so best to embed subject expertise into the policy making process. This way a certain expertise is maintained. • Formal mechanisms to enhance this relationship. Develop a register around the key policy areas in the Programme for Government.

12. Moving forward: • From an academic perspective: • Make research findings accessible to non-academic audiences • Learn about the complexities of the policy making process • Bear in mind competent civil servants and MLAs are not necessarily subject specialists. • Disseminate and communicate our research widely. • Present findings in a one page summary.

13. Finally….. • We need to develop open structures and find a space to develop and foster relationships.