‘Nothing as Practical as a Good Theory’

1. ‘Nothing as Practical as a Good Theory’ RAY PAWSON r.d.pawson@leeds.ac.uk

2. The Talk’s Tasks Some new roles for theory in evidence based policy • Systematic Review • Knowledge Transfer • Dealing with Complexity

3. Programmes are ‘theories’ ‘Dishy-David-Beckham-theory’ ‘Interviewer: But do you think the fact that these good-looking blokes are footballers has any effect on girls' attitude to playing football? Girl: No, I think it has more effect on them watching football, well not the football - the guys (general laughter and agreement)’.

4. Rival approaches to systematic review Meta-analysis Narrative reviews Hybrid reviews THEORYDRIVEN REVIEWS www.evidencenetwork.org

5. Megan’s Law - Programme Theory STEP ONE Problem Identification Identify high-risk released sex offenders and create valid and reliable registers STEP TWO Public disclosure Issue bulletins, press releases, call meeting to identify released offenders to their community STEP THREE Sanction Instigation Community joins with police and probation to increase surveillance of suspicious behaviour STEP FOUR Offender response Community actions shame offenders and decrease opportunity of further offence

6. Evidence fragment one: could the law have made a difference? - a ‘prospective simulation’ 6could potentially respond to community notification 36previous offence 12 stranger predatory offences 136serious sex offences 6offender from out of state 24 known to victim 100 no previous offence Petrosino & Petrosino

7. Evidence fragment two: did the law effect recidivism? A matched trial Pre-intervention sample sex recidivism 22% Post-intervention sample sex recidivism 19% Pre-intervention sample arrest slow Post- intervention sample arrest significantly quicker Schram & Milloy

8. Evidence fragment three: how did practitioners respond? Office talk “The Law is an unfunded mandate” “Special Bulletin Notification added more work to already over-worked agents” “There is more pressure to baby sit with SBN cases simply because they are SBN cases” Zevitz and Farkas

9. Megan’s Law - an emerging theory • Opportunity for and effectiveness of surveillance by the community is low. • Increased surveillance and control by law enforcement - though this might lead to detection rather than deterrence.

10. On the shoulders of giants? • Are evaluation’s lessons transferable? • Can we recycle evaluation findings? • But would it work in my patch? • Replication and the search for enduring empirical generalisation OR • Comparison and the confederation of of diverse findings under ‘theory’

11. “THE GENERIC TOOLS OF GOVERNMENT ACTION” Rather than focusing on individual programs, as is now done, or even collections of programs grouped according to major ‘purpose’ as is frequently proposed, the suggestion here is that we should concentrate on the generic tools of government action that come to be used, in varying combinations in particular public programs. Lester M Salamon 1981

12. Carrot theory - a Cook’s Tour How Incentives Fare: • New York Tenements Early 20th C • USA - Erewon State University Campus Late 20th C • Breakfast Room Domesticville, Canada Late 20th C

13. Earmarking ‘death money’ A significant proportion of charitable poor relief, intended for basic nutrition and child welfare, was paid over as ‘death money’ to be used for the extravagant funerals of family members. The neighbours would talk if their wasn’t a “fine layout”. V Zelizer

14. Earmarking ‘blood money’ “I kind of considered it like getting twenty bucks from grandmother. It’s 20 free dollars…. Your going to go blow it on something…. I never really needed it essentially but it was always useful.” L. Anderson et al

15. Earmarking survey response incentives • Charities - NO! • Lotteries - NO! • Cash - YES! $2 $5 $10 “The response incentive lies in the middle ground between more subtle concepts of helping behaviour and naked economic self interest” K. Warriner et al

16. Carrot Theory Refined……. • Payment is the measure - but incentives are the intended mechanism • But subjects act on the measure rather than the intended mechanism • The targeting of incentives will always be distorted via ‘earmarking’ • Earmarking can be benign or a blight in policy terms • Earmarking lies between ‘egoism’ and ‘altruism’ • Earmarking practices will vary according to recipient and context • Policy makers need to consider what are the potential earmarking practices associated with a new incentive, and whether they will support or distort the intended outcome

17. COMPLEXITY New Deal for Communities - The Implementation Chain Policy Architects   Practitioners   Subjects Social exclusion  Social mobilisation  Organisational  Programme

18. Dealing with Complexity • Don’t try to evaluate everything - assume some programme theories work • Use suites of research - both live evaluations and retrospective reviews • Remember to cull evidence from other policy domains • Use horizontal (theories-of-change) cuts and follow implementation sequences • Use vertical (realist) cuts and test programme mechanisms in different contexts • THE BOTTOM LINE - IT IS IMPOSSIBLE TO EVALUATE AND CONTROL EVERY CONSTITUENT PROCESS

19. CALL-TO-ARMS GO FORTH AND THEORISE