Constructing a business case - evidence and pitfalls

1. Constructing a business case - evidence and pitfalls Tom McBride Director of Evidecne

2. Who we are

3. Starting the journey in 2011

4. What Works Centres – a culture of evidence

5. Our vision - is that all children are able to achieve their full potential • Our mission is to ensure that effective early intervention is available and is used to improve the lives of children and young people at risk of poor outcome Our Model is to • Make the case for Early Intervention • Generate Evidence • Use evidence to change policy and practice

6. Making the case Generating evidence Using evidence to change policy and practice Communications Funding Working in partnership Measuring our impact

7. Recent and Future Outputs • Language as a wellbeing indicator (Sep 2017) • What Works in early years pedagogy and practice (June 2018) Early Years • Introduction to social and emotional learning in schools (Feb 2018) • Social and emotion learning: supporting children and young people's mental health (Dec 2017) Mental Health & Wellbeing Inter-Parental Conflict • Why reducing parental conflict matters to the NHS (Mar 2018) • Commissioners Guide to Reducing the Impact of Interparental Conflict (Sep 2017) Vulnerability • Building trusted relationship for vulnerable children and young people (Feb 2018) • Intervening early to prevent gang and youth violence: The role of primary schools (Mar 2018) Supporting Evaluation • Evaluating early intervention programmes: 6 common pitfalls and how to avoid them (Feb 2018)

8. How does early intervention work? Studies show that early intervention is likely to have its greatest impact when targeting seven key outcomes, starting with the four primary domains of child development Cognitive Physical School achievement & entry into the workforce Obesity & Physical Health Social & Emotional Behavioral Crime, violence & antisocial behaviour Mental health & wellbeing

9. How does early intervention work? Studies also show that three additional outcomes are particularly important targets for early intervention Risky Sexual Behaviour Child Maltreatment Substance Misuse

10. Our work

11. Why it matters? The fiscal cost of ‘late intervention’ • EIF has shown that nearly £17 billion is spent each year by local and national agencies on acute and statutory services for children, young people and families in England and Wales. This is equivalent to £287 per person. • £6.4 billion is spent by local government, almost all of which is child protection and children’s social care.

12. How much does late intervention cost locally? • This analysis can be produced for any LA, police force or region in England. • The examples below show the profile of late intervention spend for one local authority in the East Midlands, where £195 million is spent per year in total.

13. ‘Foundations for Life’ “Foundations for Life: What Works to Support Parent Child Interaction in the Early Years” is a groundbreaking assessment by the Early Intervention Foundation of 75 early intervention programmes aimed at improving child outcomes through positive parent child interactions in the early years.

14. Key messages The evidence and cost of 75 interventions were assessed. 17 interventions were identified as evidence-based. There is already good choice of effective addressing children’s noncompliant behaviour. There is good evidence to suggest that if offered at age 3 or later, they may reduce the likelihood of persistent non-compliant behaviour. These programme are low cost. There is less choice of programmes addressing children’s attachment security. These programmes are likely to be high cost. There is less choice of evidence-based interventions that support children’s early learning through parent child interaction. These programmes should be offered in addition to centre-based programmes and are likely to be medium to high cost. Attachment Behaviour Cognitive development

16. The EIF Guidebook

17. The EIF Guidebook

18. EIF Evidence Standards: 10. SCALE 9. ADAPT Established EIF Level 4 8. REFINE 7. 2nd RCT Initial EIF Level 3 6. 1st RCT 5. PRE/POST PILOT Pilot EIF Level 2 4. FEASIBILITY (Verification) 3. BLUEPRINT (Confirmation) Confirmation/Verification NL2 2. LOGIC MODEL (Confirmation) 1. THEORY OF CHANGE (Confirmation)

19. Common Pitfalls in Evaluation • Common pitfalls identified: • No robust comparison group • High drop-out rate • Excluding participants from the analysis • Using inappropriate measures • Small sample size • Lack of long-term follow-up

20. High drop-out rate • Why does attrition undermine confidence in evaluation findings? • Result in an insufficiently large sample size. • Make the study sample unrepresentative of the target population. • Introduce bias.

21. High drop-out rate cont • How can this issue be addressed? • Prevention: Attrition can be minimised by using a range of measures – such as compensation i.e. monetary compensation, vouchers, gifts - which may increase participants’ cooperation with data collection and reduce logistical challenges. • Accounting for it once it has arisen: Examining data to check for issues. • Conduct tests to establish if attrition has undermined representativeness or introduced bias. • Statistically control for any differences that have emerged between groups.

22. Measures • Why do inappropriate measures undermine confidence in evaluation findings? • Validity is extent to which a measure describes or quantifies what is intended. Reliability is the extent to which it consistently produces the same response in similar circumstances. • The use of non-validated measures will make it unclear whether any apparent improvement in outcomes reflects a true effect or is a product of measurement error. • Examples: • Using non-validated measures • Pick n’ mix • Validated but not for the study sample

23. Measures cont. • How can this issue be addressed? • Researchers should use validated measures which are suitable for the intended outcomes of the programme, and appropriate for the target population.

24. Our work on business cases

25. General purpose of business cases Management and planning tool to support new or existing policies Five different types

26. Different types of cost-benefit analysis for Early Intervention Spending on proposedservice(s)Spending onother servicesthat will beaffected

27. Benefits of Early Intervention programmes Example: Positive relationships

28. Costs of acute/statutory services Example: Crime

29. Further information Costs and benefits of Early Intervention programmes: Social Research Unit: Investing in Children Blueprints for Healthy Youth Development Unit costs of service provision: PSSRU: Unit Costs of Health and Social Care DfE: Family Savings Calculator New Economy Manchester: Troubled Families Cost Database

30. Our current position. There are 3 types of benefits Cash Releasing Savings • Reductions in expenditure immediately or in the short-term e.g. • Reducing the number of children in the care system or • Increasing the number of people in employment • Reduced pressure and allows more efficient and effective use of resources in the short-term e.g. • Fewer referrals to social services • Fewer admissions to AnE for substance misuse Resource Savings • Long-term benefits to the individual and society in-terms of cash releasing, resources savings and wider social benefits e.g. • Higher productivity • Reduced mental health issues Economic Benefits

31. Our Current Position Can early intervention save money? • The economic case for early intervention is strong, but isn't about cashable savings • The real benefit is long-term benefits to the wider economy and society - leaving issues in childhood unresolved has pervasive negative consequences for children’s later life outcomes, leading to extra costs to the public sector • It is difficult to estimate benefits with accuracy, but there is a wide literature which shows plausibly that the pay-offs can be considerable • The value of these benefits are likely to be considerably higher than the cost of intervening

32. www.eif.org.uk tom.mcbride@eif.org.uk @Tom_EIF