Quick Overview - reducing reoffending evaluation pack

1. Quick Overview - reducing reoffending evaluation pack Introduction to a 4 step evaluation approach

2. What is the evaluation resource pack? • The evaluation resource pack is a comprehensive and user-friendly slide-show • which providesstep by step evaluation guidance and resources. It was produced • by Justice Analytical Services at the Scottish Government to help evaluators and • funders conduct better quality evaluations to assess the value of their interventions. • The main purpose of the full version evaluation resource pack is to, • Emphasise the importance of using the evidence-base to design interventions. • Promote a rigorous 4 STEP approach to evaluation which interventions of ALL SIZES and at ALL STAGES OF DEVELOPMENT can conduct. • Help interventions carry out a realistic and rigorous alternative to impact evaluations which are very difficult to do in Scotland • Describe how to structure an evaluation report to increase consistency and quality in report writing • To provide advice to funders on how to judge the merit of interventions in Scotland.

3. Who is the evaluation resource pack for? • The evaluation resource packis designed to support anyone • commissioning or evaluating criminal justice interventions. In particular it • should help, • Contractors and practitioners to • Evaluate criminal justice interventions using a robust 4 step method • Structure an evaluation report • Funders to • Commission evaluations • Judge the strength of evaluation reports • Assess the value of interventions

4. Why do we need an evaluation resource pack? the background to the 4-step method • Assessing the impact of funded interventions in Scotland is difficult due to • methodological constraints but the pressure to show that interventions are • effective has led to some poor and at times dubious evaluations. • To see whether your intervention had an impact and made a real difference to • users, you need a to compare users with a randomised or matched control group • have the same risk of reoffending as the users. You also need large sample sizes • and use statistical testing on the outcomes. • Very few evaluations are able to use this method so we devised an alternative • which mitigates the lack of impact information by emphasising the need to • embed robust evaluations from elsewhere and evaluating the • extent to which an intervention is evidence-based as part of the evaluation itself. • The evaluation then collects data to test whether the intervention was implemented • as intended and whether short and medium term outcomes materialised. • This pack was devised to support evaluators conduct this type of evaluation • which can be described in 4 steps:

5. THE 4 STEP APPROACH TO EVALUATION Interventions should be clearly structured and designed using robust evidence so it is important to be familiar with the results from the ‘what works’ and desistance literature. This knowledge should be used to evaluate the extent to which the intervention is grounded in strong and consistent evidence. You could also be explcit about how much it cost and how the funds were spent. Review the evidence Draw a logic model describing how your intervention works in practice by describing the links between inputs, outputs and outcomes. The logic model forms the basis for evaluating the whole intervention so this may provide better clues as to why an intervention acheived its outcomes or why it did not. Draw a logic model Identify Indicators and collect monitoring data Use this logic model to identify indicators for inputs, outputs and outcomes and collect data using relevent methods Then analyse the data (and collect more if necessary) to find out the extent to which your intervention was evidence-based and if it worked as the logic model predicted it would. Put as much emphasis on describing and evaluating inputs as well as outputs and outcomes Evaluate logic model

6. Contents of the full version

7. Contents continued...