90 likes | 338 Views
Evaluation – Principles and methods. Matt Barnard Head of Evaluation NSPCC. Purpose of evaluation. Definition “Examine how a policy or intervention was designed and carried out and with what results.” (Magenta Book) Asks objective questions What were the impacts? How was it delivered?
E N D
Evaluation – Principles and methods Matt Barnard Head of Evaluation NSPCC
Purpose of evaluation • Definition “Examine how a policy or intervention was designed and carried out and with what results.” (Magenta Book) • Asks objective questions • What were the impacts? • How was it delivered? • What were barriers and facilitators? • Did it deliver value for money? • Aims to provide • ‘Scientific’ basis for policy making
Evaluation Design Process Evaluation Design Evaluation implementation Evaluation findings Intervention Design Logic model Intervention implementation planning Intervention implementation
Logic models • Characteristics • Mechanisms not processes • Key steps not every step • Explanatory not descriptive • Reflects theoretical assumptions • Benefits • Sense check • Identifies realistic outcomes • Facilitates evaluation design
Strength of design Strength of design matrix
Strength – Evaluation design Randomized controlled trial Quasi-experimental design Before and after measures
Types of design • RCT • Individual randomization • Cluster randomization/ roll out • BAU/waiting list/alternative services • Quasi-experimental designs • Matched area/ groups • Matched individual • Interrupted time series • Regression discontinuity
Factors influencing methodology • Intervention stage of development • Early exploration • Defined and established but not proven • Transferability • Potential Costs and benefits • Resources • Timescales
Key principles • Clarity about key question • Avoid ‘default’ questions • Methods matched to question • Ensure methods match desired questions • Claims match evidence • Avoid over-claiming • Have a coherent story to tell