Benchmarking impact: lessons from transnational practice

1. Benchmarking impact: lessons from transnational practice

2. Context • The ESF ‘What Works’ project, based in Newhaven, East Sussex. • Funded through ESF’s ITM Innovation, Transnational and Mainstreaming programme • Delivers Welfare to Work services to disadvantaged local residents, from community based organisation • Supply and demand side measures, distinguished by: • strong ‘bottom up’ ethos; • commitment to continued support for all clients irrespective of outcomes; • continual adaptation in face of WTW framework changes

3. Context II • ITM supports innovation, research and exchange of experience with partners in other member states, as intrinsic to supported projects • Core research attempted to identify any particular advantages from NewCEP ‘model’ and feed in learning to develop practice over time – including experience from partners • Concluded that difficult to isolate significant differences in outcomes from the particular model – perhaps attributable to increasingly restrictive contractual and monitoring environment • Led to exercise to contrast monitoring and evaluation, and outcome definitions, amongst partners in other countries

4. Transnational partners • Patim, Castellón, Spain • Santa Casa da Misericórdia de Amadora, Lisbon, Portugal • Calder Holdings, Amsterdam, Netherlands • Krami, Malmö, Sweden • (Prosalis, Lisbon, Portugal)

5. Assessment driven primarily by economic model Payment by outputs NewCEP (UK) Calder (NL) Krami (Sw) Target driven Invoicing driven Santa Maria di Misericordia (Port) • Patim (Esp) Payment by inputs

6. Payment by results Payment by outputs • Calder (NL) • Multiplicity of clients / contracts • Limited data • Continual collection • Strong link to staff performance • Database design = competitive advantage • NewCEP (UK) • WTW and ESF • Shifting emphases (sustained jobs, family intermediate steps) • Contracting model prevents comparisons of models • Directly led to reorganisation Target driven Invoicing driven Payment by inputs

7. Payment by outputs Employment targeting • Krami (Sw) • 50% employment targeted – routinely met • Satisfying politicians and probation on this gives scope for more ‘holistic’, theory driven approach • Lack of comparative, learning driven evidence to support questions about high intensity model Target driven Invoicing driven Payment by inputs

8. Payment by outputs Practice delivery Target driven Invoicing driven • Patim (Esp) • Collapse of employment led to shift to ‘personal capacities model’ • Substantial use of assessment tools • Organisational survival driving data use • Santa Maria (Port) • Traditionally grant funded, increasing need to demonstrate value • Innovating by adding demand side provision to existing supply side • Monitoring and evaluation self defined = competitive advantage Payment by inputs

9. Emerging findings • Monitoring is increasing everywhere • UK and NL ‘contract culture’ creates commodities which shape organisational structures and methods • NL experience suggests output model can stimulate service innovation, but with enormous focus on jobs first • Data management central to commercial viability

10. Emerging findings II • Swedish model depends on satisfying funder partnership expectations • Krami seeks to balance delivering job targets with explicit search for individual independence – interesting refinement of hypothesis: Funders: job reduced benefits / crime Krami: job independence / self reliance reduced benefits/ crime

11. Emerging findings III • Patim in Spain shares goal of individual development – UK’s ‘holistic / whole person’ approach much more strongly theory based on the continent. Audience for progress measures in Patim becomes service users • Economic survival drives innovation in data use and greater propensity to demonstrate effectiveness

12. You are welcome to contribute to the study • https://www.surveymonkey.com/s/transnationalbenchmarking • Follow the study on ESF-Works.com Thanks for listening, questions and comments welcome