NAVCA Annual Conference 2006 The National Outcomes Dissemination Programme: Evaluation

1. NAVCA Annual Conference 2006The National Outcomes Dissemination Programme:Evaluation Mike Aiken and Rob Paton Public Interest and Non-profit Management Research Unit Open University m.aiken@open.ac.ukr.c.paton@open.ac.uk

2. National Outcomes Dissemination Programme • 1. What was the Programme? • 2. Why outcomes? • 3. Some Programme achievements • 4. How was it implemented by infrastructure organisations? • 5. What have we learnt from the programme? • 6. What next?

3. 1. National Outcomes Dissemination Programme (NODP) Aimed over 3 years to: • increase the pool of individuals and organisations across the sector with the skills and confidence to train and mentor VCOs in outcomes management. • improve the skills understanding and confidence of a diverse range of 500 VCOs throughout England. • increase the understanding of Government and sector policy makers of the support and information needs of the sector in outcomes management.

4. Programme conception and delivery • Concept: cascade training and development programme • Funded by Big Lottery • Devised & piloted by CES • Major contribution of infrastructure organisations in early concepts, recruitment and delivery (especially NAVCA, ACRE, CEMVO).

5. 2. Why outcomes? Some recent policy & practice drivers • The increasing importance of stakeholder transparency, quality processes and an outcomes focus in the voluntary sector, government departments, funding and regulatory bodies • The need to demonstrate voluntary organisations are effectively run and delivering on promises. • Outcomes have become mainstream - for infrastructure organisations too! • See (Charities Evaluation Services 2005; NCVO 2004; Home Office 2005; Big Lottery 2005; Charity Commission 2005; (Audit Commission 2005)

6. 3. Some Programme achievements • A 200 plus page manual containing curriculum and course organisation • 8 four day cascade training courses were developed and delivered • 76 Outcomes Champions trained in 9 English regions and 3 sub sectors • Outcomes Champions cascaded training to over 631 Voluntary and Community Organisations

7. …longer term effects? • The majority of Outcomes Champions have started to implement outcomes approaches in their own organisations • Most Outcomes Champions intend to continue to deliver outcomes training or built it into their role • The majority of VCOs trained are seeking to shift towards, develop or improve an outcomes focus in their organisation • Early signs that VCOs trained over a year ago are maintaining and integrating the ideas into their ways of working,

8. 4. How was it implemented by infrastructure organisations? • ‘Classic’ and ‘jazz’ styles by the Outcomes Champions in their training and development role of VCOs • OCs Striving to implement in own infrastructure organisation important • Post- programme, OCs varied roles: • Activists: ‘deliver more courses and keep active’ • Integrationists: ‘building the work into their role’ • Programmers: ‘happy but programme has ended.’

9. 5. Some programme learning • Cascade worked • Local adoption important for OCs & VCOs • Importance for policy makers of outcomes - & their need to meet their own needs • Funder education important

10. 6. What next? • Report & summary of evaluation: coming soon • New programme: in development phase by CES • Outcomes: likely to remain and become part of any new structures and crucial for infrastructure organisations • Measurement: a search and find experience? • The education of funders?

11. NAVCA Annual Conference 2006The National Outcomes Dissemination ProgrammeEvaluation: summary Mike Aiken and Rob Paton Public Interest and Non-profit Management Research Unit Open University m.aiken@open.ac.ukr.c.paton@open.ac.uk

12. Additional notes • Copies of the report and summary will be available in October from: • Charities Evaluation Service 4 Coldbath Square London EC1R 5HL • http://www.ces-vol.org.uk/ • Further notes are included in the following pages

13. NODP - Programme in outline • A phased cascade training programme devised and developed by CES • Delivered over a three year period (2003-6) • Active in all nine English regions • Also active in three ‘sub-sectors’ (BME; rural community sector and volunteer development).

14. NODP - Programme detail • Lead Trainers set up by CES provided training, information and mentoring support to 76 intermediary trainers known as ‘Outcomes Champions’) • Outcomes champions recruited in partnership with infrastructure networks in each region & 3 sub-sectors (BME, Rural, volunteer development • The aim was to equip Outcomes Champions’ to pass on their learning to VCOs in their networks. • Outcomes Champions agreed to offer training and support to 10 further VCOs within 6 months

15. Programme delivery • Outcomes Champions (OCs were recruited in partnership with infrastructure networks, such as CVSs • Recruitment in each of the English regions (54 OC intermediaries in all) • …and the three sectors of BME, rural and volunteer development organisations (22 OC intermediaries in all).

16. Outcomes Champions adoption strategies from cascade • The ‘classical’ players - stayed close to the programme as written • The ‘jazz players’ - improvised more freely from the programme • Between classic and jazz - largely stayed but also improvised

17. Approaches to implementation by VCOs • (A) No problem - some VCOssaw it as unproblematic • (B) Step-by-step - saw it requiring significant shifts. • (C1) Enthusiastic integrators - some had taken it on enthusiastically and rapidly. • (C2) Culture change integrators - some VCOs found that implementation would take them longer and require a culture change. • (D) Not right now, thanks! A few VCOs were sceptics & felt they had sufficient processes.

18. Programme achievements (1) • A comprehensive 200 plus page manual containing curriculum and course organisation devised, piloted and delivered to Outcome Champions • 8 four day cascade training courses were developed and delivered by CES’s Lead Trainers to Outcomes Champions (target 7) • 76 Outcomes Champions were trained (target: 60) • … in 9 English regions and the 3 sub sectors

19. Programme achievements (2) Outcomes Champions trained included: • 5 working specifically with Black and Minority Ethnic voluntary organisations; • 8 in Volunteer Development support organisations • 9 in rural development infrastructure organisations • Outcomes Champions cascaded training to over 631 Voluntary and Community Organisations (target 500) • The VCO came predominately from local or community organisations • Reports from OCs and VCO participants: course satisfaction was very high.

20. Programme impacts • The impact of the programme is likely to be considerable: • The majority of Outcomes Champions have started to implement outcomes approaches in their own organisations • Most Outcomes Champions intend to continue to deliver outcomes training or have built this into their role • The majority of VCOs trained by the Champions are seeking to shift towards, develop or improve an outcome focus of some kind in their organisation • There are early signs that some VCOs trained over a year ago have maintained or integrated some of the outcomes thinking into their ways of working.