Reaching Communities Programme Evaluation

1. Project Networking Event, Birmingham Monday 29th October 2007 Reaching Communities Programme Evaluation

2. Welcome Allice Hocking ECOTEC Research & Consulting

3. Introductions Peter Bailey, Big Lottery Allice Hocking, ECOTEC Nicola Hall, ECOTEC Zoey Breuer Rebecca Handley Alison Murray Evaluation Team Project Director Project Manager Research Team Research Team Research Team

4. Today�s networking event 2.00 pm Welcome and Introductions, Overview of the Evaluation 2.15 pm Introduction to Evaluation: The Toolkit 3.00 pm Afternoon tea and networking 3.15 pm Parallel breakout sessions (coloured stickers) 4.30 pm Final session: Round up 5.00 pm Close

5. Purpose of the day Introduce us and the programme level evaluation Introduction to evaluation and explaining the Toolkit Inform you about other support available Projects meet each other Learning from each other and sharing experiences Answering any queries about evaluation

6. Reaching Communities Evaluation ECOTEC Research & Consulting has been commissioned to evaluate Reaching Communities in England and Northern Ireland Introduction to ECOTEC Working with Boyd Consulting in Northern Ireland to cover that aspect.

7. Reaching Communities Evaluation Four year programme evaluation 2007-2010 Various strands of the study including: Evaluation Toolkit Networking events for projects (annual) Website Survey (annual) Case studies Annual reports Events to enable projects to network, meet and share experiences

8. Aims of Programme Evaluation How have projects identified need, most in need and hard to reach groups? Have projects considered local strategies to provide evidence of need? Have beneficiaries been involved in identifying need? Have beneficiaries been involved in project planning/ delivery? Have projects met the needs they identified? Have projects and the programme reduced disadvantage and exclusion? What different approaches have projects used to identify and address need and reduce disadvantage?

9. Introduction to Evaluation: The Toolkit Nicola Hall ECOTEC Research & Consulting

10. The Evaluation Toolkit We have developed a Toolkit to help you to evaluate your project Toolkit in your packs, by post and on the web Ideas of methods and approaches to use Examples of tools to use How to analyse and write up your findings Enable you to input to the programme evaluation Details of other sources of support

11. Benefits of Evaluation Improve the planning and management of your project Help understand how your project is working and how to make changes so it achieves even more Events to enable projects to network, meet and share experiences Involve your beneficiaries in feedback Help you try to secure extra funding by providing evidence about successes

12. How your evaluation fits in Not compulsory but BIG strongly urge you to undertake evaluation We can help! Your evaluation reports (at end of project or annual) can be incorporated into our final reports to Big Lottery Fund Please send your evaluations to us (details at end)

13. What is Evaluation? Different from monitoring Provides an assessment of your project based on the information you collect Monitoring tells you whether you have met your objectives, evaluation tells you �how� or �why� you met them Helps you to understand what worked and what didn�t work as well in meeting the need you identified Self-evaluation involves staff and beneficiaries

15. Getting Started Who will do the evaluation? What do you want to find out? (could use the programme evaluation objectives) What has the project achieved? Inputs (what goes in) Activities (what you do) Outcomes (what comes out) You have all agreed SMART outcomes with BIG so evaluating them should not be too much extra work Use �Explaining the Difference� for information on outcomes

16. Getting Started (cont�d) Can your outcomes be sustained/ continued? Was your project cost-effective? Identified any best practice? Have you successfully involved beneficiaries? What longer impact did your project have on beneficiaries (tracking)?

17. Doing your evaluation Who needs to be consulted? stakeholders, staff, beneficiaries Involving all or a sample? random or targeted sample 10% rule of thumb Setting a baseline Collect data/surveys from the start Measure progress or development For example numbers attending, levels of confidence

18. Doing your evaluation (cont�d) Start with what you have already done! Any data/evidence you used during your application (e.g. local reports or statistics) Recent research or data on the area you are working in Data you already collect (e.g. registration forms) Feedback forms/happy sheets Use this to provide the �context� for your project Why is the project needed? What needs does it meet (evidence of needs) Why did you design it the way you did? Did the needs change? Has it worked?

19. Types of research Quantitative numbers and counting measuring �how many� surveys data analysis Qualitative feelings, perceptions and thoughts explore the reasons �why� words and pictures depth interviews, focus groups, diaries There are two main types of research � quantitative and qualitative Explain each using bullets.There are two main types of research � quantitative and qualitative Explain each using bullets.

20. Types of research (cont�d) Quantitative Postal survey Telephone survey Face-to-face survey Email survey Data analysis Qualitative In-depth interviews Focus groups Diaries Peer research Photographic research Workshops Within those two broad types of research there are a number of different approaches that can be used, you may already be using some of them, where as some may be new to you. Some but not all may be appropriate for your project. Quantitative research includes surveys, such as postal surveys, wheer you send surveys out to people by post, or hand them a paper version when they come to the project Telephone surveys, done by phone Face to face, is where a member of your project team helps beneficiaries to complete the surveys, and asks them the questions as they go along Can also do email surveys using a website, or by sending people an email with the questions. Qualitative includes in-depth interviews (1 to 1) private interviews Focus groups, most well known form, a group discussion Diaries, where beneficiaries record their feelings and participation Photographic research � taking photos of important things Workshops � bit like today where you gather a group of people for a meeting/discussion and to get their feedback. Could be especially good for involving stakeholders. I am now going to talk about each of these types of research in a little more detail, looking at when you might use each one and giving some examples of tools to use.. Within those two broad types of research there are a number of different approaches that can be used, you may already be using some of them, where as some may be new to you. Some but not all may be appropriate for your project. Quantitative research includes surveys, such as postal surveys, wheer you send surveys out to people by post, or hand them a paper version when they come to the project Telephone surveys, done by phone Face to face, is where a member of your project team helps beneficiaries to complete the surveys, and asks them the questions as they go along Can also do email surveys using a website, or by sending people an email with the questions. Qualitative includes in-depth interviews (1 to 1) private interviews Focus groups, most well known form, a group discussion Diaries, where beneficiaries record their feelings and participation Photographic research � taking photos of important things Workshops � bit like today where you gather a group of people for a meeting/discussion and to get their feedback. Could be especially good for involving stakeholders. I am now going to talk about each of these types of research in a little more detail, looking at when you might use each one and giving some examples of tools to use..

21. Quantitative research

23. Surveys � hints and tips Keep it short and simple Closed questions / tick boxes / scales Give clear instructions Use clear language Ask only relevant questions Collect background information (e.g. gender, age) for analysis Check data protection

24. Qualitative research

25. Topic guide example Example for focus groups or in-depth interviews. Shows you how to intridyce yourself to the group, how to gain their confidence. Some ideas of questions you could use with them. Available on the website to download.Example for focus groups or in-depth interviews. Shows you how to intridyce yourself to the group, how to gain their confidence. Some ideas of questions you could use with them. Available on the website to download.

26. Diary example

27. In-depth interviews � hints and tips Accessible, comfortable , confidential location Take notes or record the interview (get permission!) Build the rapport, be attentive Use open questions and probe Focus on how? why? How did this affect you? How did this make you feel?

28. Peer research A great way to involve your beneficiaries in your project Enable beneficiaries to interview each other Provide them with skills and tools to do it Provide guidance / advice Consider some form of incentive? Unique feedback to include in your reports and self-evaluation

32. Interpreting your findings: Qualitative

34. Sharing your findings Writing an evaluation report Summary Introduction / context Summary of method (what did you do?) Key findings Conclusions and recommendations Send us your report ! Presentation, events Media and newspapers Newsletters

35. www.reaching.communities.org reaching.communities@ecotec.com Send us your evaluation report to include in the annual report by December 15th this year Sources of help

36. What�s next�? Afternoon tea Then at 3.15pm go into three breakout groups: B�ham: Zoey Breakout room Becky Main room (front) Nicola Main room (back)

37. Final session: Round up Key themes emerging: Bullet point 02 Bullet point 03 Bullet point 04 Bullet point 05 Bullet point 06

38. Final session: Round up Thank you for coming and joining in Visit our website: www.reaching.communities.org Email us: reaching.communities@ecotec.com Give us your feedback on this event (feedback form in packs) Send us your final evaluation reports Fill in the annual survey for all projects Look out for the end of year one evaluation report (January 2008) Give us your feedback on this event (there is a feedback form in packs please fill in and leave on your seat or hand to a member of the team)Give us your feedback on this event (there is a feedback form in packs please fill in and leave on your seat or hand to a member of the team)

39. Thanks for coming! www.reaching.communities.org reaching.communities@ecotec.com Give us your feedback on this event (there is a feedback form in packs please fill in and leave on your seat or hand to a member of the team)Give us your feedback on this event (there is a feedback form in packs please fill in and leave on your seat or hand to a member of the team)