InTandem Evaluation: Impact, Outcomes, and Monitoring

1. intandem evaluation

2. intandem evaluation • Presentation contents • Quick exercise • The intandem approach to data collection - evaluation • The intandem approach to monitoring • Recording the data • External evaluation

3. intandem evaluation – quick exercise In pairs (or in threes) identify five different groups who would be interested in knowing the impact the service is having – and explain why they would be interested.

4. intandem evaluation – quick exercise

5. intandem evaluation – five core outcomes Children have increased confidence Children have improved self-esteem Children have improved social relationships Children are more engaged with the local community Children experience reduced isolation 4 3 2 1 0

6. intandem evaluation – confidence and self-esteem • Confidence is a belief in being able to do things – and confidence can vary from one area of your life to another. I am confident I can cycle 40 miles in a day, but I am not confident to speak in front of people without notes. • Self-esteem is how you see yourself in the world, what value you place on yourself. Self-esteem tends to be more consistent across different areas.

7. intandem evaluation – confidence indicators I know the things I am good at and enjoy doing them I enjoy trying to do new things I am good at asking for help I can cope well in new and unexpected situations I am comfortable meeting new people All scored on the basis of Strongly Agree (4), Agree (3), Disagree (2) and Strongly Disagree (1), and on the assumption that we are talking about how the child feels most of the time.

8. intandem evaluation – self-esteem (Rosenberg) I feel that I am as good as other people I think there are lots of good things about me Sometimes, I feel like I am a failure

9. intandem evaluation – self-esteem (Rosenberg) • I am able to do things as well as most other people • I like being me • I feel good about being me • Sometimes I think I am no good at all • I feel I do not have much to be proud of • Sometimes I feel useless • I wish I liked myself more

10. intandem evaluation – self-esteem (Rosenberg) • I feel that I am as good as other people • I think there are lots of good things about me • I am able to do things as well as most other people • I like being me • I feel good about being me • Sometimes I feel like a failure • Sometimes I think I am no good at all • I feel I do not have much to be proud of • Sometimes I feel useless • I wish I liked myself more • What do you think? Reliability, validity and appropriateness or fewer questions and softer statements.

11. intandem evaluation – social relationships and isolation People who care about me People I can talk to about anything People I really like being with People I talk to and play games with

12. intandem evaluation – community engagement Across five areas of community: Sports facilities In school (out of hours) activities Community centres/youth club/libraries Medical and dental services Local facilities – shops, McDonald’s, cinema Four levels of engagement: 0. Zero awareness Awareness Access Participation Ownership / leadership

13. intandem evaluation – community engagement Sports 4 3 2 Shops 1 Out of school 0 Health Entertainment

14. intandem evaluation – when and who? • Baseline data: as soon as practical after the referral has been accepted by you: just knowing that they will be getting a mentor could have an impact. • So, at first home visit, by the co-ordinator. • Difference measurement: every six months – from the date of the baseline data. • So, within the mentoring relationship, by the volunteer mentor.

15. intandem evaluation – recording the data • Use your own systems initially • Small numbers, so no great administrative issue • Always keep the source paper work until information transferred • We are reviewing options for a web-based approach to recording this data • Keep a note of those children whose baseline data is captured before the web-based approach is available

16. intandem evaluation – other data • Recording progress against individual objectives • Significant SHANARRI indicators e.g. safe • Monitoring data for the children • Monitoring data about the mentors • What do you think? Should we track achievement of individual objectives? If so, how should we do this?

17. intandem evaluation – monitoring data: mentees • Age • Gender • Time as LAC@H before matching • Local authority • Post code • Living in: rural/village, small town, large town/city • Number of parents at home; number and age of siblings at home • Outcome: matched or not matched (if not matched, reason for this) • Match history: start date, end date, name of mentor, typical meeting time, activities undertaken • Education: attendance records and instances of exclusion; educational attainment during period of match

18. intandem evaluation – monitoring data: mentors • Age • Gender • Employment status: full-time, part-time, student, unemployed, retired • Local authority • Post code • Initial commitment (in months) • Additional training attended • Outcome: matched or not matched • Match history: start date, end date, age/gender of mentee, travel time to mentee, transport used to meet mentee

19. intandem evaluation – external evaluation • In April we will be discussing with potential evaluators what is required for a successful evaluation of intandem. • What we already know is we need: • Fidelity (all mentors volunteers, same frequency of meetings) • Consistency in data collection • What do you think? How can we ensure that the data is collected consistently if we use the volunteers to be involved in collecting it? And, what do you need from us?

20. intandem evaluation • Questions and Comments…………….. • ……………then lunch.

21. intandem evaluation – Rosenberg self-esteem

22. intandem evaluation – summary

23. intandem evaluation – external evaluation