Monitoring and evaluation

1. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Monitoring and evaluation

2. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Focus of this workshop session Psychosocial interventions Monitoring Evaluation What they are?Why they are important? Who is involved? PlanningMethods Indicators

3. Write a word or phrase that you associate with the term ’monitoring’ on the cards you have received. When you have finished, display your cards on the board/wall space. If someone has written the same word as you then put your card on top of that card. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What is monitoring?

4. Monitoring is the regular and continuous process of collecting and analyzing data to assess progress and development. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What is monitoring?

5. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Two main types of monitoring in RCRC Process oriented Progress and development of response Activities implemented as planned? Use of resources Problems? How should they be dealt with? New opportunities for improving response?

6. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Two main types of monitoring in RCRC Results oriented Direct results of the interventions Relevance of present and planned activities Are objectives still realistic/relevant? Changes in targeted population / environment (impact) Is more information needed?

7. Is the response still relevant to the needs of the population? PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What is monitoring? Keeps track of inputs, outputs and outcomes using indicators

8. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What are inputs, outputs and outcomes? • Outputs: Measurable achievements that have been ‘put out’ or produced as a result of the intervention • E.g. numbers of people trained, numbers of meetings held, etc. Input Any resources that is ‘put in’ to the program E.g. funds, materials, personnel, time • Outcomes: Changes that have come about as a result of the program • E.g. improvement of psychosocial wellbeing, skills and knowledge on PSS improved,

9. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION INDICATORS Brainstorm: What are indicators?

10. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What are indicators? • Criteria used to measure the data collected • Measure changes related to implementation • Examples of (outcome) indicators in PS responses: • Decrease in stress related symptoms of population A • Increase knowledge and skills in providing PFA • Children regain desire to play • Beneficiaries regain capacity to relate to others

11. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Examples of Input, output and outcome indicators • Output indicators: NUMBERS of people trained; NUMBERS of meetings held; NUMBERS of PS kits distributed Input Indicators: AMOUNT $ spent; NUMBERS of personnel working; NUMBERS of hours • Outcome indicators: NUMBER of children playing (quantitative); self-reported INCREASE in desire for social interaction (qualitative); LEVEL of skill improvement in PSS

12. Keeping track of inputs, outputs and outcomes for activities - An example

13. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Why is monitoring important? Relevance Responsibility and communication Accountability Simply put: Are we on track? Are we on the RIGHT track?

14. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Who is involved in monitoring? Responsibility Including monitoring and evaluation Data collection Data collection Other partners

15. In buzz groups of 2 or 3, discuss amongst yourselves what evaluations are and how they differ from monitoring? Try to write a simple and clear definition of what an evaluation does in a psychosocial response. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What are evaluations?

16. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What are evaluations? “Evaluations explore whether the interventions succeeded in achieving the overall goals and aims of the psychosocial response” Discussion points: Evaluations are meant to be OBJECTIVE – what does it mean and why is it important?

17. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION What are evaluations? Evaluations = Monitoring + more Final evaluation Mid term evaluation M1 M2 M3 M4 M5 M6 M7 M8

18. Evaluations focus on assessing OVERALL AIM/GOAL

19. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Types of evaluations

20. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Evaluating impact • How do you measure the impact of an intervention? • What do you need? Data Data Data Data Data SAME INDICATORS

21. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION How to define Psychosocial indicators • Assess - How is the population affected? • What kind of change is desired? • (Goals/objectives/activities of response) • What is the local definition of psychosocial wellbeing?

22. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Indicators of psychosocial wellbeing

23. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Psychosocial indicators 1. Write three things about yourself or your life that you think show to others that you are feeling good and doing well. 2. Now think and write about how others could measure whether you felt better from one day to the next. 3. In plenary share some of your indicators of wellbeing. We aim to come up with a good variety of indicators, and also some indicators that may be the same for the whole group.

24. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Psychosocial indicators • People have different things that indicate their psychosocial • wellbeing, but if we spent some time we would probably be • able to find some common indicators that everyone agree • would be both general, and specific, enough to determine • the groups’ psychosocial wellbeing. • Indicators of psychosocial wellbeing focus on: • How the individuals are doing • How the group is doing. • The nature of psychosocial is a focus on the individual’s psyche • and on the social interaction and network.

25. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Psychosocial indicators But how will we KNOW if they feel better? We need to help the children feel better…. I KNOW.. We will ask them how THEY know when they feel better and then we will find ways to measure that…

26. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Why is evaluation important? Relevance (response relevant to needs of pop.) Efficiency (time-frame, cost) Impact (succeeded or not) Effectiveness (objective of program met) Sustainability (benefits continue beyond the program)

27. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Who is involved in evaluations apart from staff, volunteers, partners? Responsibility Including monitoring and evaluation Data collection Data collection Other partners External consultants

28. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Sampling Broad and varied Broad & varied Targeted population

29. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION When to monitor and evaluate

30. PSYCHOSOCIAL INTERVENTIONS ASSESSMENT How to conduct assessments? • Two main categories of data collection methods Quantitative Qualitative Surveys, Key informant interviews Questionnaires Focus Group Discussions Psychometric tools Observations (measure reactions, behaviour, feelings, using a scaled measure) 1,2,3 Word descriptions =, %, a:b

31. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Are data needed for these indicators quantitative or qualitative?

32. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Collecting the data Ethical principals to data collection • Well planned and justifiable • Coordination • Clarifying aims and procedures • Participatory and collaborative

33. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Collecting the data Ethical principals to data collection • Comparison groups • Conduct and consent • Privacy and confidentially • Anticipate adverse consequences

34. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Planning for monitoring and evaluation Group work: What needs to be planned to ensure good monitoring and evaluation?

35. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Planning for monitoring and evaluation • Timing • Response length (short vs. long term) • Appropriate time for affected pop. (ex. Do not plan monitoring activities with in-school children during exam time etc) • Time to plan, implement, analyze, follow up (time and resources should be allocated) • Training • Management on PSS • Data collectors • Program team – analysis and reporting

36. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Planning for monitoring and evaluation • Resources • Staff and volunteers • Financial • Time • Logistics • Information sharing • Internal standards and formats • External – different audiences • (beneficiaries/donors/public)

37. PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION Role and importance of M&E in PSS • Helps us understand local – and relevant – meaning of psychosocial wellbeing • Keeps us on the right track • Helps us do our best, now and in the future