260 likes | 444 Views
Promoting Rational Drug Use in the Community. Monitoring and evaluation. Objectives Session on Monitoring and Evaluation. Understand functions of monitoring and evaluation Define what should be monitored Formulate evaluation questions Identify methods
E N D
Promoting Rational Drug Use in the Community Monitoring and evaluation
Objectives Session on Monitoring and Evaluation • Understand functions of monitoring and evaluation • Define what should be monitored • Formulate evaluation questions • Identify methods • Make a plan for monitoring and evaluation Monitoring and evaluation Promoting Rational Drug Use in the Community
Place of Monitoring and Evaluation in the Planning Cycle Problem Goal Objective Activity Monitoring Evaluation Monitoring Refers to Activities Evaluation refers to Objectives Monitoring and evaluation Promoting Rational Drug Use in the Community
Functions of monitoring • See that everything goes according to plan • Find out if there are unexpected difficulties • Adjust plans Monitoring and evaluation Promoting Rational Drug Use in the Community
Functions of Evaluation • Prove that the intervention worked • Assess if costs were reasonable • Convince others that intervention works • Share experiences - to allow replication Monitoring and evaluation Promoting Rational Drug Use in the Community
Indicators • Indicators are needed for both monitoring and evaluation • Monitoring requires process indicators • Evaluation requires outcome indicators • Planning to collect indicators is an important part of any plan Monitoring and evaluation Promoting Rational Drug Use in the Community
What to monitor • Formative research(RAP) done? • Intervention pre-tested? • Activities implemented according to plan? • Are costs in accordance with budget? • Are staff carrying out assigned duties? • Is collaboration with others taking place? Monitoring and evaluation Promoting Rational Drug Use in the Community
How to monitor • Record keeping • Making reports on activities • Field or supervisory visits • Regular project/programme meetings Monitoring and evaluation Promoting Rational Drug Use in the Community
What to evaluate? • Preparation • Planning • Implementation • Effect Monitoring and evaluation Promoting Rational Drug Use in the Community
Preparation • How was the problem identified? • Was formative research done • Who is involved in the intervention • How comprehensive is the intervention (consider the factors which cause the problem) Monitoring and evaluation Promoting Rational Drug Use in the Community
Planning • What objectives were set? • Which activities were planned? • What target audiences were identified? • Were the interventions pre-tested? • Was a plan for monitoring and evaluation made? Monitoring and evaluation Promoting Rational Drug Use in the Community
Implementation • Which of planned activities were carried out? • What messages were disseminated? • How many people did the message reach? • Did the target audience pay attention? • Did the target audience understand? • What problems were encountered Monitoring and evaluation Promoting Rational Drug Use in the Community
Effect • Change in knowledge? • Change in behaviour? • Change in health service quality? • Change in policy? • Improvements in health? • Negative or unexpected effects? Monitoring and evaluation Promoting Rational Drug Use in the Community
Four evaluation designs • Randomised control design • Quasi experimental design • Time-series design • Pre-post design Monitoring and evaluation Promoting Rational Drug Use in the Community
Randomised control design intervention group receives education random assignment of è intervention and measure change control group; after intervention at the beginning è of the intervention control group receives no education Quasi-experimental design intervention group receives education specifically selected è intervention and measure change control group; after intervention at the beginning è of the intervention control group receives no education Four evaluation designs Type of design Action Measuringchange Monitoring and evaluation Promoting Rational Drug Use in the Community
Time-series design measure change measure baseline twice (for example è implement intervention at the beginning of at one + six month the intervention intervals) after intervention and ask questions to find out why people changed behaviour Pre-post design measure change measure baseline è after intervention at the beginning of implement intervention and ask questions the intervention to find out why people changed behaviour Four evaluation designs Type of design Action Measuringchange Monitoring and evaluation Promoting Rational Drug Use in the Community
Define key outcome measures • Review communication objectives • Identify what behaviours are likely to change • Limit the number of outcome measures • But, measure more than one dimension • Choose outcome measures that can be clearly defined and reliably measured Monitoring and evaluation Promoting Rational Drug Use in the Community
Examples of key outcome measures • The percentage of childhood diarrhea cases treated with antidiarrhoeal medicines • The percentage of total antimalarial sales which included an adequate dosage Monitoring and evaluation Promoting Rational Drug Use in the Community
Methods • Review of project documents, including monitoring reports • Semi-structured interviews • Short quantitative surveys • Focused weekly illness recalls (change in common health problems) • Structured observations Monitoring and evaluation Promoting Rational Drug Use in the Community
Problems in proving effects • Contamination • Intervention changes over time • Difficult to measure mix of methods • Unplanned interventions by others • Confounding factors Monitoring and evaluation Promoting Rational Drug Use in the Community
Summary guidelines • Decide at beginning of an intervention how you will evaluate • Prepare good outcome measures • Evaluate process and effect • Look for short- and long-term change • Encourage participation of target audience • Share successes and failures • Make an evaluation plan Monitoring and evaluation Promoting Rational Drug Use in the Community
An evaluation plan • Statement of communication objectives • Evaluation questions • Key outcome measures • Evaluation design/methodology • Data collection methods • Plan for data processing and analysis • Plan for dissemination of results Monitoring and evaluation Promoting Rational Drug Use in the Community
will depend on the complexity of behaviour, its reinforcement in culture,the presence/absence of opposing forces and the resources at your disposal Short term knowledge,skills, awareness What can be achieved? Medium term 1-3 yrs behaviour change Longer term <5 yrs Change in health Monitoring and evaluation Promoting Rational Drug Use in the Community
Activity 1: Evaluate a shopkeeper intervention/Uganda • Read the description, page 16 • Formulate: Evaluation questions • Select key outcome measures • Advise on study design: quasi-experimental or time-series design? • Propose limited set of data-collection methods Monitoring and evaluation Promoting Rational Drug Use in the Community