Promoting Rational Drug Use in the Community

1. Promoting Rational Drug Use in the Community Monitoring and evaluation

2. Objectives Session on Monitoring and Evaluation • Understand functions of monitoring and evaluation • Define what should be monitored • Formulate evaluation questions • Identify methods • Make a plan for monitoring and evaluation Monitoring and evaluation Promoting Rational Drug Use in the Community

3. Place of Monitoring and Evaluation in the Planning Cycle Problem Goal Objective Activity Monitoring Evaluation Monitoring Refers to Activities Evaluation refers to Objectives Monitoring and evaluation Promoting Rational Drug Use in the Community

4. Functions of monitoring • See that everything goes according to plan • Find out if there are unexpected difficulties • Adjust plans Monitoring and evaluation Promoting Rational Drug Use in the Community

5. Functions of Evaluation • Prove that the intervention worked • Assess if costs were reasonable • Convince others that intervention works • Share experiences - to allow replication Monitoring and evaluation Promoting Rational Drug Use in the Community

6. Indicators • Indicators are needed for both monitoring and evaluation • Monitoring requires process indicators • Evaluation requires outcome indicators • Planning to collect indicators is an important part of any plan Monitoring and evaluation Promoting Rational Drug Use in the Community

7. What to monitor • Formative research(RAP) done? • Intervention pre-tested? • Activities implemented according to plan? • Are costs in accordance with budget? • Are staff carrying out assigned duties? • Is collaboration with others taking place? Monitoring and evaluation Promoting Rational Drug Use in the Community

8. How to monitor • Record keeping • Making reports on activities • Field or supervisory visits • Regular project/programme meetings Monitoring and evaluation Promoting Rational Drug Use in the Community

9. What to evaluate? • Preparation • Planning • Implementation • Effect Monitoring and evaluation Promoting Rational Drug Use in the Community

10. Preparation • How was the problem identified? • Was formative research done • Who is involved in the intervention • How comprehensive is the intervention (consider the factors which cause the problem) Monitoring and evaluation Promoting Rational Drug Use in the Community

11. Planning • What objectives were set? • Which activities were planned? • What target audiences were identified? • Were the interventions pre-tested? • Was a plan for monitoring and evaluation made? Monitoring and evaluation Promoting Rational Drug Use in the Community

12. Implementation • Which of planned activities were carried out? • What messages were disseminated? • How many people did the message reach? • Did the target audience pay attention? • Did the target audience understand? • What problems were encountered Monitoring and evaluation Promoting Rational Drug Use in the Community

13. Effect • Change in knowledge? • Change in behaviour? • Change in health service quality? • Change in policy? • Improvements in health? • Negative or unexpected effects? Monitoring and evaluation Promoting Rational Drug Use in the Community

14. Four evaluation designs • Randomised control design • Quasi experimental design • Time-series design • Pre-post design Monitoring and evaluation Promoting Rational Drug Use in the Community

15. Randomised control design intervention group receives education random assignment of è intervention and measure change control group; after intervention at the beginning è of the intervention control group receives no education Quasi-experimental design intervention group receives education specifically selected è intervention and measure change control group; after intervention at the beginning è of the intervention control group receives no education Four evaluation designs Type of design Action Measuringchange Monitoring and evaluation Promoting Rational Drug Use in the Community

16. Time-series design measure change measure baseline twice (for example è implement intervention at the beginning of at one + six month the intervention intervals) after intervention and ask questions to find out why people changed behaviour Pre-post design measure change measure baseline è after intervention at the beginning of implement intervention and ask questions the intervention to find out why people changed behaviour Four evaluation designs Type of design Action Measuringchange Monitoring and evaluation Promoting Rational Drug Use in the Community

17. Define key outcome measures • Review communication objectives • Identify what behaviours are likely to change • Limit the number of outcome measures • But, measure more than one dimension • Choose outcome measures that can be clearly defined and reliably measured Monitoring and evaluation Promoting Rational Drug Use in the Community

18. Examples of key outcome measures • The percentage of childhood diarrhea cases treated with antidiarrhoeal medicines • The percentage of total antimalarial sales which included an adequate dosage Monitoring and evaluation Promoting Rational Drug Use in the Community

19. Methods • Review of project documents, including monitoring reports • Semi-structured interviews • Short quantitative surveys • Focused weekly illness recalls (change in common health problems) • Structured observations Monitoring and evaluation Promoting Rational Drug Use in the Community

20. Problems in proving effects • Contamination • Intervention changes over time • Difficult to measure mix of methods • Unplanned interventions by others • Confounding factors Monitoring and evaluation Promoting Rational Drug Use in the Community

21. Summary guidelines • Decide at beginning of an intervention how you will evaluate • Prepare good outcome measures • Evaluate process and effect • Look for short- and long-term change • Encourage participation of target audience • Share successes and failures • Make an evaluation plan Monitoring and evaluation Promoting Rational Drug Use in the Community

22. An evaluation plan • Statement of communication objectives • Evaluation questions • Key outcome measures • Evaluation design/methodology • Data collection methods • Plan for data processing and analysis • Plan for dissemination of results Monitoring and evaluation Promoting Rational Drug Use in the Community

23. will depend on the complexity of behaviour, its reinforcement in culture,the presence/absence of opposing forces and the resources at your disposal Short term knowledge,skills, awareness What can be achieved? Medium term 1-3 yrs behaviour change Longer term <5 yrs Change in health Monitoring and evaluation Promoting Rational Drug Use in the Community

24. Activity 1: Evaluate a shopkeeper intervention/Uganda • Read the description, page 16 • Formulate: Evaluation questions • Select key outcome measures • Advise on study design: quasi-experimental or time-series design? • Propose limited set of data-collection methods Monitoring and evaluation Promoting Rational Drug Use in the Community