Monitoring, supervision and quality assurance

1. Monitoring, supervision and quality assurance Integrated Disease Surveillance Programme (IDSP) district surveillance officers (DSO) course

2. Preliminary questions to the group • Were you already involved in monitoring and supervision of surveillance? • If yes, what difficulties did you face? • What would you like to learn about monitoring and supervision of surveillance? 2

3. Outline of the session • Monitoring indicators • Supportive supervision • Quality improvement 3

4. Monitoring • Under the responsibility of the district surveillance office • Based upon standardized indicators reviewed on a regular basis Monitoring 4

5. Surveillance activities to be monitored • Collection and compilation of data • Laboratory • Analysis and interpretation • Follow-up action • Feedback Monitoring 5

6. Weekly indicators of collection and compilation of data • Assess • Timeliness of reports • Completeness of reports • Indicators • Number of reporting units providing regular/timely/complete reports • Number of sentinel private practitioners providing regular/timely/complete reports Monitoring 6

7. Creative monitoring solutions, Haryana, 2007 • District register keeping track of reports • Reporting units that are not timely are flagged with a highlighter Monitoring 7

8. W EEKS 1 2 3 4 5 6 7 8 9 10 Baripa d a (DHH) B.Pahad CHC B.po s i Bahal d a CHC Baras a hi CHC Betnat i CHC Bijat a la PHC Dukura CHC G. m ahisa n i PHC Ja m da PHC Ja s hipu r CHC K.tand i UHPHC Kapti p ada CHC Karan j ia, SDH KC Pur PHC Khunta PHC Kosth a CHC Kulia n a PHC M anada CHC Ranga m at i a PHC Raruan PHC RG PHC RR Pur , SDH SC Pur PHC Shirs a CHC Sukru l i PHC T . m unda CHC T ato PHC T irin g PHC Udala, SDH Colour-coded monitoring of reporting, Mayurbhanj district, Orissa, India, 2004 The colour coding stimulates reporting units that are not performing Monitoring 8

9. Proportion of complete surveillance reports, Dhenkanal district, Orissa, India, 2002 100 90 80 70 60 Completeness (%) 50 40 30 20 10 0 W1 W3 W5 W7 W9 W11 W13 W15 W17 W19 W21 W23 W25 W27 W29 W31 W33 W35 W37 W39 W41 W43 W45 W47 W49 W51 Monitoring 9 Weeks

10. Proportion of timely surveillance reports, Dhenkanal district, Orissa, India, 2002 100 90 80 70 60 Percentage of timeliness 50 40 30 20 10 0 W1 W3 W5 W7 W9 W11 W17 W19 W21 W29 W31 W33 W41 W43 W45 W13 W15 W23 W25 W27 W35 W37 W39 W47 W49 W51 Weeks Monitoring 10

11. Proportion of reports from district to state in Integrated Disease Surveillance Programme (IDSP) phase 1 states, India, 2006-7 100% 90% 80% 70% 60% % of districts reporting 50% 40% 30% 20% 10% 0% Week 48 Week 52 Week 4 Week 8 Week 12 Weeks 11 In Time (Within Week) Late Reporting Not Reporting

12. Proportion of reports from district to state in Integrated Disease Surveillance Programme (IDSP) phase 2 states, India, 2006-7 100% 90% 80% 70% 60% % of districts reporting 50% 40% 30% 20% 10% 0% Week 48 Week 52 Week 4 Week 8 Week 12 Weeks In Time (Within Week) Late Reporting Not Reporting 12

13. Reported varicella and typhoid cases (log scale), Darjeeling, West Bengal, India, 2000-4 Interpretation: The parallel increase between varicella (that should be constant) and typhoid suggests that increasing rates of typhoid are secondary to improved reporting 13 CDC for TPP

14. Laboratory performance indicators • Proportion of laboratory specimens received in good condition • Proportion of laboratory specimens received with completed form • Proportion of results reported within seven days after receipt of specimens Monitoring 14

15. Analysis and interpretation of data • Number of outbreaks detected by the system • Production of analysis reports Monitoring 15

16. Number of diarrhea outbreaks detected before and after a new surveillance system, Dhenkanal district, Orissa, 1997-2002 60 Orissa multi-disease surveillance system 50 40 Number of outbreaks 30 20 10 0 1997 1998 1999 2000 2001 2002 Monitoring 16 Years

17. Follow-up action indicators • Proportion of outbreaks detected within one incubation period • Proportion of outbreaks that have been investigated • Proportion of outbreaks investigated within 48 hours of detection Monitoring 17

18. Feedback indicators • Number of monthly meetings at community health centre with surveillance staff, over last 6 months • Number of village health committee meetings attended by primary health centre staff every 3 months • Number of newsletters published Monitoring 18

19. Input indicators • Percentage of positions filled • Percentage of staff at each level trained • Percentage of districts with functional rapid response team • Percentage of districts with functional laboratories Monitoring 19

20. Proportion of vacancies among commmunity health workers, Dhenkanal, Orissa, India, 2002 100% 90% 80% Vacant 70% In position 60% Proportion (%) 50% 40% 30% 20% 10% 0% Health workers, female Health workers, males Monitoring 20

21. Prerequisites for supervision • Job description • Should clearly describe the surveillance activity to be performed by each category of health staff • Resources • Required by supervisory team to perform this activity • Attitude • Supervision should not be a fault finding mission Supervision 21

22. Preparing a supervision visit • Review the previous supervisory visit reports • Prepare supervisory plan • Visit each reporting unit at least quarterly • Make a check list to review activities Supervision 22

23. The supervision visit • Activities during the visit • Use checklist • Observe • Review records • Conduct focus group discussions with staff • Provide feedback • Underline achievements • Mention opportunities for improvement • Recommend actions with a time frame Supervision 23

24. Potential activities to improve the quality of surveillance (1/2) • Ensure that necessary resources and incentives identified in the programme have been provided to the functionaries • Identify the right person for the right task and change work responsibility to suit the individual • Conduct site visits to identify and supervise the activities of the functionaries • Identify weakness and help with additional training of personnel on site as required Quality 24

25. Potential activities to improve the quality of surveillance (2/2) • Keep personnel contact to understand problems associated with functionary • Discuss persisting problems to identify solution at the district surveillance monthly meetings and take appropriate actions specified by the group • Give written instructions on the solutions suggested by the group to the functionary Quality 25

26. Data quality issues • Data entry errors • Missing values • Attraction to round figures • Bias related to lack of representativity • Cases more severe • Urban > rural • Source not represented (private sector, GPs) 26

27. Notifications x 1000 14 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Day of the month Data quality signs: Frequency distribution of dates of onset for all notifiable diseases, USA, 1989 Peaks at “easy to remember ” datessuggesting artifacts 27

28. Central agencies responsible for quality control • Central surveillance office • National Institute of Communicable Diseases (NICD) • Indian Council of Medical Research (ICMR) Quality 28

29. Other agencies identified by the Ministry of Health and Family Welfare for external evaluation • World Bank • World Health Organization • United States Centers for Diseases Control and prevention (CDC) • INDIACLEN Quality 29

30. Take home messages • Follow monitoring indicators • Supervise regularly and supportively • Look at all opportunities to improve quality 30

31. Additional reading • Section 4 of IDSP operations manual (Report 1, page 56-7) • Section 9 of IDSP operations manual • Module 11 of training manual 31