KMC Workshop Group E Monitoring and Evaluation

1. KMC WorkshopGroup EMonitoring and Evaluation

2. Clarification of Concepts • Monitoring: vigilance of a process • Evaluation: assessment, value judgment about a process and its results. • The group decided that it was going to focus on monitoring, rather than on the specifics of evaluation

3. Clarification of Concepts • Monitoring in KMC can have several meanings: • Vigilance of the implementation of KMC at different levels • Global (WHO) • Regional • National • Local, etc. • Vigilance of an ongoing program, again: • Global • Regional, etc.

4. Clarification of Concepts • Prior to defining what to monitor during implementation, goals and plans for implementation should be stated. • Main purposes of monitoring then, would be: • Surveillance of the compliance with implementation tasks and steps, timetable, etc. • Quantification of performance (e.g. number of trained health workers, etc.) • Quantification of achieved goals (e.g. mortality reduction)

5. Clarification of Concepts • Monitoring of an ongoing program can also have several purposes and objectives • The group identified monitoring a as a health care quality assurance tool as a very important purpose. • Most of the following discussion was centered then on “evidence-based quality assurance” for KMC programs • The importance of monitoring for implementation was nevertheless acknowledged as well as the need for address it a next step

6. Objectives • To identify elements and domains relevant for developing appropriate monitoring tools for quality assessment-improvement (quality assurance) of an ongoing KMC program. • To generate a series of statements which can help to define good practices for monitoring KMC programs • To develop the process, a hypothetical program for delivering KMC in a health facility was sometimes used. • Extrapolation to other settings and to broader scopes can be made.

7. The EB-Quality improvement cycle • Evaluate current practice, identify problems • Separation of mother and infant after birth • Low breast feeding rates • Undesired variability in practices and/or in outcomes • Plan and implement interventions • Set quality standards for practice (e.g. clinical practice guidelines, evidence-based) • Set quantitative goals • Implement practices • Monitor • Compliance with requirements • Performance • Compliance with guidelines • Frequency of selected outcomes • Adjust performance according to monitoring • Evaluate results (close the cycle, go back to 1st step) and start all over

8. Delivering KMC implies • A KMC program: • Resources, administration, planning, management, put together to deliver • The KMC intervention • Set of specific processes (interventions ) for caring for the health care of newborn infants involving and empowering their mothers-families • Using a specific method or technique • The KMC method, a complex non-pharmacological intervention clearly standardized, defined and supported by scientific evidence. • The method is defined as EB recommendations, usually in the form of structured detailed protocols.

9. Components of a KMC program as related to health care quality • Structure • Processes • Outcomes (Donabedian)

10. Structure • Stable part of the Health care system, that provides the support and setting in which health KMC is delivered: • Physical structure • Administrative structure and processess • Norms • Resources • Technical • Human • Capital

11. Process • Actual specific health care interventions and procedures employed for providing care • Delivered interventions should be backed by scientific evidence supporting that they do more good than harm • They should be feasible: • Available • Affordable • Proficiently performed • Properly trained personal • Technically appropriate equipment • Acceptable for • Target population and • Health care personnel • Ethically appropriate

12. Outcomes • Changes in different aspects of health associated with previously delivered interventions • Disease (condition) centered • Mortality • Morbidity, complications and sequels • Time to event • Patient centered • Growth and development • Satisfaction • Health related quality of life

13. Monitoring KMC programs for Quality assurance • Structure: • Standards should be set • Rate of nurses to patients • Locative facilities for mothers to provide kangaroo position • Etc. • Verification that minimum acceptable standards are met (Basic part of certification? Accreditation?) • Process • Recommendations (evidence-based guidelines) have identified processes known to do more good than harm • Compliance with recommendations is monitored • Outcomes • Given that there should be evidence showing that recommended processes do more good than harm not every specific health outcome needs to be monitored • Nevertheless, given that despite having evidence-based recommendations, thing can go wrong (poor performance, other quality issues) some important and selected health outcomes should be monitored.

14. Proposed framework to identify what to monitor

15. An example

16. An example (cont.)

17. Examples of specific indicators • Performance indicators of a KMC program • LBW Infants in KP first visit to the clinic/total LBW infants discharged from hospital • Minimum acceptable 90% • Optimal 95% (there are infants discharged after leaving KP) • Measures compliance with KP between discharge and first visit to out KMC clinic • Hours a day a parent can stay at the NICU • Minimum acceptable 12 /24 • Optimal 24/24 • Measures appropriateness of structure: • Regulations and norms to remove access barriers • Amenities, comfort, space to allow parents to stay

18. Examples of specific indicators: the “SKIND” score

19. Recommendations • Setting Standards for structure and making sound evidence-based recommendations which guide processes is indispensable prior to proposing any reasonable plan for monitoring • Identification of “key indicators” amenable for “universal” use can be attempted afterwards • A group within INK should work on a guide for developing good monitoring practices for KMC quality assurance • An inventory of available instruments for monitoring and assessing implementation and for quality assurance on ongoing KMC should be assembled • The group already identified several research and monitoring instruments both general and specific • Instruments for monitoring the progress of Implementation • Instruments for monitoring quality of care