Evaluation and Documentation of Innovations in Public S ervices

1. Evaluation and Documentation of Innovations in Public Services MeenakshiGautham, PhD Research Fellow & IDEAS Country Coordinator – India London School of Hygiene and Tropical Medicine, UK Improving health worldwide ideas.lshtm.ac.uk

2. A public service innovation can be…. A PRODUCT… • Drugs /diagnostics • Technology • Seeds and crops • A PROCEDURE… • Training or educational strategy • Therapy • Waste disposal • A SYSTEM... • Home births to institutional deliveries • New markets for traditional crafts • Privatisation of a public service

3. Definition • The replacement of a previous method with a new approach that provides SUSTAINABLE and COMPETITIVE benefit

4. Why is evaluation important? • Benefit of the innovation is uncertain; to fund or not to fund? • Uncertainty can lead to two risks: - services of limited benefit are financed with public money OR - access to much more beneficial services is denied or not explored • Good evaluation can minimise both risks

5. Steps in evaluation of a public service innovation STEP 1: Test feasibility and usability of the innovation Examples: ease of application, lower cost, extensive participation and acceptance, no immediate side effects STEP 2: Confirmation of the benefits with surrogate parameters Examples: Reduction in tumour size, increased frequency of interactions, increase in enrolment, increase in standardization of healthcare STEP 3: Confirmation of the desired end point benefits Examples: Improves quality of life, improves survival, changes behaviour, sustainable improvements in quality and access to services

6. Things to keep in mind • Underlying theory of change: how will the innovation work to bring about the desired result ? • Impact of context: Innovation + context = outcome • Pay attention to the process as well as the outcomes • Link evaluation with continuous feedback

7. An example from IDEAS • Do innovations in maternal and newborn services lead to improved maternal and newborn survival? -excerpts from a presentation prepared for NRHM senior officials

8. Learning questions to be addressed: In areas where Gates grantees are working: • Do innovations lead to enhanced interactions between families and frontline workers, and does the coverage of critical interventions/best practices increase as a result? • Are the innovations cost-effective? • How do enhanced interactions affect coverage of critical interventions?

9. Theory of change Credit: Dr BilalAvan Credit: Bill & Melinda Gates Foundation Innovation introduced in grantee key districts SCALE UP adoption of innovations beyond grantees’ areas Innovation scaled-up

10. Theory of change More: Better: Equitable: Cost-effective: Num. interactions between FLWs and individual women increase; More women have FLW interactions FLW and facilities properly equipped to provide good quality care; FLWs are well informed about the components of good quality care; FLWs take appropriate actions during interactions; Women know more about danger signs; Women improve their care seeking Improvements happen for all women irrespective of socio-economic status The innovations that are implemented are cost-effective Credit: Dr BilalAvan Credit: Bill & Melinda Gates Foundation Innovation introduced in grantee key districts SCALE UP adoption of innovations beyond grantees’ areas Innovation scaled-up

11. More pregnancy interactions: Women with a birth in last 12 months

12. More + equitable pregnancy interactions: % with ≥4 pregnancy care interactions by household SES

13. Better pregnancy interactions: Women with a birth in last 12 months

14. Coverage of pregnancy interventions: Women with a birth in last 12 months

15. More intra-partum interactions: Women with a birth in last 12 months *N advised to seek extra care: Intervention=13; Comparison=19

16. Coverage of post-natal interventions: Women with a birth in last 12 months *Don’t know responses excluded

17. Summary Overall: Frequency of interactions, institutional births increased, but quality is lacking PREGNANCY • Almost all pregnancy care delivered in facilities, not at home • Three –quarters of women had at least 1 pregnancy care interaction • One-third had at least 4 interactions – predominantly the least poor • Frontline Worker knowledge of the core elements of focussed pregnancy care was low and just 7% of women received all elements • Approximately three-quarters of women had TT vaccine protection, one half had iron prophylaxis POST-NATAL • Delayed bathing not being practiced in government facilities

18. Documenting for advocacy Policy discussion paper Newspaper article Web based social media – blogs etc Scientific papers

19. A thinking exercise • Think of an innovation in public services • Think of what ultimate public benefit it will bring about and how • Discuss with your partner : 1 or 2 indicators for Step 1, Step 2 and Step 3 • Any two types of documentations that you could develop related to this innovation

20. Thank You! IDEAS Project London School of Hygiene & Tropical Medicine Keppel Street, London WC1E 7HT In India: c/o Public Health Foundation of India Plot 47, Sector 44, Gurgaon Institutional Area Email: Meenakshi.gautham@lshtm.ac.uk Website: ideas.lshtm.ac.uk

21. Contact us IDEAS Project London School of Hygiene & Tropical Medicine 4th floor Keppel Street London WC1E 7HT Email: ideas@lshtm.ac.uk Website: ideas.lshtm.ac.uk Newsletter sign up: eepurl.com/j3iBz Twitter: @LSHTM_IDEAS