Finding and using evidence about local conditions

1. Finding and using evidence about local conditions • What local evidence is needed? • How can it be found? • How should its quality be assessed?

2. What is ‘local evidence’? • Evidence that is available from the specific setting or settings in which a policy decision and action will be taken • ‘Local’ can refer to district, regional or national levels, depending on the nature of the policy issue being considered

3. Why is local evidence needed? • Evidence about local evidence is always needed, alongside other evidence, to inform health policy decisions • Global evidence – the best evidence from around the world – is the best starting point for judgements about the effects of programmes and factors that modify those effects, and for insights into ways to approach and address problems • Ideally from systematic reviews • Local evidence is needed for most other judgements about what decisions and actions should be taken

4. The role of local evidence

5. What local evidence is needed? Local evidence may be needed to • Clarify the problem • Contribute to assessments of options for addressing the problem • Assess barriers to implementing options • Contribute to assessments of implementation strategies

6. Ways in which local evidence might be used

7. Questions or comments about what local evidence is needed?

8. How can local evidence be found? Sources of local evidence include • Routine health information systems • mortality and burden of disease • health service coverage • health care expenditure • Survey data • household conditions • health • demographics • Studies • trials conducted locally • studies of consumers’ views regarding a particular health issue • cost-effectiveness evaluations

9. Searching for local evidence • The process of searching for, including, and assessing local evidence should be transparent and systematic • The selective use of local evidence should be avoided – this may result in important data being omitted during the decision making process

10. Starting points for finding routinely collected data • Health information departments of ministries of health • National statistics offices • Local health authorities • Local research institutions • Non-governmental organisations (NGOs) • Bilateral or multi-lateral agencies, such as WHO country offices • Commercial databases (e.g. for local prices for drugs or for drug availability)

11. International databases • WHO Statistical Information System (WHOSIS)www.who.int/whosis/whostat/2010/en/index.html • World Bank Data data.worldbank.org • HRH Global Resource Center www.hrhresourcecenter.org

12. Locating relevant studies • PubMed • Can limit searches to a specific country • Can use hedges (validated search strategies) to search for administrative databases studies, community surveys and qualitative studies • www.nlm.nih.gov/nichsr/hedges/search.html • Google Scholar • Many local studies, such as operational research on health services, are published as reports on the web and may not be published in journals • WHO Library Information System • dosei.who.int/uhtbin/cgisirsi/Mon+May++4+21:00:46+MEST+2009/0/49 • Personal contact with local researchers, including unpublished study reports • Health networks or observatories such as • EQUINET Africa • www.equinetafrica.org • Africa Health Workforce Observatory • www.afro.who.int/hrh-observatory

13. Questions or comments about how to find local evidence?

14. How should the quality of local evidence be assessed? • Like other forms of evidence, the quality of local evidence needs to be assessed • Where data quality is poor, interpretation can be difficult and there is a danger that faulty conclusions may be drawn

15. Key questions for assessing the quality of local evidence • Is the evidence representative? • Does the evidence correctly represent the wider population from which it is drawn or to which the findings are generalised? • Is the evidence accurate? • Do the available data match, or are they likely to match, the actual value of the outcome measured? • Are appropriate outcomes reported? • Are the measures reported in the data suitable for addressing the question for which the data will be used?

16. Limitations of local evidence • Need to be cautious about using local evidence alone to assess the likely impacts of policy options • Local evidence may be more directly relevant than studies conducted elsewhere • But may be less reliable due to important limitations in studies that were done locally • Limitations include risk of bias and imprecise results due to the small size of the studies

17. Questions or comments about assessing the quality of local evidence?