Background

1. Background • Identifying reliable ways of assessing how evidence informed policy making processes are (esp. in parliaments) • Identifying barriers to provision of evidence and working out how can they be overcome.

2. How evidence informed is parliamentary scrutiny? • Surveys and focus groups with MPs and staff in the Parliament of Uganda • Analysis of outputs • Internal policy briefings • Parliamentary debates

3. Policy briefing analysis • 17 internal briefing papers produced by Ugandan parliamentary staff for members • Sent to external experts who were asked to rate: • Use of evidence • Accuracy • Completeness • Clarity • Objectivity

4. Findings • Wide variation in standard, from “sounding objective and impartial” and “fairly well referenced” through to “biased, poorly referenced, badly written and containing factual inaccuracies”

5. Recurring themes • Missing out key sources of evidence – particularly from authoritative international sources e.g. WHO, IPCC; • Omission of key evidence which then skews the conclusions of a briefing; • A tendency to express personal views and draw conclusions not backed up by evidence; • A need to improve readability of briefings to make them more accessible to policymakers.

6. Authors surprised by the results; • Followed up with training in information literacy and summarising skills; • Plan to repeat exercise.

7. Hansard debates • What is the nature of the ‘evidence’ cited; • How factually accurate is this evidence; • Are both sides of the debate represented; • Are any key areas omitted?

8. Debate centres around anecdote • “Every time I travel home, [. . .] I would come back with malaria [. . .] but since the spraying of DDT in my place, I have been there now three times [. . .] but I have not come back with malaria.” • “I am talking from experience. I have men and women who have been stationed at high voltage substations where EM [electromagnetic radiation] is the highest. They do not have cancer.”

9. Factual inaccuracies • “There is no evidence that DDT causes toxic effects.” • “80 per cent of Ugandan women suffer from cervical cancer.” • Selective use of “evidence” • “ A nutrition expert has found that taking tea in poly[thene] cups can be a cause of ulcers and cancer.”

10. Scope and balance • “…the people who carried the day did not get down to look at the scientific issues… They were carried away by the one sided research.” • “the debate is far from comprehensive. Many important aspects of a balanced and evidence based discussion of the issues at hand are not addressed”

11. Lack of focus • “Frequently, Members are not clear about what the debate is about … The debate is not very focussed and Members frequently digress to talking about irrelevant matters”

12. Surveys and focus groups show willingness and aspiration for policy making to be more evidence informed, • But does not translate into action because of a number of underlying factors (for both staff and MPs).

13. Underlying factors • Availability of information (not just research evidence.. order papers etc); • In-country linkages; • Literacy levels; • Understanding of what constitutes “evidence”; • Understanding of objectivity; • Institutional processes; • (Internal) dynamics – e.g. Who turns up? Who speaks in debate? • Resources (e.g. number and skill of available staff).

14. Key messages • Need to develop objective (quantitative?) methods of assessing how evidence informed policy making processes are. • Need to look at wider context which affects the demand and uptake of evidence, • Need to think about when in the process should the evidence be provided (in what form, from whom, to whom),