Developing a National Framework for Monitoring the Iraqi NDP MEASURING GOVERNANCE

1. Developing a National Framework for Monitoring the Iraqi NDP MEASURING GOVERNANCE Marie Laberge Global Programme on Country-Led Governance Assessments UNDP Oslo Governance Centre

2. Types of indicators & data sources • Examine the limitations of global composite indicators • Discuss the rationale for using complementary indicators (input vs. Process vs. outcome – de jure vs. de facto) • Identify indicators that are ‘actionable’ • Identify 4 ways in which indicators can be made pro-poor and gender sensitive • Identify common sources of governance data

3. Part One International sources of governance indicators

4. 1974 76 78 80 84 86 88 90 92 94 96 98 00 02 04 06 08 Global composite indicators of governance Global Accountability Report Index of Democracy Democracy Index World Governance Assessment World Democracy Audit Human Rights Indicators Governance and Democracy Processes Gender Empowerment Measure State Failure Dataset Weberian Comparative State Project Institutional Profiles Database Women in Parliament Press Freedom Index Index of Economic Freedom Countries at the Crossroads BEEPS CIRI Human Rights Databse Political Terror Scale Commitment to Development Failed States Index Bertelsmann Transformation Index Global Competitiveness Index Open Budget Index Journalists killed Governance Matters Opacity Index Polity International Country Risk Guide CPIA World Values Survey Integrity Index Corruption Perceptions Index Press Freedom Survey Bribe Payers Index Freedom in the World Global Peace Index GAPS in Workers’ Rights 82

6. How to explain these discrepancies in rankings? • 1) What is each index measuring? • TI CPI  Public sector only • WB ‘Control of Corruption’ indicator  public & private sectors • Global Integrity Index  anti-corruption mechanisms • 2) What types of indicators make up each index? • High-level corruption vs. petty corruption • Frequency of bribes vs. size of bribes

7. 3) What data sources are used?

8. And… 4) Are data sources the same every year?

10. Part two National governance indicators

11. Monitoring the input, process and outcome sides of governance reforms

12. Three levels of indicators • Outcome indicators will tell us: • What do we want to achieve? • Have we achieved it? • Process indicators will tell us: • How are inputs used to generate outcomes? • What are the ‘best’ processes? • When & how do processes need adjustment? • Input indicators will tell us: • What is needed to create these processes? • Whether it is available?

13. Complementarity in the use of indicators • De jure (input) indicator: In law, is there an agency with a legal mandate to address corruption? Does it receive regular funding, a professional & full-time staff? • Measuring a change in law, in procedures, in resources • De facto (process / outcome) indicator: In practice, is the anti-corruption agency effective? • Measuring improved governance in practice (how are citizens benefiting from this new institution?)

14. Combining indicators to show discrepancies • De jure indicator: • In law, is there an agency with a legal mandate to address corruption? (input) • De factoindicators: • In practice, is the anti-corruption agency effective? (outcome) • When necessary, is the ACA able to independently initiate investigations?(process) • Does the ACA make regular public reports (e.g. to the parliament)? (process) • Can citizens complain to the ACA without fear of recrimination? (e.g. whistle-blower mechanisms) (process) • Does the ACA act on citizen complaints within a reasonable time period? (process)

15. Input / process indicators are more ‘actionable’ • Outcome indicators are useful to assess progress towards the desired objectives of governance programmes (new laws / increased expenditures mean nothing in and of themselves) • But they lack ‘actionability’ (they don’t tell us what needs to be fixed / points of interventions) • Example: A government cannot ‘choose’ to lower a crime rate (an outcome indicator) • It can, however, choose to put more police on the streets (input) or toughen penalties for offenders (prosecution process)

16. Actionable indicators: Experience from Morocco

17. A word of caution: “Not everything that counts can be counted, and not everything that can be counted counts.” – Albert Einstein

18. A word of caution: “Not everything that counts can be counted, and not everything that can be counted counts.” – Albert Einstein • Measuring is not an end in itself, but rather a means to an end (actual governance reforms). • So what we measure should help us implement reforms – if not, why measure it?

19. A word of caution • Risk of measuring things because they are easily measurable, leading to ‘reform illusion’ • Example: Measuring the number of corruption cases brought to trial (as an indicator of the efficiency of the judicial system in combating corruption): • Does an increase in this indicator mean an increased level of confidence in the reporting mechanism, and in the courts? • Or rather, does it indicate a higher incidence of corruption? • Or both...? •  In other words, are you really measuring what you intended to measure?

20. Do you have a balanced basket of indicators that measures progress towards a single aim? Is the ambiguity inherent in each indicator reduced by the presence of the others? Balanced baskets of indicators

21. What is the aim you want to achieve? What are the many reasons why your aim might NOT be achieved? For each “reason”  develop one indicator Balanced baskets of indicators

22. Unbalanced baskets of indicators Aim: “Equal access to justice” Indicator 1: Number of new courts opened in rural and urban areas Indicator 2: Number of courts per 100,000 residents Indicator 3: % citizens who say that they have access to court systems to resolve disputes Indicator 4: % accused not represented at trial

24. Balanced baskets of indicators • Checklist for building a well-balanced • basket of indicators • Avoid narrow activity indicators • What is the strategic aim you want to achieve? • Draw one indicator from each institution/service that contributes to the overall aim • Or... • What are the many reasons why your aim might NOT be achieved? • For each “reason”  develop one indicator

25. Balanced baskets of indicators 4 key services/institutions for achieving “equal access to justice” 4 indicators: 1. Court system 2. Legal aid service 3. Police service 4. Prosecution service

27. How to make governance indicators ‘pro-poor’ and gender sensitive

28. What makes a governance indicator ‘pro-poor’ or gender sensitive? • 4 ways to make indicators sensitive • to vulnerable groups: • Disaggregating by poverty/gender • Specific to the poor/women • Implicitly poverty/gender sensitive • Chosen by the poor/women

29. 1. Disaggregating by poverty/gender (Parliament) •  Information is collected for the general population, then disaggregated by sex or income • % seats held by women in parliament • Level of confidence among female citizens that the Parliament represents their interests • % of Parliamentarians from poorer districts • that have functioning and accessible local • offices to meet with constituents

30. 2. Specific to the poor/women • Measuring governance issues that are specific to the poor or to women • Size of funds allocated to legal aid in provincial budgets (per capita) • Number of attorneys as % of citizens in need of one • % of local governments practicing gender-sensitive budgeting

31. 3. Implicitly poverty/gender sensitive • Backlog of small cases of little financial value •  Makes no explicit reference to poverty status or gender, but by its nature, clear that the indicator is of particular relevance to low-income groups / women • The number of hours that polling booths are open during on election day • Frequency of engagement of CSOs in consultations on the legislation-making process

32. 4. Chosen by the poor/women •  Identified & measured by using participatory techniques (surveys, focus groups, etc.) • Acceptance of documentation other than birth certificates in the process of voter registration • Women’s trust in the police and its ability to provide women with redress if they file a complaint

33. Sources of governance data

34. Complementarity of data sources • Important to monitor both the ‘supply’ and ‘demand’ for good governance • Parallelwithprivatesector: • Assessment of the company’s charter, balance sheet, internalprocesses and assembly line OR / AND ? • Assessment of whetherconsumers are actuallybuyingtheirproduct, whethertheyweresatisfiedwithit, and likely to keep on buying it

35. Implications for data sources? 1) Objective data – Qualitative: Administrative sources: Policy and legal documents, codes of conduct, organizational set-up and management systems, processes for decision-making Other narrative reports: Newspaper reports, reports by CSOs 2) Objective data – Quantitative: Government statistics, data gathered by domestic NGOs, international organizations and academics, expenditure tracking and budgetary information, audit reports, court records 3) Opinion and experience-based data from citizens, specific groups (MPs), specific institutions, or private sector through surveys, focus groups, etc.

38. Opportunity to draw from ‘fresh’ survey data • New ‘Arab Democracy Barometer’ • Joint initiative by Arab Reform Initiative & Arab Barometer • Nationally representative public survey • In 10 countries – including Iraq • Data to be collected by country teams (starting autumn 2010) • 75 questions

39. The Arab Democracy Index • First governance index produced by an Arab institution (“The Arab Reform Initiative”) • Data collected by consortium of Arab research centres & universities • Covers 10 countries • 40 indicators • Some measure the ‘means’ of democratic transition (legislation) – ‘de jure’ indicators • Some measure the ‘results’of democratic transition – ‘de facto’ indicators • Sources of data: • Government & non-government • Citizen’s impressions (public survey) used for 25% indicators • 4 themes: • Strong & accountable public institutions • Rule of law • Respect for rights & freedoms • Equality & social justice