Implementing reforms to improve governance is inherently difficult

1. PERSPECTIVES IN IMPACT EVALUATION:APPROACHES TO ASSESSING DEVELOPMENT EFFECTIVENESS31st March – 2nd April 2009Semiramis Hotel, Cairo, Egypt What does Implementing Anticorruption Projects in Bulgaria Tell Us about Importance of Impact Evaluations? MilenaMinkova (University of Sofia “St. ClimentOhridsky)

2. Implementing reforms to improve governance is inherently difficult Anticorruption reforms- resistance from all levels Considerable effort and finances involved, yet, little evidence so far for that they have meaningfully contributed to reduction of corruption levels Arguments to justify deficiencies in assessing anticorruption interventions measurement of corruption problem laying the blame with recipient governments focus on long term objectives of anticorruption makes it difficult to expect immediate effects

3. AC in Bulgaria as a case To highlight the critical role of impact assessment for implementation of effective anticorruption interventions. Learning from anticorruption interventions has increasingly become instrumental to fighting corruption

4. Anticorruption in Bulgaria is at important crossroads Bulgaria entered the EU on 1 January 2007 Critical for its progress is its ability to remedy certain shortcomings in the fight against corruption Progress monitoring reports of the EU commission so far, have been critical. Corruption in Bulgaria is characterized as “endemic’ July, 2008 EU monitoring report “institutions and procedures look good on paper but do not produce results in practice; recommendations are made but are not followed up “. Simultaneously, public perceptions of the importance of tackling corruption- record highs

5. Anticorruption has been the focus of continuous interventions for more than 15 years First phase - early to late 1990s centered largely on awareness raising Second phase - late 1990s to mid 2000 centered on international conventions, institutional reform and capacity building Third phase- focused technical assistance projects 2004-2008, interventions with the frame of the OPs- as of 2008- institution strengthening, capacity strengthening Confronted with high perceptions of corruption as a major policy issue, results are referred to as unsatisfactory by all national and international actors.

6. Impact assessment of anticorruption interventions Persistently managed to remain outside the focus of concern Little is known about the effects of anticorruption assistance, more important, little has been asked Beyond accounting for output indicators there is no effort to capture impact effects. This of itself has left impact on the anticorruption agenda as a whole.

7. Assessing impact - instrumental to relevant, effective and efficient AC interventions. No longer a technical issue as there is an important strategic aspect related. Projects have succeeded in raising demand for reform, solutions to respond to this demand have yet to be found High risk of free riders of the anticorruption rhetoric, taking advantage of it to suit their own political agendas, or to the public’s frustration, increasing voter apathy and disillusionment with equally damaging results for democracy

8. Assessing impact of first anticorruption interventions All these large scale projects indicated reducing corruption among their core objectives; none of them claimed that they had effectively reduced corruption Indicators or proxies of success—immediate effects of intervention: creation of new structures to fight corruption- coalitions, networks; capacity building of civil society Focused analysis of the impact of projects, or its broad effects, both intended and unintended, is generally lacking

9. Raising public awareness of the importance of corruption Coalition 2000- created to influence the political discourse on anticorruption, to push political actors to reflect on the issue of corruption An increased public awareness of corruption has been generally accepted as a positive good, for lack of evaluation, it is impossible to know whether other objectives have been met, for example: to ensure that a better understanding of corruption is achieved, to create public intolerance to corruption, to pressure the government into accepting reforms

10. Enactment of new legislation Reported as a successful intervention in the anticorruption agenda, for example, Coalition 2000 is credited for pressing for the implementation of the ombudsman law practical achievement- to have successfully lobbied government to adopt the law For lack of evaluation, little is known if the intervention itself has contributed to implementation of anticorruption legislation

11. Strengthening the Capacity of Civil Society Groups The clearest effect of AC interventions Because of the lack of baseline surveys impact on capacity cannot be really measured

12. Institution building /creation of institutions/ and strengthening Twofold approach: in-depth institutional reform projects, and efforts to change the interface between government and citizens, usually at the local municipal level Immediate objective- institutional change, second, long-term objective - to reduce corruption With respect to these long term effects, the stakeholders involved generally did not consider them as being anticorruption projects Administrative reform projects, usually small sized, that intended to change the citizen/government interface using pilot approaches- generally assessed as successful in achieving the immediate objectives set

13. Lessons learned from institution building initiatives Beyond feasibility studies on the transfer of EU best practices, no ex post assessment The case of Regional Public Anticorruption Councils Institutional changes must be followed with adequate monitoring if they are to be successful Relevance, effectiveness and efficiency of institutions- recently put on the agenda, within the frame of functional analysis. It is yet to be seen whether and how results will be incorporated into the decision making process

14. Operational Programs- Opportunities for AC Opportunities for anticorruption initiatives are open to government, judicial system and civil society. All of them list reduction of corruption as their primary objective It is far too early to assess their impact as none of these projects have been completed Absence of outcome indicators for most projects- no built-in check up mechanism for the realistic potential of anticorruption measures to meet the major goal of reducing corruption

15. The Road Ahead ??? AC efforts so far no matter the resources and the supporting good will of stakeholders, have not been successful in attaining their major goal- combating corruption Lots of doing- limited learning Critical-to search beyond the concrete faces in the anticorruption chain and associate poor results with concrete individuals/ political parties/ partisan interests, and try to identify whether there is a system problem in managing corruption risk through targeted interventions, i.e. policy, projects, measures.

16. The Road Ahead Need to rethink, reshape and reform the way we design, implement and evaluate anticorruption interventions Within this ambitious format, to search for appropriate format to assess impact of anticorruption efforts Hear the voice of constituents to see the change Improve project design to move from accounting to learning Build knowledge and commitment at various levels improve coordination between various actors to maximize gains and optimize efforts/ costs Improve ownership in the AC reform initiatives- engage the full range of stakeholders in the process, this would require to provide the necessary incentives

17. The Road Ahead Resource impact evaluation as one element of a robust system for monitoring and evaluation; to the extent possible integrateimpact assessment in the cycle of managing anticorruption initiatives The Bulgarian experience so far suggests the need to not only link planning and implementation of anticorruption efforts into a coherent process, but also find ways how to make it in an efficient, effective and sustainable manner.