INDICATORS OF REGULATORY QUALITY

1. INDICATORS OF REGULATORY QUALITY Project website Claudio M. Radaelli Brussels, 25 January 2005

2. Description of the project • Regulatory quality is a complex concept. In our project we acknowledge the role of different stakeholders and institutional contexts • Quality measured in terms of design, activity-output, and real-world outcome • Assessment of indicators and indexes proposed in the literature • Analysis of better regulation programmes: focus on four tools (IA, Simplif., Access, Consultation) • Questionnaire sent to support network in May 2004 • The project is about the DESIGN of indicators. We have not calculated indicators, neither have we scored Member States

3. Why quality? • Quality versus quantity • Quality and efficiency of regulations • Better regulation as a public policy (actors, resources, instruments, and decision-making structure) • Quality and the Lisbon strategy • Quality and indicators

4. Measuring regulatory quality in EU Member States (questionnaire) • Diffusion of the Mandelkern report principles • Most countries have opted for an experimental approach to indicators – one based on the gradual introduction of simple measures • Setting quantitative targets is an effective way to encourage public administration to implement measurement systems (as shown by the case of B, DK, NL and S) • No systematic use of cost-benefit analysis in the EU Member States. Rather, the emphasis is on how stakeholders are affected by proposed regs

5. Diffusion of principles but different practice: three clusters • Tools of better regulation are still at the stage of pilot project. Measurement of quality has been considered, but limited activity. • Basic systems of quality assurance and focus on administrative burdens. • Sophisticated quality assurance, well-embedded impact assessment, initiatives for the measurement of quality which target the whole regulatory process

6. Combining diversity of approaches and common progress in the EU • Different practice is compatible with common progress • Acommon approach, shared beliefs about regulatory quality and its components, and sense of direction towards the Lisbon goals • Three specific sets of indicators

7. The hard questions • FROMINSTRUMENTS TO POLICY:How do we know that a good IA produces better regulation? • FROM POLICY TO REGULATORY OUTPUT: How do we know that ‘better regulation policy’ produces better regulations? • FROM REGULATORY OUTPUT TO FINAL ECONOMIC OUTCOME: What is the causal chain going from good regulation to real-world economic outcomes? How robust is it?

8. How we answer these questions • Focus on how regulation changes the behaviour or stakeholders • Selection of five indicators that measure the quality of better regulation policy without going too far in the causal chain • We use Lisbon indicators, but we do not test the quality of better regulation policy directly on economic indicators like the growth of the knowledge-based economy

9. The three systems • Design of better regulation policy and tools. Macro and ex-ante • Ex-post and micro • Bridge between measurement of regulatory quality and the systematic evaluation of better regulation as public policy

10. Where do the indicators come from? • Operationalisation of the principles of better regulation • Questionnaire and consultation with support network • Experience of MS and other countries • Criteria of “usable knowledge” = indicators that can be embedded in policy processes

11. Indicators – characteristics of system 1: quality of the process • an ex-ante macro-system - does not go down to the level of individual impact assessments or consultations • based on objective indicators, it covers the design of better regulation policies,instruments (contents of formal guidance), and supporting and quality assurance activities • can be managed by coordinating units in MSs and the SG in the Commission

12. Quality assurance

13. Indicators – characteristics of system 2: internal evaluation • An ex-post micro-level system focused on what the tools of better regulation deliver in a given time-period • We look at guidance as designed in system 1 and measure what has been achieved

14. System 2 contains the following: • A simple checklist on the quality of individual impact assessments and major consultations • Real-world measures of the impact of better regulation policies (subjective: surveys over time) • Five real-world measures of the impact of better regulation policies (objective) • Limited number of important indicators of economic outcome – role of causal chains

15. Survey of regulators Percentage of regulators who think that IA has the following effect: [1] Makes policy formulation more transparent [2] Speeds up / delays policy implementation [3] Reduces conflict at the stage of policy formulation [4] Reduces conflict at the stage of policy implementation [5] Be more likely to represent the preferences of citizens [6] Final policy decision will be more likely to achieve goals [7] Final policy decision be more likely to represent a consensus among affected entities [8] Final policy decision be more likely to create open markets

16. Survey of citizens and firms [1] % of citizens who think that their views are taken on board in the development of single market policy in the EU [2] % of firms who think that the quality of regulation in the EU (including both EU and domestic rules) has improved over the last three years [3] % of firms who think that participation in EU-level consultation makes an impact on the final decision [4] % of firms who think that EU policy has an impact in opening markets, making it easier to compete, and generating competition in the professions [5] % of firms who think that in last three years restrictions and obstacles to their business have disappeared altogether or significantly reduced [6] % of firms who think that regulatory changes at the EU level are predictable

17. The Five Real-World measures of better regulation policy [1] Estimate of total regulatory costs delivered by regulations for which impact assessments (IAs) were prepared in year xxxx / Estimate of total regulatory benefits delivered by regulations for which IAs were prepared in year xxxx [2] Net benefits delivered by regulations for which IAs were prepared in year t / Net benefits delivered by regulations for which IAs were prepared in year t-1 [3] Total cost reduction resulting from simplification [4] Annual rate of reduction in the total administrative burdens stemming from EU legislation [5] Cost of administrative procedures eliminated in year t / Cost of administrative procedures eliminated in year t-1

18. System 3: external evaluation“All policies can be evaluated, and better regulation is no exception” • no matter how sophisticated, indicators are just a component of quality assurance • the latter requires strategic and operational management, specific structures, and dedicated tools • one important element is external evaluation • indicators must be complemented by interviews, case studies, etc. • external evaluation should be done by think tanks, academics, and stakeholders - notion of pluralistic evaluation

19. Indicators – which system is suitable for whom? • system 1 could be used by the cluster of Member States with basic approaches to regulatory quality and impact assessment in experimental-pilot stage. They should move as soon as possible to the second system • system 2 could be used by the cluster in which consultation, simplification, and the assessment of administrative burdens are already well embedded - however, some simple indicators of this system can be adopted by all EU governments now, no need to wait

20. Indicators – which system is suitable for whom? • finally, system 3 is applicable to countries with sophisticated quality assurance mechanisms (as well as to the Commission)

21. How to use indicators • Open method of coordination • High-level group on Competitiveness – Competitiveness Council, this can be a body that can adopt indicators • DBER can be the forum where data are collected and technical discussions on indicators take place

22. Why use open method of coordination? • Better regulation policy is an essential component of the Lisbon agenda. • It involves different levels of governance. • A responsibility for the EU and the Member States

23. There is an OMC in the making: • Principles (Mandelkern) • National programmes of regulatory reform • Peer review of reforms (within DBER) • Timetable (Lisbon) • Need to complete OMC with Indicators and iterative processes