THE FRAGILE COMPLEXITY OF REGULATORY QUALITY: DEFINITIONS AND MEASUREMENT Claudio M. Radaelli

1. THE FRAGILE COMPLEXITY OF REGULATORY QUALITY: DEFINITIONS AND MEASUREMENT Claudio M. Radaelli Centre for European Studies Presentation delivered to CARR, Centre for the Analysis of Risk and Regulation, LSE, 16 March 2004

2. Project: Stage 1 Review of initiatives and indicators used by EU member states to measure regulatory quality (plus others with interesting experiences) Stage 2: Elaboration of a set of indicators to be used by the European Commission Timing: 14 months, interim report in May 2004, report on indicators in September 2004 IRQ Team Bradford team (Radaelli, De Francesco, Cameron, Jalilian) Advisor: Scott Jacobs Advisor: Bruno Dente Support network: 27 national experts on ‘better regulation’ policy from 16 EU M.S. (incl. 6 CEECs) IRQ – Indicators of regulatory quality

3. Aims in this presentation • To discuss regulatory quality, the quality of indicators, and indicators of quality. Proliferation of OECD and DBER initiatives… ‘pie-chart’ mania • To collect feedback, suggestions, and information

4. Table of contents • Getting to grips with the concept: what is regulatory quality? • Complexity of regulatory quality • How does one measure quality? • Classification of indicators • From classification to the analysis of the performance of indicators and system of indicators

5. 1. Regulatory quality (RQ) • Governments • How the EU defines RQ • Definitions provided by international organisations • Academic approaches

6. UK: Better Regulation Task Force • Proportionality, • accountability, • consistency, • transparency, and • targeting

7. Commission • RQ is part of the strategy for better regulation and good governance. This implies that, in addition to properties such as efficiency, effectiveness, coherency, simplicity, and clarity, good regulation should also respect: • Standards for the re-vitalisation of the acquis (COM71 Final,) Commission 2003 • Standards for the democratic use of expertise • Standards for consultation • And most importantly good regulation balances environmental protection, social conditions, and economic growth sustainable development

8. Mandelkern Report (Council’s experts – EU) Common principles: • Necessity • Proportionality • Subsidiarity • Transparency • Accountability • Accessibility • Simplicity MR is focused also on how to achieve RQ. The main tools are: • Alternatives to regulation • IA • Consultation • Simplification • Access to regulation

9. OECD: evolution of ideas on RQ • 1995: Good practices -> standards -> checklists • 1997: Three stages of development of regulatory reform: de-regulation, regulatory quality (instrument by instrument approach), and regulatory management (more horizontal) • 2002: From RQ to regulatory governance (transparency, citizens as partners, and bringing principles of better regulation across levels of governance and regulatory actors (legislature, judiciary)  grounded in the wider approach to good democratic governance  two main perspectives on regulatory quality: economic and juridical, both correlated with the design and implementation of policies, tool and institutions

10. Academics: Baldwin and Cave • They derive RQ from the claims to legitimacy [of regulation]: • Legislative mandate • Accountability • Expertise • Efficiency: the legislative mandate is implemented correctly

11. Baldwin, Hood, and Rothstein 'Assessing the Dangerous Dogs Act: When Does a Regulatory Law Fail?' Public Law Summer (2000) • Principles do not help much. Paradoxically, one can demonstrate that even bad legislation can pass the test of complying with the principles of the Better Regulation Task Force. • We need to evaluate the results achieved by regulation rather than use principles and key-words. More emphasis on systematic evaluation (ex-ante and ex-post) and on effective implementation of proposed regulation  Indicators, not slogans?

12. “Economists” • They take issue with the narrow notion of efficiency presented by Baldwin and Cave • They have a different benchmark: maximisation of the collective welfare • Quality regulation is about the economic impact of regulation, not about implementing the mandate

13. Dimensions of quality • Economists: They do not talk about RQ, they talk about the efficiency of regulation. • But efficiency is only one element of quality or one claim to legitimacy (in Baldwin and Cave’s typology). • Another element of RQ is the quality of the process through which regulations are produced. • Doubt: do economists measure quality or quantity (extent of regulation)?

14. RQ tells us something about quality of governance RQ is neutral with respect to size of government or scope of intervention (claim made by OECD). Hence it does not express a judgment on how big government should be. Unless one follows the economists and ‘confuses’ quality and quantity. The notion of RQ is a bit fuzzy There are too many attributes in the current definitions of regulatory quality, but how does one recognise regulatory quality in the real world? What are our empirical sensors? To conclude on RQ: Advantages and limitations

15. How our project approaches RQ • Design of the process (structures, inter-organisational linkages, etc.) • Activities and outputs • Real-world outcomes

16. The puzzle of real-world outcome • Cross-country differences revealed by indicators: do they really mean differences in policies and their enforcement? Or do they simply register different models of capitalism (Hall and Soskice)? • Jaffe et al, JEL 1995: the evidence of the impact of regulation on competitiveness and locational choices of firms is far from clear. Same results for environmental regulation. But see Koedijk, K. e J. Kremers (1996), Market opening, regulation and growth in Europe, Economic Policy. Regulatory governance matters • Literature on regulatory competition: evidence tells us that countries do not compete by using regulation in the way predicted by simplistic economic models of ‘races’ • OUR SUGGESTION: for us real-world outcome makes more sense if we look at changes in the behaviour of regulatory stakeholders. Regulatory governance as learning

17. Tools for RQ • Impact assessment • Consultation • Simplification, codification, recasting • Access to legislation and regulatory transparency • Ex-post evaluation of regulatory tools and institutions

18. 2. Complexity of RQ • Quality is not a monolithic concept • Quality for whom? • (A) Context shapes RQ • SEE PAPER ‘HOW CONTEXT MATTERS’ (Radaelli) • (B) Models of the policy process • (C) Models of actors • (A), (B) and (C) are often neglected in the one-size-fit all approach to RQ

19. How different stakeholders look at RQ

20. Implicit models of actors and implicit theories of the policy process • Different stakeholders bring into the discussion diverse logics and criteria of quality and quality assurance mechanisms. • It is not clear what is the model of stakeholders implicit in RQ programmes. • On theories of the policy process see the paper HOW CONTEXT MATTERS

21. Implicit theories (2) • Are these programmes based on the assumption that politicians are rent-seeking, hence quality assurance mechanisms should target this problem? • Or does ‘quality’ mean that a government is trying to curb bureaucratic power? Is RQ an approach to limit the power of business in the policy process or an instrument to provide more systematic access of corporate actors to the regulatory process? • What do corporate actors want in a regulatory process? Efficiency or protection? Profit maximisation or satisfying behaviour? Are they a unitary actor or in competition among themselves?

22. Implications • It is important to acknowledge the complexity of RQ up-front. • It is indispensable to discuss the criteria, theories, and models of actors upon which our approach to RQ is based.

23. 3. How does one measure quality? • Tests • Indicators: • Quality of the RIA process • Quality of the regulations produced via RIA

24. LEVELS TOOLS TYPES OF REGULATION SECTORS OF REGULATION DEGREES OF SOPHISTICATION TYPES OF MEASURE PROVIDERS LEVELS OF MEASUREMENT PURPOSES PRINCIPLES 4. Classification of indicators

25. BY LEVEL • Design of the process • Activities and Outputs • Real-World Outcomes

26. BY TOOL • Impact assessment • Consultation • Simplification, codification, recasting • Access to legislation and regulatory transparency • Ex-post evaluation of regulatory tools and institutions

27. BY TYPE OF REGULATION • ECONOMIC REGULATION • SOCIAL REGULATION • ADMINISTRATIVE REGULATION

28. BY SECTOR OF REGULATION • Regulation of Entry • Regulation of Labor • Environmental Regulation • Product Regulation • TAX and Social security law • Planning regulation • Etc…

29. BY DEGREE OF SOPHISTICATION • Simple measures • Composite measures (Factor analysis, aggregation of simple indicators on the basis of weighting). • Note that composite measures are not necessarily better than simple measures. For example, factor analysis may not be explicit enough on the variables that can be controlled by program administrators. Or it can be the statistical artifact of a questionnaire with too many questions.

30. BY TYPE OF MEASURE • Objective • Subjective

31. BY PROVIDER • Governments (self-assessed responses to regulatory quality questionnaires) • Experts (think tanks responding to Word Bank questionnaires or reviewing IA) • Stakeholders

32. BY CHARACTERISTIC • Categorical (ex. Dichotomous (Yes/No); • Ordinal (ex. 1 to 5 scales) • Continuum (ex. Number -- no. of procedures, steps, etc.; Ratios; indexes)

33. BY PURPOSE: INDICATORS FOR WHAT? • Learning • Explanation • Transparency and accountability • Program management • Participation and ‘good regulatory governance’

34. By PRINCIPLE of better regulation policy • Australia • Canada • UK • US • Academics • Mandelkern Report • OECD • See Section 1 on the concept of RQ for the list of principles

35. An example: combining values and levels

36. 5. From classification to the analysis of performance • Assessing quality of individual indicators • Assessing quality of systems of indicators • Controlling for adverse effects • Crucial step: Managing indicators – using indicators to improve management, to learn, and to produce better regulation.

37. From classification to the assessment of quality of each indicator (1) EASY STEPS • Validity (relation between indicator and phenomenon) • Quantifiable at regular intervals, so that the indicators can be monitored • Reliable (two different people produce the same value given the same conditions) • Comparability and cross-checkable • Easy to gather, easy to communicate

38. From classification to the assessment of quality of each indicator (2.1) ENTERING COMPLEXITY • Efficient: a sophisticated indicator (arising out of factor analysis) contains more information than simple indicators. • However, there may be a trade off between the quantity of information contained in an indicator and how easy it is to communicate it within the organisation and to the stakeholders. • It is important that the variables upon which the complex measure is built are easy to understand and coherent (no trade-off between the variables).

39. From classification to the assessment of quality of each indicator (2.2) • Relation with principles or norms. This is to avoid ambiguity. The indicator relates to an outcome that can be unambiguously judged satisfactory or not. • Do we measure the quality of the regulatory environment or the quality of better regulation programmes? In the long term they should converge. In the short-term, the environment can improve independently from the policy. Example: more people employed to issue licenses, the time needed to get a licence goes down, but the overall regulatory complexity is the same.

40. From classification to the assessment of quality of each indicator (3) ENTERING PRINCIPLES • Take the example of principles of better regulation. Think of participation. Is this a dimension of quality? It depends on why one is designing an indicator Always ask the question: if the indicator is the solution, what is your problem? Do you want to learn, to manage, to steer, to empower people,….? This example tells us that the definition of quality is sensitive to the main purpose one has in mind when developing the system of indicators.

41. From classification to the quality of systems of indicators • Coverage … but also simple and selective • Good balance among design, output, and real-world outcome. Good balance between providers (stakeholders and governments) • The system should be targeted and focused on those themes in regulatory reforms programmes that have large implications for decision-making

42. Dynamic systems of indicators • Start with simple system of indicators, then in year two add more sophisticated indicators • Use indicators to allow member states to measure their own progress year by year and in relation to other member states. Indicators and scoreboard of regulatory quality? Drawback: learning may be hampered by the ranking syndrome. This is not a race, but a mutually beneficial exercise. • Use two sets of indicators. A simple set of those who use indicators to monitor complex regulatory reform programmes and another set of indicators for those who manage individual programmes.