Integrating the policy agendas of Better Regulation and Sustainable Development in Regulatory Impact Assessment

1. Integrating the policy agendasof Better Regulationand Sustainable Developmentin Regulatory Impact Assessment Michal Sedlacko Research Institute for Managing Sustainability WU Vienna

2. A short terminological note • policy appraisal as the framing term: RIA, SIA, evaluation (Turnpenny et al, 2009) • sub-field of public policy or a sub-field of evaluation research? • performance of appraisal systems judged on a number of criteria: deregulation, compliance costs, better regulation, sustainable development (sustainability appraisals)

3. SD Governance and good governance

4. The governance background • normative concepts: • ‘good governance’ → better regulation → RIA • sustainable development • other governance dimensions: • knowledge-based governance • multi-level governance • network governance and ‘new governance’ • deliberative governance • ‘governance by evaluation’

5. Governance for SD 1 • The major governance challenges raised in Agenda 21: • “Integrating environment and development in decision-making” (chapter 8) • Increasing the coherence of policies between different jurisdictions (chapters 8, 38f) • “Strengthening the Role of Major Groups” such as local authorities, workers or businesses (Section III) • Facilitating a long-term strategic perspective (chapter 8 and throughout the document) • Achieving all this by utilizing different types of information and knowledge for decision-making (chapters 35, 40)

6. Governance for SD 2

7. Linking principles of SDto ‘good governance’

8. Linkage at EU level • EC’s impact assessment: • a full IA is now required for all items in the Commission’s Legislative and Work Programme (CLWP) • pursues objectives of better regulation: necessity, efficiency, effectiveness, openness... • fosters horizontal policy integration and policy coherence as well as deliberation: impacts in the economic, environmental and social domains, examines possible synergies and trade-offs • many shortcomings and slow improvement

9. ‘Governance by evaluation’ • Multiple types of appraisals around the policy cycle, with varying roles of the evaluator • needs assessment, feasibility assessment, RIA, various IAs, accompanying evaluation, ex post evaluation, value-for-money studies etc. • Able to provide: • legitimisation and justification of costs • conflict moderation • social and policy learning • However, neither procedurally co-ordinated nor having equal legal footing, no overall framework • New style of policy-making?

10. A brief introduction toregulatory impact assessment

11. The RIA tool • RIA is a process that accompanies (and to some extent structures) policy development and supports decision-making • however, can be used also for regulation already in place (ad interim, ex post) • typically conducted by administrative staff (planners) and attached to the regulative proposal when submitted to governmental debate or decision-maker • numerous forms of output: ‘explanatory memoranda’ attached to legislative proposals, ‘impact analyses’, background studies, ‘justification reports’ attached to policy proposals, fiscal statements…

12. Where does RIA come from? • quantification of administrative burdens of implementation, monetization of financial impacts on the state budget • technical assessment procedures (EIA, HIA, risk assessment) and their move upstream • good governance principles current conception of RIA

13. RIA and regulation development • where is the border between RIA and policy development? • tasks such as problem definition, analysis, consultation, setting up of monitoring mechanisms, communication etc. are all to some extent already present in and understood as part of policy development • RIA becomes distinct through the examination of impacts • RIA should integrate (not duplicate) existing procedures into one coherent process

14. Pros and cons • Many advantages: • better regulation • streamlined, formalised procedures • can ensure that all major interests compete on a level playing field (Radaelli, 2007) • fit into administrative culture: enables to explore preferences between competing groups and regulators (‘who wants what’) • receiving a lot of attention, especially at the EU level

15. Pros and cons • Risks and challenges: • practicality vs. complexity: having to deal with non-linear, indeterminate processes and complex causal chains and at the same time provide usable knowledge to decision-makers • sensitive to timing and political will • risk of strategic use • requires capacity • danger of formalism (then the ‘meat’ is going to take place elsewhere) • institutional challenges stemming from cross-sectoral nature

16. Lessons fromevaluation research

17. Purposes of an evaluation • Substantive rationales (Vedung, 1997) • internal and/or external accountability • basic knowledge advancement • intervention improvement • Strategic rationales (Suchman, 1972; Vedung, 1997) • posture • postponement • eye-wash • ducking responsibility

18. The accountability purpose • accountability: whether what was supposed to be done was done (summative function) • internal: elected politicians hold the administration responsible for its actions • external: citizens (or clients) hold elected politicians and their agents responsible for their actions • aspects of accountability (Rossi & Freeman, 1989): legal a., fiscal a., delivery a., coverage a., impact a., efficiency a. • the purpose of accountability to external parties calls for an external evaluation (greater credibility as objective procedures) • high risk that internal evaluator would: focus on components that work well; be reluctant to reveal negative results; perform deceptive balancing (playing field for strategic behaviour) • external evaluator: has more expertise and experience; is less afraid of job loss; has a reputation to care for

19. The improvement purpose • go/no-go, live-or-die decisions are relatively rare (due to vested interests) and therefore evaluation is rather used for improvement (Weiss, 1972), i.e. formative function • evaluation should be internally conducted: • evaluation is quick to initiate • achieves rapid learning (done by the same people) • there is no-one to conceal findings from • sound methodology is less important • better access to people • deeper understanding • the evaluator can facilitate implementation afterwards

20. The knowledge purpose • learning: a meta-evaluation is an evaluation of one or more evaluations that intends to systematically establish their value and merit (Widmer, 2005) • primary potential audience (the users) are typically the evaluation community, academic researchers, public policy and administration institutes etc. • done externally by universities or research institutes (although can also be conducted in-house)

21. A case for theory-based evaluation • “there is too little examination as to how the proposed regulation will be applied, enforced and monitored in practice” → the evaluators need to: • push the regulators to elaborate the regulation proposal in more detail • make the implicit assumptions and ideas of the regulators explicit → theory-based evaluation

22. Howsustainabledevelopmentaffects appraisals

23. Reflection of SD in appraisal 1 • assessing and aggregating impacts occurring in multiple domains (holistic approach) • technical incommensurability • weak or strong sustainability? (substitutability of individual forms of capital) • variety of data collection methods • participatory observation, focus groups, elicitation of preferences through economic methods, mixed qualitative/quantitative designs • acknowledgement of values (SD is a normative concept) • long-term designs (long-term impacts) • perspective in many ways radically different from mainstream economics (limits to growth, human wellbeing, rationality of economic agents) • indirect impacts

24. Reflection of SD in appraisal 2 • different and non-reducible viewpoints • social incommensurability, weak comparability of values • concern for public/stakeholder participation and deliberation • wider range of stakeholders involved in data collection or consultations • deeper involvement of stakeholders, up to the point of stakeholders framing the issues of relevance; empowerment • dealing with different types of information • facts and values, quantitative and qualitative, expert and lay knowledge, insights and judgements • dealing with different territorial and temporal scales of ecosystems and social structures and processes • dealing with complexity and uncertainty: shift from substantive rationality to procedural rationality and learning

25. SMCE/MSIA: cutting-edge instruments in sustainability appraisal • social multi-criteria evaluation (SMCE) and multi-scale integrated assessment (MSIA): • replace ‘consultation’ as they are highly participatory and deliberative • process-oriented (iterative) • integrate various kinds of knowledge • clarify the role of expertise (discussion support/decision support) • stress on procedural rationality instead of substantive rationality • address complexity and uncertainty • address social and technical incommensurability • support comparison of alternatives

26. Considerations for an SD-supportive RIA system

27. RIA setup 1 • central oversight unit fit into the structure of the central government (e.g. PM’s office) or an independent agency, advisory council, perhaps a place for SD council or commission? • helpdesk • RIA units in every department (legal, economic, SD-related expertise) • horizontal network for exchange and cooperation – especially relevant for SD • horizontal policy integration • (sectoral) peer reviews • stable personnel, administration of the network • external experts (researchers, consultants, auditors/evaluators) • peer reviews, peer review board? • national evaluation society • a body responsible for the RIA system and regulatory reform as a whole (inter-ministerial task force, audit agency, parliamentary commission)

28. RIA setup 2 • requires not only changes in administrative procedure (streamlining of regulation development), but a change of incentives structures and policy-making culture • strengthening responsibilities: RIA signed by a minister • give political weight to the central oversight unit • developing micro/meso/macro capacity is critical • ensure learning through ex post assessments and metaevaluations • regulatory planning/agenda: opportunity for an umbrella national SD strategy (move upstream): • provide strategic SD frame → identify regulation needs and conduct stakeholder analyses → regulation development and RIA • would also simplify assessment of effects

29. Initiating RIA • Timing: • “… the RIA process has to start at the beginning of the regulation process, not after the regulation has been written out.” (Flemish RIA guide) • “timing of RIA may be more important than the methodology employed” (Jacobs, 2006) • if IA occurs before modifications are made through inter-sectoral/public consultation and government debate, it loses objective relevance • Who should conduct a RIA? • “Regulatory impact analysis is best carried out by the project team which is preparing the regulation.” (Flemish RIA guide) • independence & threat vs. insight & sufficient capacity? • society-wide effects are “outside of the scope of ordinary policy analysts and RIA environmental economists” (Radaelli & De Francesco, 2007)

30. Scoping RIA 1 • how costly and time-consuming can a RIA be? • on the other hand, how formalised/flexible should RIA be? • ‘importance’ of a given regulation should influence: • the number of investigated options (including combinations of options) • the number and level (aggregation) of utilized assessment criteria and examined impacts • the minimum significance (magnitude, reversibility…) of investigated impacts • how far in the chain of effects should the analysis go • the extent of quantification • range of examined target groups/stakeholders • breadth and depth of consultation • the number of ‘partial’ RIAs and supportive analyses description of effects

31. Scoping RIA 2 • ‘importance’ of a given regulation should not influence: • calculation of administrative costs • analysis of legal aspects of proposed regulation • making assumptions and hypotheses explicit • regulatory coherence • balanced consideration of impacts across all three domains of sustainable development • consideration of at least the zero alternative (basic justification for regulation) • communication and accessibility of RIA report • how decide on ‘importance’ of regulation? (this procedure should be formalised) • monetary thresholds • qualitative criteria • screening through a first phase or a ‘light’ RIA • opportunity for strategic use?

32. What methods for RIA? • ‘Traditions’ of impact assessment: • economic methods (monetization & CBA, macroeconomic modelling) • difficult to foresee the extent of induced behaviour modification (despite of what neoclassical economists would want you to believe) • technical methods (scientifically proven pathways; EIA, HIA) • often formal checklists (lists of impacts) – needs regular review • expert judgment (decisions on complex issues; shift from substantive to procedural rationality) • choice of experts is extremely important • Other recommended methods: • participatory modelling, participatory scenario-building • ‘social’ multi-criteria methods: SMCE/MSIA • theory-based evaluation • multi-method designs

33. Is participation pursued to… • … elicit preferences or values in a process pre-structured by experts –or– to explore different problem definitions and categorizations? • … improve regulation through accessing knowledge –or–is knowledge sought to legitimate decisions rather than shape action? • instrumental reasons: • defuse conflict • provide knowledge • normative reasons : • Aarhus → good governance • substantial reasons : • legitimisation of decisions

34. Issues to address • participation in what capacity: as members of the public, as neutral experts or as stakeholder representatives? • different roles and rationalities • stakeholders: issues of internal accountability and representation • what barriers to participation exist? (requirement of highly technical sector-specific knowledge, costs to participate, legal/institutional forms etc.) • risk of culture of institutionalised dependence → flexible and multi-track participatory designs to cover institutionalised stakeholders, regulation-specific marginalised groups and individual citizens, as well as their various information needs and capacity

35. Concludingremarks

36. Key messages • ‘political dimension’ of RIA: • motivation of politicians for adopting/resisting RIA, incentive structures and strategic use by administration • RIA is being introduced into environment which was not ‘empty’: in addition to existing administrative procedure in regulation development symbolic exchanges, negotiations and coalition building, co-ordination, establishment of control structures (principal-agent relationships), strategic and symbolic use of knowledge, rent seeking etc. were taking place • expectations, usage and quality are linked

37. Key messages • there are lessons to be taken from evaluation research (e.g. on the internal/external character of RIA and its purposes, the political context, as well as theory-based evaluation) • SD constitutes a set of specific challenges; the field of ‘sustainability appraisal’/SD evaluation addresses these issues (cutting-edge instrument of SMCE/MSIA) • design of consultation and participatory exercises is extremely important • making RIA to better serve SD concerns is an opportunity for overall improvement of regulatory reform processes • horizontal policy integration and policy coherence • upcoming challenge: vertical policy integration

38. michal.sedlacko@wu.ac.at Thank you.