Performance Management: Overview of Issues

Performance Management: Overview of Issues Bill Dorotinsky The World Bank August 25, 2008

Outline • Background • Basic questions • Why measures performance? • What to measure? • How to measures? • When, how often to measure? • How to give impact to measurement? • Current state of measurement • Major pitfalls of measurement systems • Common challenges

Today, a thousand performance flowers blooming…… • Performance contracts (Denmark, NZ, Thailand, UK) • Program budgeting (Armenia, Burundi, Romania, South Korea, Ukraine) • Performance auditing (SAI’s globally) • Performance plans (US) • Program evaluation (Brazil, Chile, formerly US) • Performance Monitoring (Brazil) • Activity-based costing (Iran) • Client satisfaction surveys (Philippines) • Benchmarking (Iran) • Balanced Scorecard (US municipal, Thailand) • Program assessment (US) • Monitoring and Evaluation (Egypt) …and there are more terms added daily….results-oriented budgeting, etc.

Performance reforms have many objectives and drivers • MoF need for basis for reallocation of funds • Desire to get agencies to think about linking inputs and outputs • MoF desire to know what they buy • Desire for more accountability and control • Desire for greater efficiency, effectiveness • Ministry/agency desire to manage better, deliver results • Government desire to deliver results • … are introduced in many situations • Weak accounting, reporting • Absence of budget offices (Burundi) • IT rich (Brazil) and IT poor (Burundi) situations • … and frequently with other reforms • Budget classification • IFMIS • Tracking spending • MTEF • Deconcentration of spending authority • Civil service reform and performance pay

Why measure? What gets measured gets done • Eight Purposes that Public Managers Have for Measuring Performance • Evaluate: How well is my public agency performing? • Control: How can I ensure that my subordinates are doing the right thing? • Budget: On what programs, people, or projects should my agency spend the public’s money? • Motivate: How can I motivate line staff, middle managers, nonprofit and for-profit collaborators, stakeholders, and citizens to do the things necessary to improve performance? • Promote: How can I convince political superiors, legislators, stakeholders, journalists, and citizens that my agency is doing a good job? • Celebrate: What accomplishments are worthy of the important organizational ritual of celebrating success? • Learn: Why is what working or not working? • Improve: What exactly should who do differently to improve performance?

What to measure? • Performance has many dimensions • Achievement of national plans • Government policy • Ministry strategy or mission • Agency or activity performance • Personnel • Financial • Impact • Process • Efficiency, effectiveness

Balanced Scored Card

Performance of what, for whom? • Different systems have been developed to meet different intents, problems • Government to society (Oregon – accountability) • Government to external stakeholders? • Bureaucracy to Elected officials (UK Next Steps – accountability; Brazil PPA; US High Impact Agency Initiative; Mexico National Plan) • Agencies to their managers (US GPRA – performance) • Ministries/programs to MoF/budget office (US PART – efficiency, effectiveness, parsimony) • Service providers to ‘purchasers’ (balanced scorecard, accountability) • Many reforms have been initiated without addressing what area of performance is being targeted, who will use the information, and how • Few also distinguish between performance monitoring (short-term) and assessments of effectiveness and impact (medium to long-term)

How to measure performance? • Approaches • Baselines – historical • Targets – plan objectives or performance targets • Benchmarking – against similar institutions, domestically or internationally • Techniques • Regular monitoring, reporting • Evaluations, audit • Special studies, reviews • Surveys, customer satisfaction • Peer review

Inclusion of Performance Targets • Only about one-fifth of countries include performance targets in performance data for all programs, and about one-third include performance targets for more than half of programs. • About 70 percent include targets for at least some programs. Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Measurement Challenges • Type of program, activity • Research, service delivery • If, when outcomes can be known * Bureaucracy. James Q. Wilson. Basic Books. 1989. See especially Chapter 9, Compliance

Characteristics of Good Indicators • indicators should • be relevant - having a logical relationship to the users’ needs including a clear relationship to objectives which define the desired outcomes • be appropriate - of a form which will assist the user in assessing the performance of the agency in the discharge of objectives established • have adequate notes - performance indicators are required to have adequate notes which relate the indicator to the objective, explain why the indicator is considered to be a key measure of performance and how the outputs produced link with the outcome achieved

Characteristics of Good Indicators • Indicators should be • Quantifiable - implying a measurable relationship to attainable benchmarks as a means of determining the extent to which outputs and desired outcomes have been achieved • Free from bias - the information used to indicate performance should be impartially gathered and impartially reported • Verifiable - appropriately qualified individuals working independently should be able to come to essentially similar conclusions or results about performance indicators

When, How often to measure? • Outcomes multi-year, outputs annual • Too frequent gives too much data, shows no results • Too infrequent means timely corrective action may not be possible • Frequency depends on purpose • Financial – frequent • Outcomes – every few years

Is performance continuously monitored? • Two-fifths of countries do not continuously monitor performance against targets • Portfolio ministry most likely to monitor performance • In one-third, the MoF monitors performance, and in one-third the relevant agency monitors performance internally. • In only about one-sixth of countries does the legislature monitor performance continually against targets, with OECD legislatures three times more likely to do so than LIC/MIC legislatures Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

How to give impact to measurement? • Incentives • Budget • Performance Pay • Reputation • Passive – transparency • Active - awards • Natural incentives (better information to help manager do their job) • Report in a user-friendly, useful, timely way appropriate to the target audience • Develop good measures • Develop better, clearer goals

Non-financial performance data in budget documentation • About one-third of all countries do not provide non-financial performance data in budgets • But, nearly seventy percent of all countries provide at least some non-financial performance data routinely in budget documentation. • OECD countries are more likely than LIC/MICs to include non-financial performance data in budget documentation for all programs and for at least half of programs. Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Expenditures Linked to Strategic Goals? • About three-quarters of all countries have at least a fewexpenditures linked to strategic goals. • Only one-fifth of countries have allexpenditures linked to strategic goals – true for OECD and LIC/MICs. Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Expenditures Linked to output/outcome targets? • About one-fifth of all countries do not have performance targets specifically linked to expenditures. • About one-third of all countries have at least some expenditures specifically linked to performance targets. Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Evidence performance results affect budget allocations? • About one-third of all countries state there is no evidence performance results are used in determining budget allocations. LIC/MICs are more likely to state there is no evidence. • Nearly one-half of all countries state there is evidence performance results are used in determining budget allocations by MoF between programs. Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Do politicians use performance measures in decisions? • Fifty percent of all countries state that it is not common for performance measures to be used by politicians in decision making. • Nearly fifty percent of all countries state that it is common for portfolio ministers to use performance measures in decision making. Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Rewards or Sanctions for performance? • Nearly 70 percent of all countries report no rewards or sanctions being applied if performance targets are met or not met. • OECD countries report unit budget affected (22 %) Source: World Bank – OECD budget procedures database athttp://ocde/dyndns.org/

Implementing effective performance monitoring * requires • Setting organizational incentives to support performance monitoring • Getting performance monitoring consistent with organizational culture • Need for central unit to play active and effective leadership role in defining criteria and implementing practical performance monitoring • Linking ongoing performance measurement with more periodic program and policy evaluations • Attention to client/customer/citizen in developing performance monitoring * Monitoring Performance in the Public Sector: Future Directions from International Experience. Mayne, John and Eduardo Zapico-Goni, eds. Transaction Publishers; London, 1997. Pp.22-4.

Setting better goals • SMART Goals “Must Have” Criteria • Specific – detailed outcomes criteria • The goal should state the exact level of performance expected. • Measurable – measurement criteria • To achieve objectives, people must be able to observe and measure their progress. • Attainable – realistic criteria • Goals should challenge people to do their best, but they need also be achievable. • Relevant – significance criteria • Goals need to pertain directly to the performance challenge being managed. • Timeframe – answers “by when?” criteria • Deadlines help people to work harder to get a task completed.

Current state of performance monitoring • Much passive provision of data • Less linkage to management (targets) or strategic goals • Presumption portfolio ministry uses data • Less linkage of data to budget decisions • Little hard evidence of impact • Less linkage to sanctions/consequences Performance may have marginal impact within portfolio ministry, but does not yet appear to have much impact in general budget management.

Major Pitfalls of Measurement Systems • Amassing too much data - It results in “information overload.” So much so that managers and employees either will ignore the data or use it ineffectively. • Focusing on the short-term - Most organizations only collect financial and operational data. They forget to focus on the longer-term measures of customer satisfaction, employee satisfaction, product/service quality, and public responsibility. • Failing to base business decisions on the data - A lot of managers make decisions based on intuition and past experience rather than the data being reported to them. If the data is valid, it should be used appropriately. • “Dumbing” the data - Sometimes data can be summarized so much that it becomes meaningless. If business decisions are going to be based on the data, then the data needs to be reported clearly and understandably. • Measuring too little - Making business decisions with too little data is just as problematic as basing them on too much data. Some organizations (particularly smaller ones) tend to measure too few key variables to get the “whole picture” of the health of their organization. Mostly, their focus is on financial indicators. • Collecting inconsistent, conflicting, and unnecessary data - All data should lead to some ultimate measure of success for the company. An example of conflicting measures would be measuring reduction of office space per staff while, at the same time, measuring staff satisfaction with facilities. • Driving the wrong performance - Exceptional performance in one area could be disastrous in another. Arrests versus crime.

Major Pitfalls of Measurement Systems • Encouraging competition and discouraging teamwork - Comparing performance results of organizational unit to organizational unit, or one employee to another, sometimes creates fierce competition to be “Number 1” at the expense of destroying a sense of teamwork. • Establishing unrealistic and/or unreasonable measures - Measures must fit into the organization’s budgetary and personnel constraints and must be cost-effective. They also must be achievable. • Failing to link measures - Measures should be linked to the organization’s strategic plan and should cascade down into the organization (horizontal and vertical linkage). Measures without linkage are like a boat without water. They’re useless and they’re not going anywhere. • Measuring progress too often or not often enough - There has to be a balance here. Measuring progress too often could result in unnecessary effort and excessive costs, resulting in little or no added value. On the other hand, not measuring progress often enough puts you in the situation where you don’t know about potential problems until it’s too late to take appropriate action. • Ignoring the customer - Management often wants to measure only an organization’s internal components and processes. That way they can “command and control” it. • Asking the wrong questions/looking in the wrong places - Sometimes business executives ask who’s too blame instead of asking what went wrong. They look for the answers in the people instead of the process. A faulty process makes employees look faulty. • Confusing the purpose of the performance measurement system - The purpose of a performance measurement system is not merely to collect data but rather to collect data upon which to make critical business decisions that will in turn drive business improvement. Knowing that you’re 10 pounds overweight is just knowledge of a fact. Taking improvement actions based on that knowledge is where ”the truth” lies.

Common Challenges • For beginners: • Program definition • Performance indicator development • Setting targets • For all: • Making performance information available • Using performance information • Linking performance to budget • Linking performance with reward/sanction

Performance Management: Overview of Issues