Does Measuring Performance Lead to Managing for Performance?

1. Does Measuring Performance Lead to Managing for Performance? Examining a Poorly Understood Relationship M. Bryna Sanger Deputy Provost & Senior VP, Academic AffairsJune, 2011

2. Performance Measurement is de rigueur in U.S. Cities • Many cities devote considerable resources to measuring performance • Multiple organizations support, promote, fund, and reward performance measurement • Performance measurement efforts respond to citizens pressure for increased accountability • Performance measurement is increasingly viewed as a sign of “good government” and best practice Improving the Quality of Public Services APPAM / Moscow Conference

3. Many Reasons to Measure Performance • To evaluate, control, budget, motivate, promote, celebrate, learn and improve (Behn, 2003) • To enable learning and improvement, key elements of management are: • Understand what drives performance • Collect, analyze, and disaggregate (by district or precinct) timely data frequently • Take risks, experiment, and tolerate well-conceived failures • Evaluate and provide feedback relentlessly (about what works and doesn’t) • Sustain leadership to build a culture supportive of innovation and problem-solving Improving the Quality of Public Services APPAM / Moscow Conference

4. “Does developing a culture of performance measurement lead to building a system for performance management?” • We hypothesized that it would: Improving the Quality of Public Services APPAM / Moscow Conference

5. Research Design – Data Collection • Identify US cities that measure their performance • Conduct a search of citywide and agency level public documents for evidence of their performance measures. • Evidence was found in documents of 190 cities, including: • Strategic plans • Budgets • Performance reports • Annual reports on service efforts and accomplishments • Citizen surveys Improving the Quality of Public Services APPAM / Moscow Conference

6. Research Design – Data Analysis • Evaluate the quality and character of their performance measurement systems based on: • Nature of their measures and the quality of their reporting • The frequency of measurement • Benchmarked against other cities, other time periods • Disaggregated on a sub-jurisdictional level (looking for outliers) • Demonstrated evidence of target setting • Analyze difference between cities with mature systems, and those without • Assess whether predicted demographic and environmental conditions are related to these differences Improving the Quality of Public Services APPAM / Moscow Conference

7. Research Design – Interviews • Select most developed systems (27) for qualitative interviews with public officials at city and agency levels • To understand: • The history of their systems • The motivation for their efforts • The way performance data was used,whether they used it to manage, and how (management structures) • The kind of leadership responsible for system elements and the culture around measurement and learning • The impact of the recession on their efforts Improving the Quality of Public Services APPAM / Moscow Conference

8. Robust Performance Measurement is the Result of a Complex Set of Conditions • A city’s performance measurement system is not a simple result of factors suggested by the literature: • Measures of professional management • High levels of social capital • Progressive environments with high levels ofpoliticalengagement • Demographic and environmental characteristics Improving the Quality of Public Services APPAM / Moscow Conference

9. Few Cities Use Performance Data to Manage (even the most mature) • Only 5 of 24 cities demonstrated the use of data to manage for performance • Longevity is dependent on the leadership of a performance champion • Changing leadership and level of resources committed pose regular threat to systems • Few cities go beyond agency level to measure citywide • Cities seldom audit data to insure its integrity • Few cities have developed a true culture of learning and measurement necessary to imbed practices and internalize values that sustain systems Improving the Quality of Public Services APPAM / Moscow Conference

10. Performance Driven Change is Highly Constrained • works bestwhen: • Managerial autonomy is allowed, usually at the agency level • Leadership is strong and stable • Learning culture is built with imbedded routines • Managerial discretion with in the organization is promoted • External support is available Improving the Quality of Public Services APPAM / Moscow Conference

11. Have we Oversold the Performance Movement? • Our findings do not support the view that having a culture of measurement leads inevitably to a performance managed system • Significant accomplishments can be identified – as in the concrete accomplishments of police departments and cities using other CompStat-or balanced scorecard type systems • However, even cities with exemplary measurement systems fail to move naturally to developing a learning organization and building the management structures and routines to improve performance • Performance managed systems have not yet systematically demonstrated their cost-effectiveness Improving the Quality of Public Services APPAM / Moscow Conference

12. Characteristics of Cities that Measure Performance Racial Characteristics Political Environment Region Form of Government

13. Performance Measurement Maturity Measures

14. Performance Measurement Maturity Measures Improving the Quality of Public Services APPAM / Moscow Conference