Using Performance Measurements to Evaluate Service Delivery and Resources

1. Using Performance Measurements to Evaluate Service Delivery and Resources Proposed Suggested Guidelines for Voluntary Reporting, Service Efforts and Accomplishments Reporting The opinions expressed in this presentation are those of Mr. Bean. Official positions of the GASB are established only after extensive public due process and deliberation.

2. SEA Reporting—Scope • What the project is: • Focus on voluntary reporting • Focus on suggested guidelines • Focus on clarifying GASB’s role • What the project is not: • Establishing performance measures • Establishing performance benchmarks • Establishing reporting standards • Requiring SEA reporting in any form

3. GOVERNMENT PERFORMANCE MANAGEMENT SYSTEM Sole Focus Of GASB Efforts 3

4. Suggested Guidelines • Suggested Guidelines for Voluntary Reporting, SEA Performance Information composed of three parts: • Four essential components of an effective SEA report • Six qualitative characteristics that are appropriate for reporting SEA performance information • A discussion of how to effectively communicate SEA performance information

5. Suggested Guidelines • Suggested guidelines, although voluntary, are intended to assist governments in improving the quality of their reported SEA performance information • Traditional financial statements provide information about fiscal and operational accountability but not the degree to which the government was successful in helping to maintain or improve the well-being of its citizens by providing services

6. FOUR ESSENTIAL COMPONENTS Provide guidance to assist preparers of SEA reports in effectively communicating SEA performance information to users • Purpose and Scope • Major Goals and Objectives • Key Measures of SEA Performance • Discussion and Analysis of Results and Challenges

7. ESSENTIAL COMPONENTS AND USER NEEDS

8. PURPOSE AND SCOPE • Purpose- • Why a government is publishing the report • Intended audience of the report • What the SEA performance information is intended to communicate • How the reported SEA performance information can assist users in assessing performance and making decisions – Are they operating efficiently and effectively? • How SEA performance information may be used for assessing resource allocation decisions and whether these decisions are consistent with the goals and objectives

9. PURPOSE AND SCOPE • Scope- • What departments, agencies, programs, or services are included • Reasons why they were chosen for inclusion • Period covered by the report • Date the report was issued • Ways to contact the government to obtain more information • Identifies the level of detail within the SEA report and the reasons why a government has decided to report at that level • * Overview─city as a whole • * More detailed level─by neighborhood • What level of assurance, if any, was obtained on the SEA performance information

10. MAJOR GOALS AND OBJECTIVES • Major Goals and Objectives- • State the major goals and objectives of the programs and services being reported or what those programs or services are intended to accomplish • State the sources of the major goals and objectives • Explain the link between the major goals and objectives and the measures being reported

11. MAJOR GOALS AND OBJECTIVES *Also recognizes that depending on users needs, SEA performance information may be presented in other ways, such as by themes, which require a government to integrate results across the organizational structure.

12. KEY MEASURES OF SEA PERFORMANCE • Key Measures of SEA Performance- • Limited number of measures to focus on information that is essential for assessing the level of performance of programs and services in comparison to what it had planned to achieve • Provide users with enough information to develop their own conclusions without overwhelming them • The number and type of key measures may vary depending on the level of reporting • Key measures can be identified based on the following questions:

13. KEY MEASURES OF SEA PERFORMANCE • Key Measures of SEA Performance- • Key measures can be identified based on the following questions: • Does the measure report on results for a major goal or objective? • Does the measure address an issue receiving considerable public discussion? • Does the measure report information considered important by elected officials, citizens, or experts in the field? • Does the measure report information about a program or service that accounts for a large proportion of a government’s use of resources? • Does the measure report information about a program or service that affects a large proportion of the population or has potentially significant economic, social, or environmental effects?

14. DISCUSSION AND ANALYSIS OF RESULTS AND CHALLENGES • Discussion and Analysis of Results and Challenges- • Provides an objective explanation of the results being reported • Highlights the major and critical results being reported─both positive and negative • Includes managements understanding of the reasons why the actual results differ from the expected or intended results • Addresses currently known facts and circumstances that could affect results in the future

15. DISCUSSION AND ANALYSIS OF RESULTS AND CHALLENGES • Discussion and Analysis of Results and Challenges- • Discusses the major challenges a government is facing in achieving its goals and objectives • Discusses the consequences (both intended and unintended), and the results (positive or negative) of providing services

16. SIX QUALITATIVE CHARACTERISTICS Provide further guidance in the application of the essential components—assist users in comprehending and assessing government programs and services • Relevance • Understandability • Comparability • Timeliness • Consistency • Reliability

17. QUALITATIVE CHARACTERISTICS AND USER NEEDS

18. RELEVANCE • In order for information presented within an SEA report to be relevant, it needs to: • include information that is capable of making a difference in a user’s assessment of performance (a problem, condition, or event) • have a close logical relationship to the purpose for which it is intended to be used • The relevance of SEA performance information depends on the various user needs for assessing performance and making decisions • For example- Consider the appropriate level of aggregation or disaggregation necessary to represent the performance of different geographical areas

19. RELEVANCE • Determining if information is relevant can be based on the following questions: • Does the SEA performance information present the major goals and objectives of the programs and services being reported? • Does the SEA performance information provide a basis for assessing the level of accomplishment of the major goals and objectives? • Do the major goals and objectives relate to concerns that are important to users?

20. UNDERSTANDABILITY • In order for information presented within an SEA report to be understandable, it needs to: • be expressed simply and clearly • include explanations and interpretations to help users comprehend the information • be communicated in different forms and at different levels of detail so that the information can be understood by those who may not have a detailed knowledge of a government’s programs and services • Governments need to obtain feedback from users of an SEA report in order to enhance the understandability of the reported SEA performance information

21. COMPARABILITY • In order for information presented within an SEA report to be comparable, it needs to: • provide a basis and context for users to assess SEA performance • provide users with the ability to make an assessment of whether the reported SEA performance information is improving, deteriorating, or remaining the same • The types of comparative information reported may depend on issues such as the availability of reliable and relevant information, the purpose of the report, and the needs of the users.

22. COMPARABILITY • Comparable information may include: • time series comparisons─compares results of 2 or more periods • No basis for assessing whether results are at an acceptable level • comparisons with established targets─compares to results anticipated • Targets can be too high or too low • comparisons to industry standards • comparisons to other similar governments • Significant differences in measures or circumstances need to be noted and understood by users

23. TIMELINESS • In order for information presented within an SEA report to be timely, it needs to: • be provided to users before it loses its value for assessing accountability and affecting decisions • Timeliness alone does not make information useful, but the passage of time generally diminishes its usefulness.

24. CONSISTENCY • In order for information presented within an SEA report to be consistent, it needs to: • include measures reported in the same way over time • can compare performance and monitor trends over several years • If SEA performance measures are modified or replaced or the manner of presentation is changed, then it is important to communicate to users that a change has taken place and the reasons for the change.

25. RELIABILITY • In order for information presented within an SEA report to be reliable, it needs to: • contain information that is verifiable to provide assurance that the information in an SEA report would be replicated by independent evaluators using the same measurement methods • information needs to be derived from systems producing verifiable data • independent verification is ideal • system control reviews conducted internally • program staff or director evaluations • contain information that is objective and has therefore not been modified by the government to either overstate or understate positive or negative results

26. RELIABILITY In order for information presented within an SEA report to be reliable, it needs to: • contain information that is comprehensive in coverage or thorough in its presentation of the measures • significant data is not omitted from the information that is necessary to faithfully represent major results • nothing is included that would cause the information to be misleading • contain information that is a faithful representation of the results or represent what actually occurred

27. EFFECTIVE COMMUNICATION Provide further guidance on the effective communication of SEA performance information • Intended Audiences • Forms of Communication • Multiple Levels of Reporting

28. INTENDED AUDIENCES • In order to effectively communicate SEA performance information to users, the intended audiences need to be considered • citizens • elected officials • appointed officials • investors and creditors • others • The types of decisions made by potential audiences influence the information that they need in an SEA report. • Potential audiences need different types of information and understand and analyze SEA performance information in different ways.

29. INTENDED AUDIENCES • Involving users in the process of identifying: • what SEA performance information is valuable and interesting to them • how they prefer to receive the SEA performance information • may assist preparers in effectively communicating their results in an SEA report.

30. FORMS OF COMMUNICATION • The effective communication of SEA performance information is, in part, dependent on the forms used to communicate that information • Forms of communication: • printed material mailed to households and businesses • portable document format (PDF) files on the Internet • extensible business reporting language (XBRL) enhanced files on the Internet • presentations made in person • articles in the print media • news segments in the electronic media

31. MULTIPLE LEVELS OF REPORTING • An SEA report that communicates effectively contains different levels of detail so that users can find their appropriate and desired level of detail • Levels of detail can be presented in many ways • within one document • several separate documents organized by service area

32. MULTIPLE LEVELS OF REPORTING • An SEA report needs to be organized in a hierarchical structure that proceeds through levels (layers) from overview to detail, and needs to have clearly identified links between the levels. • overview • introductory summary • information about specific programs and services as a whole • performance data that is broken down to provide measures of performance about specific strategies or activities within programs and services

33. Questions? Web Sites—www.gasb.org and www.seagov.org