Performance Measurement Tools for Justice Information Technology Projects

1. Performance Measurement Tools for Justice Information Technology Projects Facilitated by: Center for Society & Law & Justice Faculty

2. Performance Measurement Tools for Justice IT Projects A CSLJ ongoing projectFunded by the Bureau of Justice Assistance Grant # 2002LD-BX K002

3. Project Background and Purpose This project is designed to identify and validate an inventory of performance measures appropriate for justice information technology projects and develop field-friendly performance measurement tools.

4. Fact:There is an increasing focus on measuring performance in both government and industry.

5. Fact:The demand for performance measures in law enforcement andcriminal justice settings is high.

6. QuestionWhy do you think there is currently a high demand for performance measurement in law enforcement and criminal justice settings?

7. Some Reasons Why There is a High Demand for Performance Measures • Performance measurement is increasingly being mandated • Tight budgets • There is a need to justify expenditures

8. ProblemEven though the demand for performance measures in law enforcement and criminal justice settings is high, technology integration projects are not in advanced stages of performance measurement.

9. QuestionWhat are some of the unique challenges or “road blocks” to performance measurement?

10. Some Unique Challenges or “Road Blocks” to Performance Measurement • Many people are unfamiliar with performance measurement or reluctant to use it. • Measurements are imposed from the outside. • Most measures are not outcome oriented. • Existing measures are not sufficient.

11. A Definition of Performance Measures Performance Measures: a particular value or characteristic used to objectively measure results. This means using valid and reliable indicators. Valid: Do they truly measure what they are intended to measure? Reliable: Do they consistently do so ? (Time A, Time B, Time C, etc.)

12. A Useful Acronym Valid performance measures are S pecific Measurable Accountable Results-Oriented Timebound SMART

13. The Two “O’s” of Performance Measurement Performance measurement can be defined as the process of routinely measuring the outputs and outcomes produced by a project, thereby allowing one to assess the effectiveness of project investments and activities.

14. Outputs and Outcomes:The Logic Model Approach to Performance Measurement “A program logic model … provides a roadmap of your program, highlighting how it is expected to work, what activities need to come before others, and how desired outcomes are achieved.” Source: WK Kellogg Foundation Evaluation Handbook (1998)

15. OUTCOMES TheLogic Model Approach INPUTS ACTIVITIES OUTPUTS INPUTS are your resources OUTPUTS are the product of an activity OUTCOMES are the consequences of the program/initiative efforts -- changes in conditions, attitudes or behavior of individuals or outcomes for agencies and communities, not what the program or initiative itself does.

16. OUTCOMES TheLogic Model Approach INPUTS Example: Money, Personnel, Hardware, Software ACTIVITIES Example: Mobile Data Terminals in Patrol Cars OUTPUTS More Rapid ID of Suspects Increased public safety INPUTS are your resources OUTPUTS are the product of an activity OUTCOMES are the consequences of the program/initiative efforts -- changes in conditions, attitudes or behavior of individuals or outcomes for agencies and communities, not what the program or initiative itself does.

17. OUTCOMES TheLogic Model Approach INPUTS ACTIVITIES OUTPUTS • When creating performance measures, you can use the StrategicPlan and the TheoryofChange to link together the: • Inputs • Outputs • Outcomes

18. The Logic Model Approach Helps You to Define Your Theory of Change • A theory of change is a statement of how your program intends to proceed from initial outputs to produce long-term outcomes. • Once you articulate and reach consensus on your theory of change, you can then move on to identify those vital, few performance measures that are aligned with your organization’s mission, goals and objectives.

19. Thinking Through a Logic Model and Articulating a Theory of Change Goals In order to solve these Problems Activities We will do the following activities and Events Targets For these people and for this amount of time Theory of Change This activity will lead to changes in these factors __________, which in turn will lead to solving these problems Initial/Short& Intermediate Term Outcomes We will know these changes have occurred if: ___________ Long Term Outcomes We will know we are reaching our goals if: ___________

20. A Logic Model and Theory of Change Example Goals In order to solve these Problems Activities We will do the following activities and Events Targets For these people and for this amount of time Theory of Change This activity will lead to changes in these factors __________, which in turn will lead to solving these problems Initial/Short& Intermediate Term Outcomes We will know these changes have occurred if: ___________ Long Term Outcomes We will know we are reaching our goals if: ___________

21. Developing a Theory of Change Recommended Process for Surfacing & Articulating a Theory of Change • Use a Focus Group Format • Start with Long-term Outcomes • Work Backwards Toward Initial Activities • Map Required Existing Resources • Reconcile Multiple Theories of Change Adapted from J.P. Connell et al., 1995, “New Approaches to Evaluating Community Initiatives.” Aspen Institute.

22. The Logic Model/Theory of Change Method for Developing Performance Measures

23. Summary: Why Measure Performance? • To monitor project implementation • Because IT projects are high-risk • To demonstrate improvements • To correct problems and make adjustments • To ensure accountability • To illustrate progress and justify additional funding for your integration effort

24. Some Suggested Steps… • Invest in Performance Measurement • Carefully select and form a team • Identify the Business Process to be mapped • Specify the logic model using the Theory of Change method • Decide the Audience/Level of the Measures • Keep in mind that the Model should be Plausible, Doable and Testable • Reconcile conflicting assumptions among stakeholders • Revise and Refine model and corresponding measures as needed