The Basics of Performance Measuring & Outcomes

1. The Basics of Performance Measuring & Outcomes

2. What is it? • Performance measurement is the process of collecting, analyzing and/or reporting information regarding the performance of an individual, group, organization, system or component. • There are 2 basic measures of performance • Qualitative – what is the quality/does it achieve a desired outcome • Quantitative – how many – also referred to as outputs, units

3. Why Do We Do It? • To ensure best use of resources • Increase accountability • Competition for funding • Identify practices that need improvement or to be discontinued

4. Qualitative v QuantitativeWhich is Better? • Both and neither – one is useless without the other. • McDonald’s has served billions of burgers but they don’t have a Michelin Star. • The French Laundry has 3 Michelin Stars but only seats 60. (served 50,000 people in 40 yrs)

5. Why Measure Qualitatively • Need to know that the program is having a positive impact on the problem. • Generally this sort of measuring is called ‘outcome measuring’. • Can include self-reporting such as client satisfaction evaluations. • Can also include methods such as pre and post test and longitudinal studies.

6. Why Measure Quantitatively • Still need to know that the investment is a priority • Compelling evidence of need • Determining capacity to serve • Future planning

7. What is an Outcome? • Benefits for a participant during or after their involvement with a program. Can relate to knowledge, skills, behavior, condition or status. • The actual ‘outcome’ is simply a direct statement of hoped for client condition. It does not contain numbers. Parents read to their preschoolers more often.

8. Important terms • Inputs – the items it takes to run a program, i.e. desks, staff, supplies, laws, time, space. • Activities – what the program does, i.e. monitor visits, shelter victims, advocate. • Tools – the item/method you use to collect information, i.e. client surveys. • Outputs – the number of clients you serve. This is simply counting clients and is expressed in whole numbers, i.e. 700 crisis calls, 97 group members, 120 residents.

9. More Important Terms • Targets – the number of persons in the program you hope will achieve the result, usually expressed in a percentage but in the future tense, i.e. 89% of participants will be more educated. • Indicators – the number of persons who achieved the outcome, usually expressed in a percentage and in the past tense, i.e. 90% of participants were more educated. • Benchmarks – targets based on multiple years of outcome study.

10. Where to Focus • The 3 most commonly used terms are; • Outcome – a sentence describing the hoped for result. • Target/Goal – the percentage of people you hope achieve that result. • Indicator/Result – the percentage of people who did achieve that result.

11. Mission Based Outcomes • Mission - To offer and promote quality programs, services, and resources to those affected by domestic violence and sexual assault; to provide a safe environment for victims in such crisis situations; and to educate the community on domestic violence and sexual assault. • Outcomes (a sample) • Residents will feel safe at X,Y,Z shelter. • Children will feel safe visiting. • Participants will gain knowledge. • Offenders will gain empathy for victims. • Conflict related to visits will lessen.

12. FVPSA Mandated Outcomes • As a result of contact with DV program, 65% or more of DV survivors will have strategies for enhancing their safety. • As a result of contact with DV program, 65% or more of DV survivors will have knowledge of available community resources.

13. Why These • Study group in 2007 • Eleanor Lyons & Cris Sullivan • These outcomes were chosen because they indicate improved circumstances for victims and demonstrate that communities have become stronger and safer. • Improving knowledge leads to safety and well-being over time.

14. Methods of Information Gathering • Self Report - acceptable in DV because lessening crisis is the goal • Pre and post test – difficult, time consuming, not good in an environment where a person has been less focused due to trauma

15. Particular Outcome Problems in Emergency Services • Generally client satisfaction is not considered an outcome. However in emergency services and basic needs the client contact is so short term that it is impossible to measure any long term effect on their life. In these circumstances client satisfaction is acceptable as the closest thing possible to an outcome measure. • Client satisfaction measures could include, does the client feel better, were the staff helpful, would you recommend the service. (Measuring Program Outcomes, A Practical Approach, UWA. pg. 19)

16. When & How • Surveys typically given at exit. • Can be done in shelter or supportive services. • Ideally anonymous but can be asked in person or on phone. May be necessary for LEP persons or low literacy persons. • Like everything, must be voluntary. • Can be imbedded in a larger document.

17. Remember to Aggregate

18. Things to Keep in Mind • People familiar with outcomes often refer to the Indicator Statement as the outcome. Although this is technically incorrect it’s a simplified way to communicate to the public. • Targets and indicators can be communicated in whole numbers as well as percentages. • Although it is optimal to measure every participant this is rarely possible for a number of reasons including the client does not finish the program or refuses to participate in the outcome process.

19. Things to Keep in Mind • Only measure things for which you can actually be responsible. • There is no magic number of outcomes, some programs could have 1 while others could have an infinite number. • Your outcomes should directly link back to your mission statement. • Not everyone can have short, medium and long term outcomes.

20. What Outcomes ARE NOT • Outcomes ARE NOT your demographic stats, age, race, income of clients. • Outcomes ARE NOT your service logs. • Outcomes ARE NOT your monthly stat card. • Outcomes ARE NOT Head Injury, History of Abuse, Departures or Evaluations. • Outcomes may be one item located within the above.

21. Beth Meeks, MS ext. 106 Tonia Moultry, MA ext. 127 202-543-5566 bmeeks@nnedv.org tmoultry@nnedv.org