Going from Good to Great! Understanding and Using Performance Measurement

1. Going from Good to Great!Understanding and Using Performance Measurement Energy Assistance Training Conference Springfield, Illinois June 4, 2014

2. LIHEAPGrantee Initiated Performance MeasurementWeatherizationRecommendations for Subgrantee Performance Measurement 2

3. Performance Measurement is Cool (60s), Radical (80s), Awesome (90s), Sick! (current) …and a lot of different types of people/organizations are doing it 3

4. ILLINOIS EXAMPLERose is the type of player who is defined by more than his statistics. His impact on the offense goes beyond what he contributes to box scores. Before his injury issues started escalating in 2012, the Bulls were the third most efficient offense in the NBA, per NBA.com/STATS.Modern analytics have helped to shape what offenses do. By tracking shot locations, it’s become apparent that some areas of the court are more critical than others. There are two optimal areas for scoring: inside the restricted area (that little semicircle under the rim) and outside the three-point line….. By Kelly Scaletta, Featured Columnist Aug 15, 2013 http://bleacherreport.com/articles/1739774-what-is-the-derrick-rose-effect 4

5. Why would an NBA Player hire a personal Data Analyst? 5

6. Why would an NBA Player hire a personal Data Analyst?To get better!To sit beside him/her during contract negotiations! 6

7. Why would the LIHEAP Program or the Weatherization Program get involved with Performance Measurement? 7

8. Why would the LIHEAP Program or the Weatherization Program get involved with Performance Measurement?To get better!To furnish concrete data and statistics for budget discussions! 8

9. Performance Metrics FrameworkProgram Evaluation vs. Performance Measurement 9

10. Comparison 10

11. Performance Metrics ExampleA Practical Example 11

12. Fred’s Policy Issue 12

13. EVALUATION RESULTS 13

14. Proposed Intervention • Couch Potato Ratio – From 5.0 to 3.0 14

15. performance measurementoutcomes 15

16. in-depth data analytics 16

17. Updated Intervention • CPR – From 3.0 to 1.0 • PPM – From 5.0 to 0.5 17

18. tracking cpr and ppw are both needed to maintain dpw 18

19. Comparison 19

20. Performance Metrics ExampleNational LIHEAP Program Grantee Initiative 20

21. Proposed LIHEAPPerformance Measures • Benefit Targeting – Make sure that highest burden households get the highest benefit. • Service Restoration – Document annually the number of households where the program restores energy service. • Service Loss Prevention – Document annually the number of households where the program prevents the loss of energy service. 21

22. Rationale for Performance Measures • Benefit Targeting • The law says that benefits should be targeted. • Research shows that higher burden households have greater Energy Insecurity • Service Restoration/Prevention • Service loss presents the single greatest risk to clients • Preventing service loss is far less expensive than restoring service 22

23. Section 4: Explaining Energy Burden Measure Data What are we reporting? Why does it matter? Measure 2: Benefit Targeting Index Example In 2005, high burden households received the same benefit as the average household. [Index Score: 100] In 2009, high burden households received a benefit 18.6% higher than the average household. Does the LIHEAP program furnish higher benefits to high burden households? Not in 2005. But, in 2009 the program did pay higher benefits to high burden households.

24. Grantee Responsibilities • Benefit Targeting • Work with energy vendors to collect expenditure data for clients • Service Restoration/Prevention • Work with subgrantees to collect reliable and meaningful data on restoration/prevention 24

25. Subgrantee (Agency)Responsibilities • Benefit Targeting • Work with new procedures for assigning benefits • Helping to identify clients where energy burden doesn’t tell the whole story • Service Restoration/Prevention • Collect reliable and meaningful data from clients 25

26. Next Steps for LIHEAPPerformance Measurement • Benefit Targeting • How did benefit targeting change outcomes? • What further revisions are recommended? • Service Restoration/Prevention • What initiatives are undertaken to move from restoration to prevention • What are the impacts of these initiatives 26

27. Comparison 27

28. Performance Metrics ExampleRecommendation for the Illinois Weatherization Program 28

29. Weatherization EvaluationComparison #1 29

30. Weatherization EvaluationComparison #2 30

31. Special Thanks to ….Greg Dalhoff – Dalhoff Associates for Iowa Program ResultsScott Pigg – Energy Center of Wisconsin for Wisconsin Program Results 31

32. Proposed Approach • Evaluation – Who is doing best? • Assessment – What are the doing? • Factors – What results in high performance? • Replication – What can other agencies replicate? • Tracking – How can you track progress toward the goal? 32

33. Weatherization EvaluationComparison #3 – IL Agencies 33

34. Explanatory Factors 34

35. Field Inspections • High Priority Measures: Were all wall insulation and attic insulation opportunities addressed? What are the opportunities and barriers? • Furnace Replacements: What share of furnace replacements were required for health and safety? • Measure Prioritization: How were measures prioritized for each home? • Cost per Home: What are the cost implications for changing measure priorities? 35

36. Performance Goals? • Wall Insulation – From 10% to 15% • Attic Insulation – From 60% to 70% • Furnace Replacement – From 39% to 30% • Maintain ALL Health and Safety Replacements • Spending – Limit annual increase to 3% 36

37. Follow-Up • Staff Reports – Successes in and/or barriers to installing attic and/or wall insulation. • Quarterly Report – Installation rates vs. installation targets, including required H&S measures. • Annual Report – Summary statistics AND outcomes of field inspections. • Follow-Up Evaluation – Do changes in performance lead to improvements in savings? 37

38. Summary – Going from Good to Great? 38

39. Evaluation ANDPerformance Measurement 39

40. Performance Measurement is a program management framework that is guided by metrics, but that also takes account of individual circumstances.Larry Dawson - 2014 40

41. Contact David Carroll, 609-252-8010 david-carroll@appriseinc.org APPRISE 32 Nassau Street, Suite 200 Princeton, NJ 08540 41