1 / 25

Practical, Affordable Performance Measurement: Why and How?

Practical, Affordable Performance Measurement: Why and How?. Shelley H. Metzenbaum smetzenbaum@verizon.net www.complianceconsortium.org www.facilityreporting.org. Northeast Environmental Summit September 2005. Overview. Why Measure? How to Measure and Use the Measures? Examples

barbra
Download Presentation

Practical, Affordable Performance Measurement: Why and How?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Practical, Affordable Performance Measurement:Why and How? Shelley H. Metzenbaum smetzenbaum@verizon.net www.complianceconsortium.org www.facilityreporting.org Northeast Environmental Summit September 2005

  2. Overview • Why Measure? • How to Measure and Use the Measures? • Examples • Issues to Consider • Workshop Tomorrow

  3. Why Measure? • Increase Accountability • Learn to Achieve Better Outcomes • Strengthen Democracy

  4. Measurement for Accountability (Program Justification?) • Ethical: Self-Dealing/Fraud (track $ flows to whom for what – accounting & fiscal audits) • Fiscal: Unallowed spending/Waste/Efficiency ($ spent/$ allocated & goods or services bought per input cost) • Performance: Improve social outcomes/reduce risk/more cost-effective (better outcomes; better outcomes at lowest cost)

  5. Measurement to Learn so Achieve Better Outcomes • DETECTION: Detect problems needing attention; progress deserving replication- what works and what doesn’t? • Look for anomalies, unexpected direction, speed ups, slow downs, etc. • Use measurement to trigger focused follow up • DIAGNOSTICS: Why? What is the reason for variation? Is it a problem or lesson? • ASSESSMENT: Did it seem to work? Replicate, expand, or stop and fix?

  6. Goals/Measures to Strengthen Democracy • Cannot Do Everything: Goals/ Targets Communicate: • What You Will Do • What You Won’t Do • Are You Doing Right Thing? Democracies can respond when unhappy with agency choices • How Well Are You Doing It? Measurement

  7. Start with Learning Emphasis of Measurement • Detection, Diagnostics, Assessment • Do this well … Accountability & Democracy will Rise, as well • Practical, Affordable Examples

  8. DETECTION : Look for Pattern Changes to Sense Problems Why the Downturn ? Source: http://www.chesapeakebay.net/status/status_print.cfm?sid=101&print=yes

  9. DETECTION: Volunteer Data -- Look for Unexplained Variations

  10. DETECTION : Look for Pattern Changes to Find Progress Recognize Progress ! Source: http://www.chesapeakebay.net/status/status_print.cfm?sid=101&print=yes

  11. DETECT: Before & After Photos

  12. DETECTION/DIAGNOSTICS: Inspec. Data“Scout & Survey” Detect low comp. rate Diagnose: Is it a problem? Why is it happening? NH, CO

  13. Diagnostics: MA ERP Self-Audit Data 1997 & 1998 DETECT, DIAGNOSE, ASSESS: Self-Reported Practices Trends (97/98) Changes over Time: Progress? Lower than Other Areas: Problem? (10) Leak Checks (18-19) Controls (36i) Emergency Procedures MA ERP

  14. DETECTION and DIAGNOSTICS: Mapping/Segmentation Highlights Problem Areas and Progress 1996 DRY WEATHER BACTERIA Swim Standard Some Failures Failed 1996/97 WET WEATHER BACTERIA

  15. DIAGNOSTICS/ASSESSMENT:Slice & Dice by Categories to Identify Cause of Problems, Progress

  16. Detection: # Complaints(Cleveland Air Pollution) • General Environmental Mgt. – 64 • ISG – 52 • Ford Motor Company – 29 • Kirkwood Industries – 18 • Landfill – 14 • Wabash Alloys -12

  17. DIAGNOSTICS: Complaint Relevance? Source: PA DEP

  18. ASSESSMENT: Pictures ConsolidatedTailings Pile (CO) After Reclamation 1999 Prior to 1989

  19. Lots of Affordable Ways to DETECT, DIAGNOSE, ASSESS • Air and Water Quality Data • Volunteer & govt collected data useful • Water, Energy Supply – Boston Water and Sewer Commission use data to cut water use and costs • Cameras – Texas detects problem air plumes; Rutgers extension helps farmers cut pesticides and costs • Incident and Inspection Data: • Repeat Violators - Accidents • Nature of Non-Compliance – Complaints • Fires

  20. DETECTION, DIAGNOSTICS, ASSESSMENT: Interim Outcome Indicators Ideas Measure Awareness (OR) Information Attention Measure Understanding Comprehension Measure Attitudes Beliefs/Attitudes Measure Motivation Behavior Motivation Measure Behavioral Changes (ERP)

  21. Newspaper Content Analysis, Surveys, Observational Measurement Chemical Industry Trade Press Content Analysis (Hoffman 1999)

  22. STRENGTHEN DEMOCRACY: Specific Goals Clarify What Govt Will Do and What It Won’t • Where: Lower Charles River • When: 10 years • How much: Swimmable • Who: All Sources • Invite public attention • Public can respond

  23. Strengthen Democracy: Make Goals, Progress, Problems Clear Goal Not Met, Past and Future Strategy Described Goal Met; Commitment to New Goal Described

  24. Issues to Consider • Data Accuracy: Match to need. Can be lower for Detection; higher for Rules and Enforcement. For assessment, consider program $ • Project and Measurement Scale: indiv. facility level work OK only if have context/control. Count costs and cost-effectiveness when project level. Start with env. problem. • Comp asstc and P2 activity counts not useful indicator; may have occasional diagnostic value

  25. Tomorrow’s Workshop: Brainstorming Your Specific Measurement Challenges Together 1:30 - 3:30 Shelley H. Metzenbaum smetzenbaum@verizon.net www.complianceconsortium.org www.facilityreporting.org

More Related