National Action Plan Model Energy Efficiency Program Evaluation Guide

1. National Action Plan Model Energy Efficiency Program Evaluation Guide Cynthia Cummis, US EPA April 11, 2007 NAESB

2. Topics Objectives Model Evaluation Guide description Advisory Committee Why a program evaluation guide Programs addressed and audience Overview of scope and contents Process and schedule

3. Objectives The National Action Plan supports use of energy efficiency as a resource equivalent to generation. Therefore, it is important that the benefits of energy efficiency programs can be accurately documented with minimal transaction costs. The objectives of this project are: Create a Model Guide that states, cities, utilities, private companies, and others can use as a framework to define their own “institution-specific” or “program-specific” evaluation requirements Build consensus on basic evaluation approaches and definitions in order to promote consistent evaluations across different jurisdictions To support these objectives, the Guide will be widely promoted and industry organizations will be asked to endorse the Guide as a best practices document. Benefits include energy savings, emissions avoidance, economic and security benefits Having effective evaluation, measurement and verification (EM&V) infrastructures that document the energy savings and environmental benefits of energy efficiency projects and programs is critical for the success and energy efficiency and emission reduction programs. Effective measurement and verification for energy efficiency is important to many of the Action Plan recommendations. This project will create a model guide to document the energy and emission savings associated with energy efficiency programs implemented by states, cities, utilities, or private companies. This model guide will define a general approach to evaluation and options for preparing the savings calculations. Each jurisdiction (states, utilities, etc.) can select an option or options, as well as a level of rigor, to define their own jurisdiction-specific program evaluation requirements. Entities that are implementing energy efficiency programs can use the model guideline to establish their own requirements in a manner that is consistent with other entities’ requirements and that follow best practices. Having best practices and consistent EM&V based reports, that document energy efficiency benefits, support the adoption, continuation and expansion of these programs. The guide will be policy neutral. Benefits include energy savings, emissions avoidance, economic and security benefits Having effective evaluation, measurement and verification (EM&V) infrastructures that document the energy savings and environmental benefits of energy efficiency projects and programs is critical for the success and energy efficiency and emission reduction programs. Effective measurement and verification for energy efficiency is important to many of the Action Plan recommendations. This project will create a model guide to document the energy and emission savings associated with energy efficiency programs implemented by states, cities, utilities, or private companies. This model guide will define a general approach to evaluation and options for preparing the savings calculations. Each jurisdiction (states, utilities, etc.) can select an option or options, as well as a level of rigor, to define their own jurisdiction-specific program evaluation requirements. Entities that are implementing energy efficiency programs can use the model guideline to establish their own requirements in a manner that is consistent with other entities’ requirements and that follow best practices. Having best practices and consistent EM&V based reports, that document energy efficiency benefits, support the adoption, continuation and expansion of these programs. The guide will be policy neutral.

4. Evaluation Guide Description This Model Guide is intended to: Define a policy neutral, state of the art approach to evaluation for calculating energy savings and avoided emissions Provide guidance for preparing jurisdiction specific program evaluation protocols that are consistent with the Model Guide Provide basic process and technical guidance on evaluation issues and requirements for efficiency resource programs Establish common evaluation definitions and terminology Provide a single, concise reference document for program evaluation resources currently available in the public domain Provide basic process, technical and policy guidance on evaluation issues and requirements for efficiency resource programs Provide a model that can be used by individual jurisdictions (states, utilities, etc.) to establish their own, jurisdiction and program specific, evaluation requirements that are consistent in approach to other jurisdictions’ Defines a state of the art energy efficiency program evaluation approach Can be endorsed by stakeholders as the Primary Source Document for program evaluation Can be applied consistently across different states, cities, utilities, and perhaps private companies Provides guidance on key issues associated with documenting energy savings and emissions avoidance Can be applied to programs that have different policy objectives . It would define various options for doing program evaluation, a standard structure (define baseline, define program case, compare the two, adjust for net-to-gross, etc.), define terms and define and give advice on key issues. It would also provide a section on calculation emissions reductions from calculated energy savings. Provide basic process, technical and policy guidance on evaluation issues and requirements for efficiency resource programs Provide a model that can be used by individual jurisdictions (states, utilities, etc.) to establish their own, jurisdiction and program specific, evaluation requirements that are consistent in approach to other jurisdictions’ Defines a state of the art energy efficiency program evaluation approach Can be endorsed by stakeholders as the Primary Source Document for program evaluation Can be applied consistently across different states, cities, utilities, and perhaps private companies Provides guidance on key issues associated with documenting energy savings and emissions avoidance Can be applied to programs that have different policy objectives . It would define various options for doing program evaluation, a standard structure (define baseline, define program case, compare the two, adjust for net-to-gross, etc.), define terms and define and give advice on key issues. It would also provide a section on calculation emissions reductions from calculated energy savings.

5. Advisory Committee The Advisory Committee will provide input on the content and scope of the document as well as the recommended audience. The committee will also help with outreach and encouragement for jurisdictions to endorse the Guide. The current members of this group are: Commissioner Dian Grueneich, California PUC, co-chair Diane Munn, EEI, co-chair Chris James, Connecticut Department of Environmental Protection Rick Leuthauser, MidAmerican Energy Company Jan Schori, Sacramento Municipal Utility District Peter Smith, NYSERDA Others are welcome to join, contact ??????????Others are welcome to join, contact ??????????

6. Why a Program Guide There are widely recognized protocols for the measurement and verification (M&V) of energy savings from single projects (e.g., International Performance Measurement Verification Protocol) However, similar widely accepted protocols or guidance documents for measuring energy savings from programs do not exist M&V protocols do not address issues unique to program evaluation Use of good statistical samples for sites and expansion of sample analysis results to the program population Use of inexpensive and readily available site energy consumption and program tracking data Regression analysis of energy use/savings from multiple sites Analysis of attribution (free ridership, spillover, program effects) - net to gross ratios

7. Why a Program Guide - New Imperatives for Consistent Program Evaluation New state policies for reducing and measuring greenhouse gas (GHG) emissions With federal policies likely to follow Markets for peak load reductions that allow bids from energy efficiency programs Such as New England ISO Program Increasing interest in Energy Efficiency Portfolio Standards

8. Programs Addressed and Evaluation Focus Program types addressed: Primary focus - resource acquisition, downstream efficiency programs Market transformation, codes and standards, and upstream programs will also be referenced Evaluation focus Primary focus - impact evaluation (kWh, kW, therm savings) Will describe, but not provide guidance on: Process and market evaluations Potential studies Cost-effectiveness evaluation

9. Intended Audience Program evaluation managers and policy decision makers looking for guidance (e.g., a roadmap) on process and key issues relating to: Documenting energy savings (and to lesser degree cost-effectiveness) Documenting avoided emissions Program designers looking to understand how their programs will be evaluated Energy efficiency community looking for: A credible reference document that provides guidance, best-practices, and links to additional resources Common definitions and terminology Approaches to quantifying avoided emissions from energy efficiency programs

10. How the Guide Can Be Used Each jurisdiction (states, utilities, companies, etc.) will define their own jurisdiction-specific program evaluation requirements based on the Model Guide. The Model Guide itself will be policy neutral. The Model Guide will be a resource that lays out various evaluation issues that need to be addressed for each jurisdiction’s specific requirements and constraints (e.g., budgets, uncertainty tolerance, net to gross issues, time period of evaluation, etc.) The Model Guideline will allow for consistency in counting savings among jurisdictions’ energy efficiency programs, which can be particularly important for climate change mitigation programs.

11. Evaluation Approach The general approach of the model guideline can be described as follows: Gross program energy savings are determined using one of the following options: A sample of project savings are determined Statistical analyses of large volumes of energy billing data Quality assurance reviews of existing documentation Gross Program savings are then converted to net energy savings using a range of possible considerations (e.g., free rider corrections) Emission factors are derived and applied to net savings in order to determine avoided emissions Provide description of possible approaches Policy context matters

12. Expected Contents 1. Executive Summary 2. Introduction to energy efficiency 3. Scope and uses of this guide 4. Introduction to evaluation 5. Overview of evaluation approach and options 6. Decision Tree for preparing a jurisdiction specific program guideline – “bringing it all together” 7. Discussion of evaluation issues and cost-effectiveness 9. Calculating gross and net energy savings 10. Calculating avoided emissions 11. Confirming persistence of savings 12. Reporting evaluation results Appendix A - Terminology Appendix B – Uncertainty Appendix C – Resources

13. Process and Schedule Document will rely primarily on existing protocols Steve Schiller is the principal author with support of Technical Group: Derik Broekhoff, World Resources Institute Nick Hall, Tec Market Works M. Sami Khawaja, Quantec David Sumi, PA Consulting Laura Vimmerstedt, National Renewable Energy Lab The Guide will be sent out for review by a wide range of evaluation professionals and industry participants Will reach out to additional organizations for endorsement Draft available for review this summer

14. For Further Information Contact: Steve Schiller at steve@schiller.com or Niko Dietsch, US EPA, at Dietsch.Nikolaas@epa.gov