National Association of Regulatory Utility Commissioners Winter Committee Meeting ______________________________________

1. National Association of Regulatory Utility CommissionersWinter Committee Meeting_________________________________________________________ Energy Efficiency Program Performance Evaluation Issues from the States Nick Hall TecMarket Works

2. 5 Topics Covered Today • Changing Importance of Evaluation • Need for accuracy – reliability - objectivity • Reliability & Budgeting for Evaluation • Evaluation Approaches Used • Net-to-Gross Issues • Definitions change from state to state • Net savings are difficult to assess • Approaches for crediting savings • Changing way EE is viewed / assessed

3. 1. Changing Importance of Evaluation Evaluation provides decision-support information in a changing environment… • More people understand the true cost of fossil fuel & nuclear power supply and are demanding change. • Energy Efficiency represents the single largest impact potential for greenhouse gas reduction. • Energy efficiency is moving to be the resource of choice. • Public resistance to new plants, including some renewable energy facilities. • ISO’s, Commissioners, ALJs and others are asking for more and more reliable evaluations. Accurate information is essential for Policy Decisions.

4. 2. Reliability & Evaluation Budgeting Several state are worried about the accuracy and reliability of evaluation results • Accuracy & reliability are controlled by the evaluation budget & the evaluation timeline. • States are struggling with evaluation budget decisions. • Some are legislating budgets without understanding the effects. • Currently some states are setting evaluation budgets below the level of accuracy desired.

5. 2. Reliability & Evaluation Budgeting Relationship between funding and results Strong Funding – Right Approach Weak Funding – Right Approach Weak Funding – Wrong Approach Strong Funding – Wrong Approach

6. 2. Budgeting for Evaluation NY 2% Moved from 1% ILL 3% Set in legislation WI 4% Moved up from 3% MI 3-5% Proposed - Now being discussed KY 5% Proposed - Now being discussed OH 5% Proposed - Now being discussed NC 5% Proposed - Now being discussed SC 5% Proposed - Now being discussed IN 5% Proposed - Now being discussed CA 8% Moved up from 4.25%

7. 3. Evaluation Approaches Used Impact Evaluation – Energy Impacts Majority of Studies Fall Under 5 General Approaches*. States pick and choose according to budget and reliability needs. • Engineering Estimates • Statistically adjusted Engineering Estimates • Building Meter/Billing Analysis • Measure Level Metering & Monitoring • Building Modeling and Simulations * See California Evaluation Framework/Protocols for others.

8. 3. Evaluation Approaches Used Impact Evaluation – Energy Impacts Reliability Issues • Identifying and setting the baselines • On-site confirmation / metering / assessments • Sample Sizes • Representativeness of the Sample • Type of metering or monitoring used • Accuracy of self-reports

9. 4. Net to Gross Issues Gross savings tend to be defined the same way. Ex Ante - Gross • Ex ante savings x tracking system measure installs. • Ex ante savings x measure installs x verification rate. Ex Post - Gross • Ex post savings x measure installs x verification rate. Net savings tend to be defined in different ways. • Net Savings = Gross Savings +/- Net Adjustments

10. 4. Net to Gross Issues Net savings definitions and measurement approaches are not consistent across states, and sometimes across studies within a state. Different definitions from state to state • Freeriders (10% to 60% of the program-claimed savings) • Participant spillover (5% to 40% of the program-claimed saving) • Short term non-participant spillover - market effects (5% to 20% of the program-claimed savings) • Long-term non-participant spillover market effects (40% to >500% of the program-claimed savings) Study results are inconsistent and not comparable.

11. 4. NTG Definitions Used NY Gross – freeriders Can adjust for PS & ME ILL Gross – freeriders + part spillover Not yet agreed WI Gross – freeriders + ME As documented in eval MI Not yet determined Not yet agreed KY Gross – freeriders + adjustments As documented in eval OH Gross – freeriders + adjustments As documented in eval NC Gross – freeriders + adjustments As documented in eval SC Gross – freeriders + adjustments As documented in eval IN Gross – freeriders + adjustments As documented in eval CA Gross – freeriders Now looking at options

12. 5. Changing way EE is viewed / assessed State are considering changes that will impact EE evaluation and cost effectiveness assessments. • Elimination of policy-set life-cycle periods (EUL) • Valuing energy saved at market prices • Valuing greenhouse gas at total societal costs, not traded values • Setting renewable energy as the cost alternative to EE in cost comparison approaches (i.e. avoided costs = RE) • Discontinuing discounting of future savings. • EE as a profitable energy supply choice.

13. Thank You Nick Hall TecMarket Works 165 West Netherwood Road Suite A, 2nd Floor Oregon, Wisconsin 53575 608 835 8855 NPHall@TecMarket.net