1 / 42

Evaluation of Energy Efficiency Programs: Adjustments to Energy Savings Edward Vine

Evaluation of Energy Efficiency Programs: Adjustments to Energy Savings Edward Vine California Institute for Energy and Environment Center for the Study of Energy Markets (CSEM) Policy Conference Sacramento, CA December 9, 2008. Outline of Talk. History of Energy Efficiency in California

gella
Download Presentation

Evaluation of Energy Efficiency Programs: Adjustments to Energy Savings Edward Vine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of Energy Efficiency Programs: Adjustments to Energy Savings Edward Vine California Institute for Energy and Environment Center for the Study of Energy Markets (CSEM) Policy Conference Sacramento, CA December 9, 2008 1

  2. Outline of Talk • History of Energy Efficiency in California • Overview of Evaluation • Adjustments to Energy Savings: Net Energy Savings • Cost-Effectiveness Metrics • Why do We Care about NTG & TRC? • AEA’s Guiding Principles for Evaluators 2

  3. 1. History of Energy Efficiency in California • Energy efficiency is California’s highest priority resource for meeting its energy needs in a clean, reliable and low-cost manner • For more than three decades, California has adopted energy efficiency policies and made investments that are among the most aggressive in the nation • These efforts have saved more than 40,000 GWh of electricity and 12,000 MW of peak demand - avoiding the need to build 24 large (500 MW) power plants, and equal to the energy required to power 3.8 million homes 3

  4. Californians use less electricity per person than those in all other states 4

  5. California vs. US Energy Efficiency Source: California Energy Commission 5

  6. Forecast Crisis Performance Incentives Profits decoupled from sales IRP Market Restructuring 2% of 2004 IOU Electric Revenues Public Goods Charges California IOU’s Investment in Energy Efficiency RRIM EnergyAction Plan EESP Source: California Energy Commission, modified by Vine (2008) 6

  7. 45,000 ~15% of Annual Electricity Use in California in 2003 40,000 35,000 30,000 25,000 Utility Efficiency Programs at a cost of GWh/year ~1% of electric bill 20,000 15,000 Building Standards 10,000 5,000 Appliance Standards 0 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 2003 1982 1975 1976 1977 1978 1979 1980 1981 1999 2000 2001 2002 Annual Energy Savings from EE Programs and Standards Source: Rosenfeld, California Energy Commission 7

  8. Energy Action Plan • Since its enactment in 2003, the Loading Order has been integrated into the major CPUC decisions governing energy policy and procurement. • In the EAP, the Loading Order continued: • “Pursue all cost-effective energy efficiency, first.” • Energy resources are prioritized as follows: • Energy efficiency/demand response • Renewable generation, including renewable DG • Increased development of affordable and reliable conventional generation • Transmission expansion to support all of California’s energy goals 8

  9. California: The Most Aggressive Energy Efficiency Program in the Nation • Energy Efficiency goals (2004-2013) • 26,506 GWh/year • 5,000 MW/year • 444 Million therms/year • Eliminates need for 10 new power plants • Eliminates 9 million tons of CO2 emissions (equal to 1.8 million cars) • $10 billion in net savings to consumers 9

  10. Current Program Cycle • 2006-08 • ~$2 billion in funding for 3 years • $581M in 2006, $646M in 2007, and $742 M in 2008 (excludes EM&V budget) • Annual funding from utility procurement dollars and from the Public Goods Charge • Levelized cost of 3 cents/kWh and 21 cents/therm • $2.7 billion in net savings to consumers over 3 years • $163 million EM&V budget for 3 years • 8% of total portfolio funding 10

  11. Shareholder Risk/ Reward Incentive Mechanism (RRIM) • Decision 07-09-043 (Sept. 20, 2007) • Creates incentives to ensure that utility investors and managers view energy efficiency as a core part of the utility’s regulated operations that can generate meaningful earnings for its shareholders • Protects ratepayers’ financing investment • Ensure that program savings are real and verified • All calculations of the net benefits and kW, kWh and therm achievements are independently verified by the CPUC and their EM&V contractors, based on adopted EM&V protocols • Imposes penalties for substandard performance 11

  12. Earnings capped at $450 million Reward (% of PEB) ER = 12% ER = 9% 0% 65% 85% 100% % of CPUC goals (per unit below CPUC goal) Penalty 5¢/kWh, $25/kW, 45 ¢/therm below goals, or payback of negative net benefits (cost-effectiveness guarantee), whichever is greater. Penalty capped at $450 million. Earnings = ER x PEB PEB= Performance Earnings Basis ER= Earnings Rate (or Shared- Savings Rate) Adopted Incentive Mechanism Earnings/ Penalty Curve 12

  13. Earnings Claim & Recovery Process • Two interim claims during each 3-year program cycle • Progress payments towards total expected earnings • One final true-up claim after the program cycle is completed • Hold back: 30% of the expected earnings in each interim claim to provide a margin for error in expected earnings 13

  14. 14

  15. Scope of Strategic Plan • Includes everything the two state energy agencies (CPUC and CEC) are currently working on, and more • Incorporates: • Market transformation • Voluntary market actions – to become “Business as Usual in California • Collaboration -- Roles for Local Governments, other State Agencies and Private Sector Players • 4 Big Bold EE strategies as cornerstones of initial bold energy-savings/outcomes 15

  16. 4 BIG BOLDEnergy Efficiency Strategies Commercial New Construction • All new commercial construction in California will be zero net energy by 2030. Residential / Small Commercial HVAC • Heating, Ventilation, and Air Conditioning (HVAC) industry will be reshaped Residential New Construction • All new residential construction in California will be zero net energy by 2020. Low- Income Energy Efficiency • All eligible low-income homes will be energy-efficient by 2020 16

  17. Proposed Next Program Cycle • 2009-11 • ~$4 billion in funding for 3 years • IOU program applications submitted and are being reviewed by the CPUC • Decision likely next year • Bridge funding adopted by CPUC to fund some of the 2006-8 programs, but no new programs • Many key questions, including: • How do (can?) the program applications support the RRIM & EESP? • Do cost-effectiveness indicators need to be adjusted to meet RRIM & EESP needs: e.g. TRC and net-to-gross? 17

  18. California’s GHG Emissions Goals • Reduce GHG emissions to 2000 levels by 2010 • Reduce GHG emissions to 1990 levels by 2020 • Reduce GHG emissions to 80% below 1990 levels by 2050 18

  19. Magnitude of the Challenge ~173 MMT CO2e Reduction 1990 Emission Baseline 80% Reduction ~341 MMT CO2e 19 19

  20. CARB - Climate Change Scoping Plan • Climate Change Scoping Plan (Oct. 2008) • Energy efficiency strategies will provide 16% of the estimated total emission reduction (169 MMTCO2E) by 2020 • Focus: increased incentives and more stringent building codes and appliance efficiency standards • Focus is on technological improvements, reflecting national and international studies • Bottom line: Incredible challenge & we need to do much more and urgently 20

  21. 2. Overview of Evaluation • Budgets for energy efficiency program implementation and evaluation have increased over time • California: • $2 billion (2006-2008); 8% ($163M) on EM&V • Shareholders incentives and risks: $450M • $3.7 billion (2006-2008) proposed • Increased interest in evaluation results!!! • But how does evaluation relate to the program planning cycle? • And how are evaluation findings used by program managers? • How to evaluate energy efficiency programs? 21

  22. Energy Efficiency Program Planning Cycle 22

  23. Evaluation Uses by Program & Portfolio Managers • Evaluation results can be used for: • Evaluating performance (“report card”) • Accountability and demonstration of success • Improving program design and implementation • Improving engineering & behavioral assumptions • Prioritizing program and portfolio budgets • Finalizing incentive payments • Resource (strategic) planning 23

  24. Six Types of Evaluation 24

  25. Impact Evaluation In the Absence Of the Program Installation EnergyUse Impact Actual Time 25 25

  26. Impact Data Needed to Collect • Monthly energy consumption • Metered or monitored energy usage • Load shape data (day, season, year) • Hours of operation for building or measures • Physical characteristics of the building and equipment (size and location) • Other physical variables: temperature, flow, weather • Building occupancy schedules and occupant data 26

  27. Data Collection and Analysis Methods for Impact Evaluation • Engineering methods • Basic statistical billing analysis • Multivariate statistical analysis • End-use metering • Short-term monitoring • Integrative methods 27

  28. 3. Adjustments to Energy Savings: Net Energy Savings • Gross energy savings • Net energy savings [Additionality] • Baselines (naturally occurring … energy use/ energy savings/ market adoption, etc.) • Free riders • Program spillover 28

  29. Net Energy Savings & Additionality • International Perspective: • Joint Implementation and Clean Development Mechanism • Emission reductions must be “additional to any that would otherwise occur” [“additionality criteria”] [“net emissions”] • Determining additionality requires a baseline • Focus is on environmental additionality (GHG emissions) • US/Utility Regulatory Perspective: • Similar to net energy savings [versus gross energy savings] • Determining net savings requires a baseline • Accounts for naturally occurring conservation and free riders 29

  30. Baselines [Reference Case] • What would have happened in the absence of a particular project or program? [counter-factual question] • Need to account, if possible, for economic growth, technological changes, prices, policy or regulatory shifts, population changes, market barriers and trends, etc. • Baselines must be credible and realistic to prevent/mitigate gaming 30

  31. Free Riders #1 • Activities are undertaken by participant(s) who would have conducted the same activities if there had been no project • Savings from free riders are not “additional” to what would otherwise have occurred • Therefore, subtract the free riders’ energy savings from gross savings to get net energy savings 31

  32. Net Energy Savings: Net to Gross = (NTG) Gross Energy Savings Free Riders #2 • Example in California: • If NTG = .8, then Free Riders =.2 (i.e., 20% of energy savings was due to energy savings from free riders) • Free rider methodology • Self reports - measurement limitations: social desirability bias, cognitive dissonance, and attribution • Comparisons with non-program areas - measurement limitations: data availability, self-selection bias, non-equivalent comparison groups 32

  33. UNFCCC/ CDMIndicators of Project Additionality • Regulatory Additionality • Has the project been directly or indirectly mandated by law or regulation? • Investment Additionality • Would the project have occurred under the investing party’s normal investment decision rules? • Technological Additionality • Does the project involve technology or practices that go beyond conventional practice in the corresponding industry or sector? • Barrier Removal Additionality • Does the project remove or work to overcome any information, institutional or other barriers that would persist in the reference case? 33

  34. Program Spillover • Participant spillover • Not normally looked at • What program participants did “extra” as a result of participating in program • Nonparticipant spillover • Not normally looked at • What program nonparticipants did as a result of hearing about the program • Spillover methodologies • Self reports - measurement limitations: social desirability bias, cognitive dissonance, and attribution • Comparisons with non-program areas - measurement limitations: data availability, self-selection bias, non-equivalent comparison groups • Market effects (e.g., sales) - measurement limitations: key is data availability and quality 34

  35. BENEFITS [Savings]: Utility Avoided Power Cost (and small emissions adder) Total Resource Cost = (TRC) Costs: Total Project Costs (All program and participant costs) 4. Cost-Effectiveness Metrics • If TRC >1, program is cost effective • If TRC < 1, program is not cost effective (but may still want to include as part of a portfolio of programs) • Savings are adjusted by NTG • Free riders has been the sole adjustment factor so far 35

  36. TRC Concerns • Is the right avoided cost being used? • Should avoided cost be based on natural gas combined cycle gas turbine (CCGT) or renewable energy (wind, solar) power plant? • Are NTG methodological issues too overwhelming? • Should some costs be subtracted because people invest in energy efficiency for non-energy reasons? • Comfort, quiet, home value, improved air quality, health, productivity, jobs, etc. • Non-energy benefits (NEBs) are not included 36

  37. Revise TRC BENEFITS [Savings]: Utility Avoided Power Cost (New Avoided Cost Plus NEBs) Total Resource Cost = (TRC) Costs: Total Project Costs (All program and 20% participant costs) 37

  38. 5. Why Do We Care about TRC & NTG? #1 • Regulatory objectives & concerns • Are we giving away money wisely? (Efficiency argument) • Are we giving away money to people who do not need it? (Equity argument) • In CA: focus has been on resource acquisition (RA) programs - due to RRIM • Not on market transformation (MT) programs - EESP focus • Not on market (program-centric) - EESP focus • In CA: focus is on free riders - due to RRIM • Not on spillover (participant and non-participant) - EESP focus • Not on market effects - EESP focus 38

  39. Why Do We Care about TRC & NTG? #2 • Focus varies by: • Regional objectives (RA [RRIM] vs MT [EESP]) • Program maturity (new versus old programs) • Concern over reducing greenhouse gases • Gross reductions • Net reductions (cap and trade; emissions trading) - Additionality • Policy issue: • Does current use of TRC and NTG inhibit new, market transforming energy efficiency interventions? • Time to look at market effects? Non-Energy Impacts? 39

  40. IEPEC Conference • To Learn More about Evaluation and These Exciting Topics, Go to: • International Energy Program Evaluation Conference (IEPEC) • August 12-14, 2009 • Portland, OR • http://www.iepec.org 40

  41. Before Carl Speaks 41

  42. American Evaluation Association:Guiding Principles for Evaluators • Systematic Inquiry • Competence • Integrity / Honesty • Respect for People • Responsibilities for General and Public Welfare http://www.eval.org/Publications/GuidingPrinciples.asp 42

More Related