Savings Are What You Define Them To Be: The “Baseline” Issue

1. Savings Are What You Define Them To Be:The “Baseline” Issue Tom Eckman Manager, Conservation Resources Northwest Power and Conservation Council

2. Defining Savings and Baselines • Savings - The difference in energy use between the baseline and post (after measure delivery) periods, which is caused by the installation/use of a measure or is attributable to one or more “mechanisms” (e.g, programs, codes and standards, etc). • Baselines Can Be: • Current/common practice = baseline is used if the measure affects systems, equipment or practices that are at the end of their useful life. In this case baseline use is defined by the recent typical choices of eligible end users in purchasing new equipment and services. • Pre-conditions = baseline is used when the measure-affected equipment or practice still has remaining useful life. In this case baseline use is defined by typical existing conditions.

3. Baseline Types Pre-Condition Baseline Savings Current Practice Baseline Savings

4. Baseline Types – The “Special One”Because it Comes in Two-Parts Persistent savings is only equivalent to savings over “current practice,” and there’s a risk in might be less Early Retirement Baseline Savings for RUL Current Practice Baseline Savings for EUL

5. Savings Are A Function of Perspective • Resource Planning Perspective – Savings are the difference between forecast energy use and efficient case energy use • Program Planning and Evaluation Perspective – Savings are the difference between the energy use that would have occurred “absent the program” and the energy use of program participants* *Including spillover and market effects

6. Resource Planning Perspective Focuses on Total Energy Use Reduction From All “Forces”

7. Program Planning and Evaluation Perspective Attempts to Determine Attribution To Each “Force” Source: Jayaweera, PhD, Tina and HosseinHaeri, PhD. “Methodology for Quantifying Market-Induced, Non-Programmatic Savings.” Prepared by Cadmus for BPA. April 1, 2011.

8. The Relationship Between Newtonian Physics and “Attribution” • Newton’s First Law: Corpus omneperseverare in statusuoquiescendivelmovendiuniformiter in directum, nisi quatenus a viribusimpressiscogiturstatumillummutare. • Translation 1 by Dave Berry – A body at rest will remain at rest until 8:45 p.m. The night before the science-fair project is due, at which point the body will come rushing to the body’s parents, who are already in their pajamas, and shout, “I just remembered the science fair is tomorrow and we gotta go to the store right now!” • Translation 2 - An object remains at rest or in motion unless acted upon by an external “force”

9. A Power Planner’s View of the Derivation of Attribution & “Net-to-Gross” Ratios

10. A Power Planner’s View of Attribution and Net-To-Gross • We Don’t Care Who Was Responsible for Screwing in the CFL • We Do Need to Know That • It got installed • How much electricity it will use • When it will use electricity • How long it will function

11. Implications for EM&V • There is no such thing as “naturally occurring” efficiency improvements • An external “force” must be exerted to move inefficient conditions to efficient conditions • “Net-to-gross” evaluations attempt to quantify the magnitude of each “force” • The choice of a “baseline” is not an assignment of “attribution” • But it does matter!

12. Should The Resource Planning and Program Planning “Perspectives” Use Different Baselines? • No – If the reported savings are used to determine the need for additional resources • Maybe – If the reported savings are used to determine compliance with an “EERS” (Energy Efficiency Resource Standard) which was set independent of resource need • Yes – I can’t think of any logical reason

13. Why Both Perspectives Should Use Current/Common Practice Baselines • Consistency with Load Forecast • The energy use of an appliance or piece of equipment that form the basis for load forecast should also be the basis for determining the remaining potential for conservation • Determining whether all or only a portion of the savings attributed to a program actually impact the need for resources cannot be done without comparison to what was explicitly or implicitly assumed in forecasting that need

14. Why Both Perspectives Should Use Current/Common Practice Baselines • Experimental Design - Determining what is occurring prior to program launch is a better measure of what would have occurred absent the program (i.e., the counterfactual) than evaluations made after the program has influenced the market

15. Why Both Perspectives Should Use Current/Common Practice Baselines • Establishing common practice baselines requires information on current market conditions prior to program or project design and implementation. • As a result, assessments of both the need for and design of market intervention programs can be conducted prior to implementation. • This reduces the probability of initiating programs where standard practices is already efficient and increases the likelihood of the targeting of niche markets that are not.

16. Why Both Perspectives Should Use Current/Common Practice Baselines Use of a common practice baseline generally results in much smaller differences between gross and net savings (i.e., higher NTG ratios). As a result, regulatory processes, such as cost recovery proceedings, tend to be less acrimonious

17. Why Both Perspectives Should Use Current/Common Practice Baselines • Use of Common Practice baselines tests whether incremental savings and further investments in energy efficiency are cost-effective to pursue, regardless of who gets credit for them.

18. Use of Common Practice Baselines Does Not Protect Against Self-Selection Bias • However, the market research or other data analysis that must be carried out in order to estimate a common practice baseline can and should inform program designs so they more effectively target consumers who are not already adopting the efficient technology or practice But, Poorly Designed Programs Are Still Possible!

19. Use of Common Practice Baselines Does Not Capture Non-Participant Spillover • On the other hand, very few NTG evaluations capture non-participant spillover, so whether this is better or worse than the current majority paradigm isn’t clear.

20. Use of Common Practice Baselines Does Not Give Credit For Market Effects of Prior Programs • On the other hand, very few NTG evaluations capture market effects of prior programs, so whether this is better or worse than the current majority paradigm isn’t clear.