Download
1 / 31
310 likes | 445 Views
Baseline Analysis CBP, AMP, and DBP. Steve Braithwait, Dan Hansen, and Dave Armstrong Christensen Associates Energy Consulting DRMEC Spring Workshop May 7, 2014. Presentation Outline. Objectives Methodology Data Performance measures Aggregator program (CBP and AMP) results
E N D
Baseline AnalysisCBP, AMP, and DBP Steve Braithwait, Dan Hansen, and Dave Armstrong Christensen Associates Energy Consulting DRMEC Spring Workshop May 7, 2014
Presentation Outline • Objectives • Methodology • Data • Performance measures • Aggregator program (CBP and AMP) results • Demand Bidding Program (DBP) results
Objective: Assess Performance of Alternative Baseline Types • For each Utility and Notice type: • All customers, with BL adjustment as chosen • All customers, simulated with universal selection of the BL adjustment • Sum of individual BL vs. portfolio BL (constructed from aggregated customer loads), for AMP and CBP only • Examine unadjusted and day-of adjustments with 20%, 30%, 40%, 50% caps, and uncapped
Analysis Details • For actual program event days • The “true” baseline is the estimated reference load from the ex post evaluation • For event-like non-event days • The “true” baseline is the observed load
Performance Measures (1)Percentage Baseline Error • Percentage BL error for each customer/portfolio-event day is: • Percentage error = (LPd – LAd) / LAd • LAd = actual, or “true” baseline load on day d • LPd = “predicted” baseline to be evaluated • Positive value = over-estimated baseline (implies over-stated program load impact) • Negative value = under-estimatedbaseline (implies under-stated program load impact)
Performance Measures (2) Accuracy • Accuracy is measured as the median absolute percentage error (MAPE) • Calculate the absolute value of the percentage error for each customer/event-day • Calculate the median of values across customer/event-days (mean can be misleading due to extreme values) • Higher values correspond to larger baseline errors
Performance Measures (3)Bias • Bias is measured by the median percentage error, without taking the absolute value • Positive values indicate upward bias (i.e., the program baseline tends to over-state the “true” baseline) • Negative values indicate downward bias (i.e., the program baseline tends to under-state the “true” baseline)
Nominated Customers by Choice of BL Adjustment – CBP and AMP
Percentiles of % Errors – PG&E CBP-DOActual Events, by Adjustment Cap
Percentiles of % Errors – PG&E CBP-DOSimulated Events, by Adjustment Cap
Summary: Accuracy & Bias (Aggregated Indiv.; Universal Adj.; 40% cap)
Summary: Percentiles of % Errors(Aggregated Indiv.; Universal Adj.; 40% cap)
Summary of Findings • Accuracy and bias measures vary by utility, program and notice type • Suggests that factors other than baseline type and adjustment caps may be most important, such as types of customers (e.g., highly variable load) and event-day characteristics (e.g., event on isolated hot day) • Day-of adjustment often improves accuracy and reduces bias, but level of cap is less important • Largest errors typically occur for Unadjusted BL and Unlimited cap • BL with small median error (e.g., 1%) can have >10% errors in 20 percent of cases
Summary • Day-of adjustments tend to improve baseline accuracy and reduce bias • The analysis provides support for making the day-of adjustment the default option • The effectiveness of the day-of adjustment is not very sensitive to the level of the cap
Questions? • Contact – Steve Braithwait or Dan Hansen, Christensen Associates Energy ConsultingMadison, Wisconsin • Steve@CAEnergy.com • Danh@CAEnergy.com • 608-231-2266
Appendix • SCE – CBP DO • SDG&E – CBP DO • PG&E – AMP DO • SCE – AMP DO
Percentiles of % Errors – SCE CBP-DOActual Events, by Adjustment Cap
Percentiles of % Errors – SCE CBP-DOSimulated Events, by Adjustment Cap
