California Information Display Pilot Load Impact Analysis

1. California Information Display PilotLoad Impact Analysis Presentation to the WG3 Committee San Francisco, CA January 7, 2005 Craig Williamson, EPRI Solutions Boulder, CO

2. IDP Impact Assessment Goals • Assess the load impact of providing enhanced information treatments to customers, over and above the impact of enabling technology and the rate/price • Control for other factors, including • Customer size (high-low consumption) • Day-specific conditions • Treatment installation date • Groups include residential and commercial customers on the CPP-V rate (2003 Track A SPP sample) • Challenge – only 62 total customers available

3. Impact Assessment Process • Include both a treatment and a control group in the analysis, both on CPP-V • Collect interval load data during pre-treatment period for both groups, begin treatments, then collect post-treatment data • Use Energy Orb installation date as start of treatment – newsletters started at about the same time • Energy Orbs installed from 7/28/04 through 8/31/04 • Pre-treatment and post-treatment periods are different for different customers

4. Impact Assessment Methodology • Difference of Differences approach • Corrects for changes over time (pre and post) and recognizes differences between treatment and control groups • First calculate average for each customer for pre and post period (customer-specific) • Then average all customers in each sample cell for control and treatment group • First difference: Treatment – control for each hour in both the pre-treatment and post-treatment periods • Second difference: (difference in the post-treatment period) – (difference in the pre-treatment period) • Calculate weighted average of cell differences, using weights based on the number of treatment customers in each cell

5. Pre and Post Differences (after averaging)

6. 2004 IDP Sample Design • IDP Treatment Group (enhanced) • 33 residential customers on CPP-V • 29 commercial customers on CPP-V • IDP Control Group (standard) • 100 residential customers on CPP-V • 138 commercial customers on CPP-V • Residential customers located in SDG&E areas, and C/I customers located in SCE territory • IDP Control Group was the 2004 Track A

7. 2004 IDP Results - Residential Difference of differences approach, with CPP-V as common rate in treatment and control groups, and adjusting for pre and post treatments periods for both groups

8. IDP Residential – average for 2-hour event days Difference of differences approach, using “relative time” to treat different days consistently

9. IDP Residential – individual 2-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

10. IDP Residential – average for 2-hour event day with 90% confidence intervals Difference of differences approach, using “relative time” to treat days with different start times consistently

11. IDP Residential – average for 5-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

12. IDP Residential – individual 5-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

13. IDP Residential – average for 5-hour event day with 90% confidence intervals Difference of differences approach, using “relative time” to treat days with different start times consistently

14. 2004 IDP Results - Commercial Difference of differences approach, with CPP-V as common rate in treatment and control groups, and adjusting for pre and post treatments periods for both groups

15. IDP Commercial – average for 2-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

16. IDP Commercial – Individual 2-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

17. IDP Commercial – average for 5-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

18. IDP Commercial – Individual 5-hour event days Difference of differences approach, using “relative time” to treat days with different start times consistently

19. Impact Analysis Conclusions • Residential customers show an impact during Super Peak, and also during the 4 hour warning period • Commercial customers do not show a consistent impact, but something is happening • None of the results are statistically significant • Limit of small sample sizes • Variability of the commercial customer types • Customer feedback indicates that treatments are somewhat responsible for the apparent impacts, either alone or combined • Individual analysis of each customer or subgroups would be beneficial in “teasing out” specific behavioral effects

20. Recommendations for 2005 • Sample sizes are obviously too small – recommend increasing to improve significance • Residential customers may be responding to both the Energy Orb and the newsletter – recommend a bifurcation of treatments or focus on one treatment • Commercial customers are too variable – need to increase sample size and also focus on one treatment • Difference of differences approach can be repeated more effectively if Super Peak events and treatment start times are properly coordinated