ISO New England 2002 Demand Response Program Evaluation

1. ISO New England 2002 Demand Response Program Evaluation Presentation of “Major Findings” To The ISO Demand Response Working Group Townsley Consulting Group February 7, 2003

2. Evaluation Focus • Benefit Cost Analysis • Total Resource Test • Sensitivity Analysis • Congestion Cost Assessment • Impact of Class 2 DR on ECP • Retail Participant Survey • Stakeholder Surveys i.e. PUCs, LSEs • Review of Key Processes TOWNSLEY CONSULTING GROUP

3. Cost Benefit Analysis-Class 1 Total Resource Benefit Cost Ratios: All New England (NE) 0.85 SWCT 0.94 NE minus SWCT 0.32 • The B/C ratio could be improved by assuming more than a 1 year amortization year of infrastructure costs. • Significantly increasing the reliability benefit as represented by TMOR would be required to achieve program cost effectiveness in areas beyond SWCT. TOWNSLEY CONSULTING GROUP

4. Benefit Cost Analysis-Class 2 Total Resource Benefit Cost Ratio = 0.53 The B/C cost ratio could be improved substantially by increasing the average percentage of enrolled load reduced per called event hour from 9% to in excess of 20%. TOWNSLEY CONSULTING GROUP

5. Benefit Cost Sensitivity Analysis Class 1 (SWCT) Reliability Benefit Value Benefit/Cost Ratio $20 KW-month* 0.43 $63 KW-month ** 1.25 *Incremental value approved by the CT DPUC. ** Marginal value of ISO’s CT Supplemental RFP The base case Value of Load Loss for retail customers is $ 11,151/ hr. Which is well within the generally accepted range of $5K to $15K/hr. TOWNSLEY CONSULTING GROUP

6. Benefit Cost Sensitivity Analysis-Class 2 • Extending the number of called event hours to include all hours where the ECP exceeds $ 50/MWh would increase the total resource cost benefit ratio to 1.13. • Additionally, increasing the percentage of the enrolled load responding on average each hour from 9% to 27% would further increase the total resource cost benefit ratio to 2.82. TOWNSLEY CONSULTING GROUP

7. Congestion Price Duration Impact on Class 2 # of hours Congestion** Average at Hours Cost >$200 >$100 >$50 $/MWh Boston* 28 493 7,671 7,975 68.3 SWCT * 136 642 2,010 3,028 47.5 NE ECP 8 53 1,045 35.6*** * NE ECP + Congestion Costs ** Based on Jan-Nov 2002 data *** Based on Jan-Dec 2002 data TOWNSLEY CONSULTING GROUP

8. Congestion Price Duration Impact on Class 2 Hours Required to Achieve Program Cost Effectiveness as a Percent of Enrolled Load ___________Hours__________ % MW NE-ECP Boston CT SWCT 5 4 1999 993 1020 593 10 7 816 406 257 217 15 11 468 235 124 111 20 15 306 156 66 64 30 22 158 79 29 28 40 29 92 45 21 20 50 37 58 29 16 15 TOWNSLEY CONSULTING GROUP

9. Impact of Class 2 DR on ECP Analysis of Supply Curve Data for a 200MW Load Reduction. Lower Event Bid Price Price Day Hr. ECP Price Impact Savings 6/3 14 $195 $179 $16 $101,400 6/3 15 $194 $179 $15 $ 94,400 6/3 16 $115 $102 $13 $ 83,300 8/14 14 $1,000 $1,000 --- --- 8/14 15 $1,000 $1,000 --- --- 8/14 16 $1,000 $1,000 --- --- Issue: • Price caps would seem to diminish the DR price impact savings TOWNSLEY CONSULTING GROUP

10. Retail Participant Survey Results All retail DR Program participants were e-mailed an comprehensive, automated Microsoft Word survey to ascertain their opinions and thoughts regarding program participation and to develop information useful to future program planning. Approximately 17% of the participants returned a completed survey document. TOWNSLEY CONSULTING GROUP

11. Key Marketing Influencer TOWNSLEY CONSULTING GROUP

12. Information Meetings TOWNSLEY CONSULTING GROUP

13. Key Communication Methods TOWNSLEY CONSULTING GROUP

14. Satisfaction with Enroller TOWNSLEY CONSULTING GROUP

15. Program Rules/Procedures – how received TOWNSLEY CONSULTING GROUP

16. Rating of Payment/Settlement Process TOWNSLEY CONSULTING GROUP

17. Event Notification Method TOWNSLEY CONSULTING GROUP

18. Event Non-Participation TOWNSLEY CONSULTING GROUP

19. Major Load Curtailment Actions TOWNSLEY CONSULTING GROUP

20. Financial Impact TOWNSLEY CONSULTING GROUP

21. Simple Payback Period Required TOWNSLEY CONSULTING GROUP

22. Estimated Economic Loss- Unplanned Power Outage TOWNSLEY CONSULTING GROUP

23. Installed Equipment TOWNSLEY CONSULTING GROUP

24. Future Program Features TOWNSLEY CONSULTING GROUP

25. Overall Satisfaction TOWNSLEY CONSULTING GROUP

26. Satisfaction with ISO – NE Support TOWNSLEY CONSULTING GROUP

27. Stakeholder Survey-PUC staffs In person interviews were held with most PUC staffs in NE. The following feedback was received: • All felt that power supply issues were a high priority for their state, but not all thought that DR was a high priority. TOWNSLEY CONSULTING GROUP

28. Stakeholder Survey-PUC staffs • Most PUC staffs did not see the 2002 DRP as successful based on: • The amount of enrolled load. • Start-up problems (i.e., late rollout, customer notification, and settlement problems, etc.) • Class 1 resource not tested. • All felt the ISO was primarily responsible for reliability within the Standard Market Zones. TOWNSLEY CONSULTING GROUP

29. Stakeholder Survey-PUC staffs • Satisfaction with the LSE’s efforts in marketing and communicating the benefits of DR range from excellent to unreliable. • Regional Barriers to DR deployment identified: • Not easily understood by customers. • Lack of technical support for customers. • Incentive payments too low. • Absence of residential and small C/I customer aggregation. TOWNSLEY CONSULTING GROUP

30. Stakeholder Survey-PUC staffs • Comments were not favorable regarding the potential for pricing reforms to more directly align wholesale and retail prices. Some support exists for voluntary TOU rates and load shedding program participation. Political and legislative barriers to reforms were perceived as substantial. • Most PUCs did not anticipate a significant DR communication role, citing financial and people resource constraints. TOWNSLEY CONSULTING GROUP

31. Stakeholder Survey-PUC staffs • Recommendations included: • More specific goal setting for DR programs i.e. the amount of load to be enrolled reflecting sharp supply cost increases at various load levels. • Achieve greater parity in market pricing between supply and demand resources. • Create an ISO/NEPOOL standing committee on DR and related customer issues which is equivalent in stature and resources to the Reliability and Markets committees. TOWNSLEY CONSULTING GROUP

32. Stakeholder Survey-LSEs LSE’s (or distribution companies) were given the opportunity to participate in a group discussion or respond to an e-mail survey instrument. The following feedback was received: • All respondents indicated power supply issues were a high priority for them. One mentioned the targeting of Demand Side Management Resources to reduce load near substations with reliability concerns. TOWNSLEY CONSULTING GROUP

33. Stakeholder Survey-LSEs • Comments were quite divided regarding the perceived success of the 2002 Program. Concerns expressed included: • Low level of load signed up. • Low payments to customers and settlement problems. • The low value of ICAP created via the program. • Problems with establishing the baseline measurement levels and confusion regarding customer activation date. TOWNSLEY CONSULTING GROUP

34. Stakeholder Survey-LSEs • All respondents felt the ISO has the overall responsibility for zonal reliability. • Marketing and Communication initiatives reported by respondents: • Letters sent to customers 200 KW and larger. • Account representatives called on customers. • Hosted public forums and posted information on a website. No respondent indicated they planned to do anything different in 2003. TOWNSLEY CONSULTING GROUP

35. Stakeholder Survey-LSEs • Problems encountered in marketing the program: • Customers were confused about whether to sign-up for Class 1 or Class 2 program. • Explaining ICAP and TMOR to customers was difficult. • Customers seemed to be making modest enrollment commitments as a learning step and to gain access to incentives for infrastructure upgrades. TOWNSLEY CONSULTING GROUP

36. Stakeholder Survey-LSEs • Respondents were divided on whether or not program goals and objectives were communicated clearly. • Recommendations for the future: • Use the ISO’s website to communicate on FAQs and other relevant program information. • Enhance ISO support to provide faster turnaround on inquiries and settlement questions. • Improve event calling process. TOWNSLEY CONSULTING GROUP

37. Stakeholder Survey-LSEs • On the issue of the ISO’s involvement in program design, administration, and marketing, respondents offered the following comments: • Reactivate the DR working group to resolve and address issues and provide input into planning. • ISO should be involved in planning and administration; marketing should be shared with participants. • Alternatively some felt that the ISO provided insufficient administrative resources in 2002, and therefore should outsource future program administration. TOWNSLEY CONSULTING GROUP

38. Review of Key Processes During the course of the evaluation study, information emerged from survey data and interviews that indicated some administrative difficulties may have arisen during the start-up and implementation of the 2002 DR Program. Issues that surfaced as possible concerns included: • The quality of the information residing in the primary customer information data base. • The validity of data utilized in the settlement process. • The ability of the ISO to respond to and resolve in a timely manner issues emanating from the enrollment, event notification and settlement processes. TOWNSLEY CONSULTING GROUP

39. Review of Key Processes To ascertain if problems in some administrative processes were contributing to stakeholder and retail customer dissatisfaction, the information flow for the enrollment and settlement processes was reviewed with ISO staff and appropriate stakeholders to understand the information flow and the relevant history. TOWNSLEY CONSULTING GROUP

40. Review of Key Processes The review of the information flow for the enrollment and settlement processes yielded the following findings: • The staff assigned to handle initial processing of the NX11c data was probably insufficient to handle the volume and initially lacked the training necessary to evaluate and properly interpret the information. This resulted in improper information being passed along in the enrollment process and to the primary customer data base. TOWNSLEY CONSULTING GROUP

41. Review of Key Processes • In late spring of 2002 the ISO implemented a web based solution to automate and streamline the NX11c enrollment process. As the program enrollment process was well underway the new customer information database was being populated as time permitted from the original database. This required staff to rely on two separate databases for primary customer information including activation status. Also the web based solution initially had data formatting incompatibilities with the settlement data base which went undetected for a short time. TOWNSLEY CONSULTING GROUP

42. Review of Key Processes • The information management problems and some incongruence in baseline calculations between the RETX system and the ISO’s settlement system resulted in several questions from customers and LSE’s on settlement matters. The absence of well understood protocols for handling inquiries of this nature and a shortage of available staff resources led to long delays (months) in resolving settlement questions. TOWNSLEY CONSULTING GROUP

43. Review of Key Processes The ISO is currently undertaking the following measures to address the process issues identified: • Increasing DR professional staff resources from one part-time individual last spring to a full time manager and two professional support staff. • Implementing an “open solutions” software system, which will among other objectives, standardize baseline calculations for settlement and customer information purposes. • Substantially upgrade and formalize its business procedures and protocols for DR administration. TOWNSLEY CONSULTING GROUP

44. Addendum Additional Survey Results TOWNSLEY CONSULTING GROUP

45. How Easy were the program rules and procedures to understand? Percent 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% Extremely easy Very easy Easy Marginally easy Not easy How Easy Program Rules – easiness to understand TOWNSLEY CONSULTING GROUP

46. Program Rules and Procedures –supplier TOWNSLEY CONSULTING GROUP

47. Economic DRP Goals TOWNSLEY CONSULTING GROUP

48. Preference for Economic vs. Emergency Programs TOWNSLEY CONSULTING GROUP