Lessons Learned ... so far ...

1. Lessons Learned ... so far ... Lemuel Gonzalez Resource Registration & Performance Auditing DRWG Meeting July 5, 2010

2. Agenda DRWG, July 7, 2010, Lessons Learned DR Audits and June 24th Event Lessons learned from audits and events Data quality issues Notable issues Data submission Other observations Your lessons learned Next steps Questions

3. ISO, Lead Market Participants and DDEs successfully dispatched DR! DRWG, July 7, 2010, Lessons Learned • ISO audited 166 Demand Resources across 11 dispatches (doesn’t include resources using the event as an audit) • Started small in early June performing selective audits to test • ISO CFE to DDE RTU communications and dispatch instructions • DDE readiness and their ability to dispatch Demand Resources • End-use customer response and communications • Audits supported Control Room needs later in June when a real event occurred • ISO activated RTDR on June 24, 2010 where Demand Resources provided the needed relief

4. * There is no 30 minute ramp time when Dispatch MW are decreasing ** Effective time is the time the full load reduction is expected and the start time that performance for settlement purposes is measured. June 24, 2010 Real-Time Demand Response Dispatch (OP4 Action 2) DRWG, July 7, 2010, Lessons Learned

5. June 24, 2010 Forecast and System Load DRWG, July 7, 2010, Lessons Learned

6. Lessons from the DR Audits and the June 24th event DRWG, July 7, 2010, Lessons Learned • Communication infrastructure to the end-use customer needs to be reviewed and enhanced as appropriate • Lead Market Participants and DDEs should be able to: • Distinguish dispatch of RTDR from RTEG at a facility • Distinguish facilities and measures where your customers share the same end-use customer contact • Exclude certain PNodes/Assets from a dispatch • Based on limited observations: • DDE operators need to be available to the Control Room and properly trained • DDE’s ability to see and respond to unannounced dispatch instructions on the RTU may need improvement (e.g., 20 minutes into a dispatch or audit, the DDEs had not “seen” the signal)

7. Data quality challenges to establishing DR Performance for June DRWG, July 7, 2010, Lessons Learned • DDEs, Lead Market Participants and ISO are adjusting to new systems (CFE/RTU, DR MUI, CAMS, SMS) • Threshold checks in DR MUI relaxed due to poor estimates used for the checks (registration values) • Larger number of assets are clearing in the DA LRP (participation increased from 133 to 826 assets, and many have cleared across multiple days in June which as an interruption day freezes the baseline) • Meter data quality issues have been identified • Scaling, units, and sign conventions and meter problems • Correction window closes after 3 business days following Operating Day (limiting DDEs ability to correct systemic problems)

8. Data quality issues DRWG, July 7, 2010, Lessons Learned • DR audit results have been delayed by data quality issues • ISO will issue preliminary results to be used in the initial June settlement (namely for RTEG, as all other DR types had performance hours in June) • Audit results will need to be revised once meter data and other data quality issues are addressed for use in Financial Assurance Return and Commercial Operation determination • Meter problems and missing interval data may disqualify a Demand Asset from participation unless adequate remedial actions are taken or demonstrated • Patterns on missing data or repeated problems will be documented and referred to the Market Monitor as appropriate • Lead Market Participants may also be asked to expand on their controls and validations in the Annual Certification of Accuracy of Measurement and Verification Documents (see Section 12 and 14 of M-MVDR)

9. M-MVDR, Section 12: Data Validation, Retention and Management (12.2.3-4) DRWG, July 7, 2010, Lessons Learned (3) The Project Sponsor shall validate all measured data used in the Demand Reduction Value calculations. Data that has failed validation may not be used in any Demand Reduction Value calculation. (4) For Projects involving an individual facility, generator or energy consuming equipment, the Project Sponsor shall conduct the following validation checks on any interval data from an individual facility: (a) Time Check: The Project Sponsor shall validate that the measurement devices time clock is within ± two minutes of the true time as defined by the National Institute of Standards and Technology. (b) Sum Check: The Project Sponsor shall validate that the difference between the sum of the values recorded over the intervals and the value recorded by the meter over the same time period is within plus or minus two percent. This check may be done on either usage or pulse data, provided the data scaling is consistent throughout the period. (c) High/Low Check: The Project Sponsor shall establish minimum and maximum expected values for each Demand Asset, facility, or measure. The minimum and maximum values shall be based on equipment ratings or historical equipment and/or facility usage data. The Project Sponsor shall identify any and all interval data that is greater than the maximum expected value or less than the minimum expected value. Any such interval data shall be deemed to fail validation. (d) Zero Value Check: The Project Sponsor shall identify any and all interval data with a value equal to zero. The Project Sponsor shall verify whether or not the zero value is the correct value for that interval. If the Project Sponsor determines that the zero value is incorrect, the Project Sponsor shall substitute a corrected or estimated non zero value for the zero value. Under no circumstances shall the Project Sponsor substitute a zero value for missing interval data. (e) The Project Sponsor shall identify all estimated data used in the Demand Reduction Value calculations, as well as the methodology used to develop the estimate. (f) The Project Sponsor shall classify all data that has passed validation and is used in the Demand Reduction Value calculations as either: (i) actual data, (ii) estimated data or (iii) missing data.  The data classification shall be stored along with the data values in the Project Sponsor’s data retention and management system described in Section 12.1

10. Notable issues that we've overcome DRWG, July 7, 2010, Lessons Learned • Data submittal windows for On-Peak and Seasonal Peak Demand Assets in the SMS "Submit Meter Reading" user interface was closing after three business days from the Operating Day and not the settlement month • We've corrected the meter submission window for these assets • SMS Meter Reading user interface keys off the designated Meter Reader for Demand Assets • Lead Market Participant of a Demand Asset is not able to submit data into the SMS user interface for certain Active Demand Assets and Passive Distributed Generation Demand Assets • If you work with a DDE who is also designated as the Meter Reader for an asset that needs to submit hourly facility meter load data, then you will need to have them do this function for you

11. Data submission lessons (sign conventions, units, and scaling) DRWG, July 7, 2010, Lessons Learned • Load values are negative (generally, unless pushing back to the grid) when submitted through the DR MUI and the SMS Meter Reading user interface • The sign convention is the opposite in the CFE/RTU • Generation values are positive • Load and generation values submitted through the DR MUI are in MWs to the third decimal place • Values submitted through the CFE/RTU have no decimal place (i.e., whole integers in KWs) • DR MUI expects MWhrs per hour (so in some cases you needed to multiply your readings by 12, some of you did that twice)

12. Data submission lessons (Passive DG) DRWG, July 7, 2010, Lessons Learned • Data submitted through the SMS Meter Reading UI • See presentation from July 1 regarding the data submission deadlines and interfaces included in the Appendix) • CSV file upload format (and direct data entry) available • May need a macro to “format” the files correctly or view in a text editor directly • Extra commas generated by Excel when exporting to a CSV • The facility meter load value, though it may be required by ISO, is not a required field in the file (e.g., insert the NULL or leave blank) Meter Daily *** 50909,33631,FCM Demand,6/1/2010,ON_PEAK 1,null,2 2,,2

13.  Data submission lessons (cont.) DRWG, July 7, 2010, Lessons Learned • Hourly Facility Metered Load data should be submitted through the SMS Meter Reading user interface when required • Deadline is 2.5 business days from the operating day for active demand assets (RTDR, RTEG) • Deadline is 2.5 business days form the end of the settlement/obligation month for passive demand assets (On-Peak, Seasonal Peak) • Requirement to submit Hourly Facility Metered Load data applies to a very small subset of assets • Need to submit data can be verified through CAMS by navigating to the “Generator” tab (the last two flags at the bottom of the tab address this function)

14. Other Observations DRWG, July 7, 2010, Lessons Learned • For assets metered as Load Reduction with DG at another on-site asset, some participants appear to only be providing generation data during audits or events • Generation data is required at all times so that a correct "TFL" baseline is built • A number of DG-based Demand Assets, mostly RTEG, are reporting the wrong data relative to their metering configuration • FML when the asset is DG-output directly metered and • DG output when the asset is DG used to reduce load. • These assets will show a zero performance since we do not have the appropriate data to settle or evaluate the asset performance.

15. Other Observations (cont.) DRWG, July 7, 2010, Lessons Learned • Number of Demand Assets clearing in the Day-Ahead Load Response Program, as well as the number of days, was higher than expected in June • Days where a Demand Asset has a Day-Ahead schedule counts as an interruption day and is excluded from the baseline • Consequently, the baselines may not be updating as often as they would were it not for the interruption days • Makes corrections to May 2010 all the more important

16. Your lessons learned DRWG, July 7, 2010, Lessons Learned Please share, now or later, any observations or lessons learned on things that the ISO or our customers can do differently

17. Next steps for the ISO DRWG, July 7, 2010, Lessons Learned • Continue reviewing the DR audit and event performance • Evaluate the frequency and nature of changes between telemetry, initial and final corrections • Establish verified capacity ratings to use in the release of Financial Assurance and Commercial Operation determination for non-commercial resources • Work with Lead Market Participants to update relevant registration data (e.g., MIC, load, gen values) • Work with DDEs to reinstate the DR MUI meter data threshold (original intent was to have a 1.25 multiplier threshold and due to data issues that was relaxed in May) • ISO will reduce DR MUI meter data threshold starting in August

18. Next steps for Lead Market Participants DRWG, July 7, 2010, Lessons Learned • Review your dispatch process and communications • Verify your Demand Asset registration information in CAMS, most notably the: • Maximum interruptible capacity, estimated maximum load (most recent non-coincident annual peak demand), estimated maximum generation • Registered meter configuration vs. actual meters in the field • Pushback capable and Hourly Load Data required flags (for Distributed Generation measures) • Submit revised data for May and June 2010 as necessary as soon as the data submission window opens up again (after June invoice is issued) • Review and enhance data validations and controls

19. Support DRWG, July 7, 2010, Lessons Learned • Customer Support • Regular business hours • Monday through Friday, 7:30 A.M. to 5:30 P.M. Eastern Time • Contact information • Phone : 413-540-4220 • All lines are subject to recorded/monitored conversations • E-mail: custserv@iso-ne.com • Fax: 413-535-4156

20. Questions DRWG, July 7, 2010, Lessons Learned

21. Appendix

22. DR Data Submittal through ISO User Interfaces Chad NelsonKristen EmmettResource Registration & Performance AuditingJuly 1, 2010

23. Agenda DRWG, July 7, 2010, Lessons Learned Data Submittal Deadlines Data Submittal Interfaces Demand Resource Performance Hours Publishing of Seasonal Peak Hours Assigned Roles in CAMS Submitting Monthly Performance Values (CAMS) Submitting Hourly DG Output (SMS) Submitting Real-Time Data (DR MUI)

24. Data Submittal DeadlinesActive & Passive Demand Assets DRWG, July 7, 2010, Lessons Learned

25. Data Submittal DeadlinesActive & Passive Demand Assets DRWG, July 7, 2010, Lessons Learned

26. ISO New England Data Submittal Interfaces DRWG, July 7, 2010, Lessons Learned

27. Demand Resource Performance Hours DRWG, July 7, 2010, Lessons Learned • On-Peak Hours (Market Rule 1 Definition) • Summer: During the months of June, July and August, 1 p.m. through 5 p.m., Monday through Friday, non-Demand Response holidays • Winter: During the months of December and January, 5 p.m. through 7 p.m. Monday through Friday, non-Demand Response holidays • Seasonal Peak Hours (Market Rule 1 Definition) • During applicable season’s performance months, when any portion of an hour, on Monday through Friday, non-Demand Response holidays, where the actual Real-Time hourly integrated load for the pool is equal or greater than 90% of the most recent 50/50 peak load forecast for the summer or winter season

28. Publishing of Seasonal Peak Hours DRWG, July 7, 2010, Lessons Learned • The Seasonal Peak Hours Report will be published on the ISO-NE external website under the heading of Seasonal Peak Hour Data: • Markets>Other Markets Data>Forward Capacity Market>Reports • The Seasonal Peak Hours Report will be accessible the first business day of the next month (i.e. Seasonal Peak Hours Report for the month of June will be published to the web by July 1)

29. Assigned Roles in CAMS - MPV DRWG, July 7, 2010, Lessons Learned To submit monthly performance values, the assigned Security Administrator (SA) must grant the “Customer Asset Management System / External Demand Asset Maintainer” role to the person(s) who will be submitting data on the company’s behalf.

30. Assigned Roles in CAMS - SMS DRWG, July 7, 2010, Lessons Learned To submit data through SMS, the company must be granted the meter reader role and the assigned SA must grant the “SMS / Metering and Daily Tag” role to the person(s) who will be submitting data on the company’s behalf.

31. Assigned Roles in CAMS – DR MUI DRWG, July 7, 2010, Lessons Learned • To submit data through the DR MUI, the assigned SA must grant the proper role to the person(s) who will be submitting data on the company’s behalf. The roles needed depend on the company’s function and include: • “Demand Response MUI / DR Lead Participant Read Write” • “Demand Response MUI / DDE Read Write” and/or • “Demand Response MUI / Meter Reader Read Write”

32. Submitting Monthly Performance ValuesNon-DG based On-Peak and Seasonal Peak Resources DRWG, July 7, 2010, Lessons Learned Data for non-DG based On-Peak and Seasonal Peak Demand Assets will be submitted through CAMS. When viewing the Demand Asset in CAMS, click the Monthly Performance Values button. When viewing the Monthly Performance Values screen, the window to submit data for the obligation month will be open. Submit initial values for ISO approval along with supporting documentation.

33. Submitting Monthly Performance Values DRWG, July 7, 2010, Lessons Learned Click on “Customer and Asset Management System” on the SMD Applications Home Page

34. Submitting Monthly Performance ValuesNon-DG based On-Peak and Seasonal Peak ResourcesClick on the Monthly Performance Values button in CAMS: DRWG, July 7, 2010, Lessons Learned

35. Submitting Monthly Performance ValuesNon-DG based On-Peak and Seasonal Peak ResourcesInitial values along with supporting documentation for ISO review are submitted for the available months: DRWG, July 7, 2010, Lessons Learned

36. Submitting Hourly DG OutputDG Based On-Peak and Seasonal Peak Resources DRWG, July 7, 2010, Lessons Learned • The meter readings can either be submitted by file upload or entered manually in the boxes provided. • If the data is being submitted via a file upload, please review the Metering Upload File Formats document which can be found on the external web: • Support>Data Retrieval>Upload/Download File Formats>Metering Upload File Formats

37. Submitting Hourly DG OutputDG Based On-Peak and Seasonal Peak Resources DRWG, July 7, 2010, Lessons Learned • Click on “Submit Meter Reading” on the SMD Applications Home Page

38. Submitting Hourly DG OutputDG Based On-Peak and Seasonal Peak Resources DRWG, July 7, 2010, Lessons Learned Click on FCM Demand Assets

39. Submitting Hourly DG OutputDG Based On-Peak and Seasonal Peak Resources DRWG, July 7, 2010, Lessons Learned • Choose the demand asset type for which meter readings are being submitted and then click “Retrieve.”

40. Submitting Hourly DG OutputDG Based On-Peak and Seasonal Peak ResourcesSubmit data via file upload or enter manually: DRWG, July 7, 2010, Lessons Learned

41. Submitting Hourly DG OutputActive ResourcesSubmit data via file upload or enter manually: DRWG, July 7, 2010, Lessons Learned

42. Submitting Real-Time Data DRWG, July 7, 2010, Lessons Learned Click on “Demand Resource Market User Interface” on the SMD Applications Home Page

43. Demand Resource Market User Interface DRWG, July 7, 2010, Lessons Learned • Each display in the application can be found under the Navigation menu. • Click “Asset Telemetry” to manually update the telemetry for a Demand Asset • Click “Upload XML” to use the file upload/XML functionality

44. Demand Resource Market User Interface DRWG, July 7, 2010, Lessons Learned • A screen will be presented that will allow the user to upload the necessary XML file(s). • A user guide that describes this process in further detail can be found on the ISO-NE external website: • Support>User Guides>Demand Resource User Guide

45. Support DRWG, July 7, 2010, Lessons Learned • Customer Support • Regular business hours • Monday through Friday, 7:30 A.M. to 5:30 P.M. Eastern Time • Contact information • Phone : 413-540-4220 • All lines are subject to recorded/monitored conversations • E-mail: custserv@iso-ne.com • Fax: 413-535-4156

46. Questions DRWG, July 7, 2010, Lessons Learned