Market Trials Structured/Unstructured Test Overview

2. Market Trials Structured/Unstructured Test Overview December 14, 2012 Casey Catheyccathey@spp.org · 501.614.3200

3. Agenda • Overview of Market Trials Structured/Unstructured Testing • Operating Days • Operational Scenarios • Settlements Scenarios • Unstructured Testing • Reports • Key Dates • Connectivity Test Metrics

4. Section 1.0 Overview of Market Trials Structured/Unstructured Testing

5. Market Trials Timeline • Market Trials is intended to validate that SPP and Participant systems, processes, and staff are able to operate in alignment with Integrated Marketplace Protocols. • Market Trials serves as the Participant Integration Test Stage utilizing the following Participant test sub-stages:

6. Objectives of Structured/Unstructured Test • Exercise the programmatic interfaces between SPP and the Market Participant as part of functional operations • Exercise all Integrated Marketplace subsystems including Day-Ahead, Reliability Unit Commitment, Real-time/AGC and Settlements • Structured testing will be used to verify specific business processes and system functions in a controlled manner • Unstructured testing will provide Participants with an opportunity to test their end to end systems at their discretion

7. Test Approach • The Connectivity Test Sub-Stage is made up of the Connectivity Test Preparation Phase, Connectivity Test Period, and Data Exchange Certification Test Period, leading up to the Structured Testing Sub-Stage.

8. Structured/Unstructured Test Entry Criteria • SPP has completed required integration testing for functionality and provided a qualitative analysis of the testing results and recommends that SPP is ready for the start of Phase I. • Results of the testing will be available to participants. • The exit criteria of the Data Exchange Certification Trials stage • SPP has published all known, market-relevant defects to Market Participants; including criticality and narrative of functionality that will impact Phase 1 testing. • SPP, MPs with Resources and TOP’s have substantially completed the creation and testing of new and critical ICCP points (including new resource data points and tie-line data points)

9. Structured/Unstructured Test Entry Criteria cont’d • SPP has completed preliminary performance testing for Phase I functionality • MPs representing Target Mass have completed the connectivity testing and have confirmed their ability to use Web Services or the Market User Interface to submit transactions necessary to support Market Trials including the ability to submit daily transactions to support both Day-Ahead and Real Time Balancing Market activities. • Go recommendation from the Advisory Board • Published, Approved & communicated Phase 1 testing plan

10. Section 2.0 Operating Days

11. Operating Days • Operating Days will be defined as days that have supported Day-Ahead and Real-time functions. • During the beginning of Market Trials there will be two Operating Days per week progressing to three days per week, five days and finally 7 days a week prior to Parallel Operations • Any Day Ahead Market or Reliability Unit Commitment runs for the Operating Days will be available for Participant questions. • Real Time Balancing and AGC will be supported Monday-Friday 8A-5P until the final two weeks of October which will be 24X7

12. Structured/Unstructured Test Details • Market Trials Structured/Unstructured Test will consist of two phases: • Phase 1 • Run basic Day Ahead Market and Reliability Unit Commitment with co-optimization of Energy and Operating Reserves along with virtual transactions (Day Ahead only). • Run the Real Time Balancing Market with the co-optimization of Energy and Operating Reserves along with the deployment of Contingency Reserves and Regulation. • This test will be open loop and setpoint instructions will not be followed by Resources. • Participants will submit Day Ahead Bids and Offers, Real Time Offers and Meter Data • SPP may overwrite or supply bid and offer data if needed for quality solutions.

13. Structured/Unstructured Test Details • Phase 1 • SPP plans to settle each Operating Day in June • Focus on testing the MPs ability to process and download statements and invoices • During this period settlements will be posting according to the following schedule: • Initial: OD + 7 • Final: OD + 14 • R1: OD + 21 • R2: OD + 28 • Settlements will provide a market calendar prior to the beginning of Phase I in order to reflect the schedule above

14. Structured/Unstructured Test Details cont’d • Phase II • Will consist of Structured and Unstructured Testing • Scenarios will be introduced to test various aspects of the sytems. • Day Ahead • Real Time Balancing Market • AGC • Credit • Settlements • Non Functional • SPP may overwrite or supplement Participant Bids, Offers and Meter Data in order to effectuate various scenarios.

15. Section 3.0 Operational and Settlement Scenarios

16. Structured Operational Scenarios • During Phase II there will be multiple operational scenarios that will be tested • Delay Market Close • Violation of Credit Limits • RTBM outage • Max/Min Gen • Scarcity Pricing • OOME • Unit Commitment • Etc…

17. Structured Settlement Scenarios • During Phase II, SPP plans to settle each Operating Day. During this phase the supported days will contain conditions that will allow each charge type to be tested and validated. These days will also be used to perform functional testing. • During this period settlements will be posting according to the following schedule: • Initial: OD + 7 • Final: OD + 14 • R1: OD + 21 • R2: OD + 28

18. Section 4.0 Unstructured TEsting

19. Unstructured Testing • During Unstructured Testing Market Participants will be free to submit bids/offers and meter data to fully test their systems. • SPP does not plan to override participant data unless there are technical issues • Solutions could be unrealistic depending on the input data that is used for each run

20. Section 5.0 Reports

21. Reports • The plan is to have all critical data reporting listed in the document available for the start of Market Trials and that the other items will be worked on in order of priority. • Critical Reports for Testing • LMP and MCP downloads • Real-time binding constraints • Real-time LMP by electrical bus • Operating Reserve requirements • Day Ahead LMPs per Settlement Location • Day Ahead MCPs for Operating Reserves • Day Ahead binding constraints • MP Settlement Calendar

22. Section 6.0 Scope of market Trials connectivity test

23. Components in Scope • It is currently expected that all component of the new Integrated Marketplace will be in Scope for the Market Trials Structured/Unstructured Test.

24. Components Out of Scope • Currently is it expected that all major components of the Integrated Marketplace will be available for the start of Market Trials Structured/Unstructured testing. An out of scope report and major defect report will be maintained to inform Market Participants of any specific components or functionality that will be unavailable. These reports will be updated weekly (more often if needed) and shared with Market Participants during the Market Trials calls.

25. Section 7.0 Structured/Unstructured test key dates

26. Key Dates

27. Key Dates – Cont’d

28. Section 8.0 Communications plan

29. Communications Plan • General Information and Inquiries: MarketplaceMarketTrials@spp.org • SPP will distribute information related to Connectivity Test through the Market Trials Liaison distribution list. • Bi-weekly Conference Calls (twice a week) • Starting June 3, 2013

30. Issue Reporting • Discovered issues should be reported to your Customer Relations Representative via phone, email, or the Request Management System • SPP staff will investigate the reported issue and respond with a resolution or notification that an SPP system issue will be logged. • System issues will be listed and tracked in the Known Issue Log, posted weekly on www.spp.org.

31. Section 9.0 Exit Criteria

32. Structured/Unstructured Test Exit Criteria • SPP completes DA Market execution with Market Participant data, as planned • SPP completes RUC execution with Market Participant data, as planned • SPP has reviewed and communicated all test results with MPs and has updated the relevant dashboard/tracking documents as appropriate to indicate completion. • Successfully execute Integrated Marketplace Bid to Bill functionality consistently for 3 consecutive Operating Days according to protocol timelines;

33. Structured/Unstructured Test Exit Criteria cont’d • Target Mass Participates • SPP is able to sufficiently explain market solution anomalies • Demonstrate each charge type for at least one resource and load as applicable for each Market Participant

34. Market Participant Milestones