190 likes | 207 Views
Texas Nodal Market Implementation Program Update. Jerry Sullivan. AGENDA. Program Status Key Risks IBM Review given to BoD in February Quality Results Schedule and the Program Level “Health Checks” Cost. The area of emphasis at today’s TAC Meeting. Program Status: Nodal Delivery.
E N D
Texas Nodal Market ImplementationProgram Update Jerry Sullivan TAC
AGENDA • Program Status • Key Risks • IBM Review given to BoD in February • Quality Results • Schedule and the Program Level “Health Checks” • Cost The area of emphasis at today’s TAC Meeting Texas Nodal Program Update
Program Status: Nodal Delivery Nodal Delivery status remains amber; key checkpoints approaching Nodal Delivery Cost Scope / Quality Schedule Green Amber Amber Red • Number of defects discovered in EDS were high, being resolved. • Applying increased efforts to identify and resolve defects in pre-FAT and FAT • Key checkpoints in place to ensure ongoing viability of 12/1/08. • Planning for the “168 Hour Test,” which is under 200 days away. • EDS 2 is Red, but soon may be amber. • EDS 4 is Red, will go amber when EMS 4/5 completed. • Need to decide on methodology for grayboxing and/or time-boxing the handling of: • current protocols,or • future NPRRs. • Requirements for DAM in MMS are being revised. • Revised budget approved by ERCOT BOD, however we will be RED until fee filing approved. • Revised Fee Filing with the PUC this week. Legend Current product quality is not sufficient for Nodal launch Program not aligned to protocols Estimate at Complete = >$263m + 10% Red Go-live = >30 days+ Most Nodal products currently achieve quality standards Program aligned to previous protocol version Amber Estimate at Complete = < $263m + 10% Go-live = <30 days+ Program aligned with current protocols All Nodal products currently achieve quality standards Green Estimate at Complete = <$263m Go-live = 12/1/08 Texas Nodal Program Update
Program Status: ERCOT Readiness (3/15/2008) ERCOT Readiness status remains amber, but a renewed awareness of the need for preparation is apparent • Organization, Processes and Procedures, and Trainingshow progress • Organization • Completed review of Deep Dive presentations including staffing/ sizing department models for Post-Nodal implementation (2009). • Draft Vision, Mission, Core Values and Strategic Plan completed. Transition sponsors are assigned. • January Executive Readiness Survey overall indicates Amber. Key Focus Areas to work on: adequate staffing, procedures, transition planning. • Transition plans being documented for all key departments. • Processes and Procedures • Business Process Model completed to procedure level and ready to leverage for testing and validation during EDS and training. • Increased focus on the creation of procedures. • Training • Training tracking and tailoring at the department/individual level. • Validation of current individual training plans • Concerns Remain • Adequate staffing and resources for transition from project teams to employees • Procedures still in flux until delivery of systems • Completion of training in time for EDS Priority for ERCOT Staff is to ensure readiness for the “168 hour test” Texas Nodal Program Update
Program Status: Market Participants (2/28/2008) Overall status* returns to amber • Registration and Qualification[green] -Resource Asset Registration quality, as measured by metric MP11, has improved. Two QSEs represent resources that are not current with submittals for resource registration: • Barton Chapel Wind, Austin Energy • Engagement continues to be main area of concern. • Engagement remains [amber] for all segments except QSEs with Resources. • Continuing to work with MPs to improve their engagement scores. • 13 QSEs [red] - due to Engagement. The following remain red in the scorecard: • Citadel, Clearview, Duke Energy Ohio, Econnergy, EnergyCo, JPMorgan Ventures Energy, Juice Energy, Kansas City Power and Light, Liberty Power, LPT, Mirant, Texpo, WCW International • Next survey to be released March 17. Negative responses will be treated as red in the next round. • Training [green] – First pass of self reported training is green for all segments except LSEs • Connectivity and Telemetry [green] – Connectivity metrics complete for most MPs. Outliers are new to the program and need to go through EDS process * Status current as of 2.28.08 Texas Nodal Program Update
Market Readiness Six month posting of LMPs initiated on February 15 • Required by: ERCOT Nodal Transition Plan (5.4.4 (3)) • Purpose: Provide an opportunity for market participants to measure and improve the accuracy and stability of the Real Time Sequence (RTS). • Accuracy: LMPs are reasonable given the input data, but the input data needs improvement. These are not “settlement grade” LMPs. ERCOT is limited in its capacity to handle all MP questions based on the quality and complexity of the data. • Use: Should be used for trends and to assist the EDS team to improve the accuracy of the input data, but not for financial analysis. • Actions: ERCOT is continuing to resolve a number of problems known to impact the SCED results: Working with QSE’s to resolve a number of SCED data input issues; similar exercise is in progress with TSP’s to improve the quality of the tele-metered data feeding the State Estimator. Texas Nodal Program Update
Key Risks as 5 March 2008 These are the key, key risk Risk Description Likelihood Impact on Go Live Mitigation • Lack of source systems and infrastructure limits ability to validate integration points, impacts re- factoring cycle times and costs • Building interfaces in advance of applications delivery based on available information expecting re-factoring • Testing application and integration in available, sub-optimal hardware / software environments EIP (Enterprise Integration Project) M M • Uncertainty over ability to integrate CIM models may impact data and system quality and testing CIM (Common Information Model) M H • Working with 3 main vendors on multiple iterations of development and validation of CIM import/export • Investigating contingency plan to preserve April 1st start of DAM / RUC EDS trials • Prioritizing build out of environments and applications based on EDS schedule • Planning active / standby EDS environment (Austin / Taylor) to eliminate short-term ITest need • Building test environment in Blue Building server room • Removing all non-essential equipment from Austin data center • Compressing Nodal deployment footprint as much as possible Infrastructureconstraints (power, space, cooling) • Limits speed and scale of environment build outs, impacts development projects, integration and EDS trials H M • ERCOT working with vendor to rectify defects/outstanding. issues • Single entry model must be in place by July to verify go-live model NMMS • Single entry model could be late M H Texas Nodal Program Update
IBM Review - Dec 07 – Jan 08 - Some issues involve quality processes Texas Nodal Program Update
Quality - Summary Severity 1 – Data Loss/Critical Error Severity 2 – Loss of Functionality w/o workaround Severity 3 – Loss of functionality with workaround Severity 4 – Partial Loss of a feature set Severity 5 – Cosmetic/Documentation Error Severity 6 - Enhancement Quality has experienced recent successes but concerns remain • Successes • NMMS has fixed a number of high severity defects (2 fixed of 3) • CRR has reduced the overall number of defects; currently only 3 Severity 2 defects exist (No severity 1 defects currently) • Areas of Concern • Defect resolution time • High percentage of EMS defects not detected till EDS Mitigation Measures • Passive data center site with live data feed will allow for earlier defect detection • Defect Classification Texas Nodal Program Update
Quality details: Nodal Test Results per Project • Since December 2007, the Program undertook a significant documentation and tracking process. • Significant visibility into quality status of each project: • December 2007: 5 projects tracked • March 2008: 14 projects tracked (others are not in the testing phase) • Failure rates within acceptable limits, especially with Severity* 1 and 2 defects. * Note: Severity 1 and 2 defects are the most significant of the 4 types. December 17, 2007 March 4, 2008 Texas Nodal Program Update
Quality Details: Nodal FAT Active Defects by Severity by Project • Few severity 1 or 2 defects in the Market Management System (MMS) • No severity 1 defects in Energy Management System (EMS) • Commercial System Registration (COMS) is still early in development. In Pre FAT • The Network Model Management System (NMMS) has a few Sev 1 defects • Overall, active defects are being tracked by project and by classification. • The relative low number of Sev 1 and 2 defects is somewhat encouraging. (Note - Does not include Closed and Deferred) Texas Nodal Program Update
Quality Details: Nodal Active Defects by Severity Trend • Severity 3 defect trend is rising due to focus on fixing higher priority severity 1 and 2 defects • Severity 2 trend is remaining steady which indicates “defect open rate” is only slightly higher than defect closure rate Sev 3 Sev 4 Sev 2 Sev 1 Texas Nodal Program Update
Quality Details: Average Days to Fix Defect by Vendor • The larger project vendors are doing fairly well in their turn around time for defect fixing. There are known and understood reasons for the other vendors. • What is not shown here is the turnaround by defect classification. • Overall, the time to fix defect in various categories gives us a good indication on: • how long the product will be in testing • what the next release will look like • how long will it take to get to market trials. • a good indication of ability to meet the 168 hour test. Texas Nodal Program Update
Quality Details: Nodal Defects Reopened % (After Fixing by Vendor) • ABB and Areva are fixing their products with a good low reopen rate. • Good indication that the Energy and the Market Management System are conducting good quality reviews at the vendor site prior to releases being given back to ERCOT and the market. • This metric is being tracked to determine trends. Indicates what percentage of fixes delivered by a vendor retested unsuccessfully. Texas Nodal Program Update
Quality Details: Defects found in the Early Delivery System Severity 1 – Data Loss/Critical Error Severity 2 – Loss of Functionality w/o workaround Severity 3 – Loss of functionality with workaround Severity 4 – Partial Loss of a feature set Severity 5 – Cosmetic/Documentation Error Severity 6 - Enhancement • We released EMS 2 through 3 too early, resulting in defects showing up in the Early Delivery System • No severity 1 defects in Energy Management System (EMS) • Commercial System Registration (COMS) is still early in development. In Pre FAT • The Network Model Management System (NMMS) has a few Sev 1 defects • Overall, active defects are being tracked by project and by classification. • The relative low number of Sev 1 and 2 defects is somewhat encouraging. Severity 6 Severity 5 Severity 4 Severity 3 Severity 2 Severity 1 Texas Nodal Program Update
Schedule: “Health Checks” Jan Jun Feb Jul Aug Mar Apr Sep Oct May Nov Dec Still targeting a 12/1/08 Go-Live date; critical pathruns through EDS activities 1 EDS 1 E1 R2 ICCP / P2P EDS 2 E2 R3 SE Tuning E2 R4 MP Trials & NM Verification EDS 3 E3 R5 SCED & LMP Posting for 6 months E3 R6 LFC Testing E3 R7 CRR Testing EDS 4 E4 R8 CMM, DAM & RT Settlements, Verifiable Costs & Disputes E4 R9 DAM, SASM & RUC execution, DAM Stability Test, EECP, DST, Compliance Business Processes Today Go-Live (Go/No-Go) Performance 2 CIM Defect Integration Health Checks – measure of schedule viability EDS 4 168 Hr Test Go Live Sequence Dictated by PRR 727: Market Readiness Certification 3 32 Days to complete Test ERCOT BoD: 10/21 Sept Oct Nov 168 Hr Test Start TAC Meeting: 10/2 First Market Notice: 10/31 Second Market Notice: 11/21 Texas Nodal Program Update
Schedule: This is the metric status for each of the Early Delivery Systems Jun Jan Jul Feb Aug Mar Sep Apr May Oct Nov Dec EDS activities are monitored and measured by 59 metrics EDS 1 E1 R2 (1 EDS, 2 MP) ICCP / P2P EDS 2 (5 EDS, 3 MP) E2 R3 SE Tuning E2 R4 (3 MP) MP Trials & NM Verification EDS 3 (2 EDS, 2 MP) E3 R5 SCED & LMP Posting for 6 months (2 EDS, 1 MP) E3 R6 LFC Testing (5 EDS, 1 MP) E3 R7 CRR Testing EDS 4 (3 EDS) E4 R8 CMM, DAM & RT Settlements, Verifiable Costs & Disputes (12 EDS, 2 MP) E4 R9 DAM, SASM & RUC execution, DAM Stability Test, EECP, DST, Compliance 168 Hr Test Go Live Sequence Today (xx) – Number of metrics associated with a particular Release Overall Readiness: (15 metrics) EDS1 – Complete EDS2 – Overall Red resulting from ongoing issues with State Estimator EDS3 – Yellow resulting from LMP performance and delays in LFC / ACE testing EDS4 – Mostly unmeasured at this point, but inheriting Red status from recently moved Alarms Processing metric. Commercial Ops metrics being measured are Green Overall Readiness – Yellow due to open issues on telemetry, modeling, and CIM Texas Nodal Program Update
Outage Scheduler - Need for periodic duel entry of outages • ERCOT will develop scripts to alleviate duel entry during most periods: • Between minor tests • Between the 168 hr. test and go-live • There will be times, especially during the 168 hour test where zonal and nodal systems will require MPs to enter data into two systems • Solid Bars below indicate duel entry requirement, estimated to be a total of 45 days August December September October November Dual Entry to Support 168 Hour Test Dual Entry just prior to Go – Live. Minor tests * (hashed bar if 168 hour test extends to the full month Texas Nodal Program Update
Cost: Element Summary EAC Revised Budget and Forecast remain constant; spending in January expected to be lower and should reverse much of the overrun through Dec. January actuals are still unavailable as of 5 March Texas Nodal Program Update