1 / 38

Prepared by Opinion Dynamics Corporation May 2006

Prepared by Opinion Dynamics Corporation May 2006. Introduction. Opinion Dynamics was contracted to conduct a primary research effort to measure Market Participant perceptions of ERCOT’s performance with respect to meeting its responsibilities.

yeardley-duke
Download Presentation

Prepared by Opinion Dynamics Corporation May 2006

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prepared by Opinion Dynamics Corporation May 2006

  2. Introduction • Opinion Dynamics was contracted to conduct a primary research effort to measure Market Participant perceptions of ERCOT’s performance with respect to meeting its responsibilities. • Follow up and expansion of the 2004 Market Participant Survey. • Results allow for comparisons between market perceptions and operational realities and trending of performance over time.

  3. Methodology Three Phases: Phase 1: Review 2004 Market Participant Survey Instrument Phase 3: Survey of ERCOT’s Market Participants Phase 2: In-depth interviews with key ERCOT stakeholders

  4. Survey Sample ODC developed a sample of 920 unique Market Participants using the following lists provided by ERCOT staff: • Appropriate points of contact at market participant firms provided by Account Managers • Current ERCOT Board members • Current Committee members

  5. Response Rate287 completed surveys from a sample of 920 Market Participants (31%) Response Rate

  6. By Market Participant Firm Type

  7. Survey Approach: 10 point scale • Many questions based on a 10 point scale: 1-3 = negative response, 8-10 = positive response. • Mean responses will trend toward the middle of a 10 point scale – only those with passionate opinion are likely to provide a rating in top or bottom 3. • In general, mean responses of 6.6 or above are favorable ratings, 7.5 and above are extremely positive responses.

  8. Background and ContextMarket Participant Opinions RegardingERCOT Staff’s Role In Developing Market Rules Findings

  9. Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure Findings

  10. Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure Findings

  11. Background and ContextMarket Participants’ Understanding of ERCOT’s Committee Structure (cont.) Findings

  12. Background and Context Interest in Future Training Findings

  13. Overview of Perceived Strengths • Performance of ERCOT staff -- particularly account managers -- officers and directors • Communications -- ERCOT website • Timeliness, accuracy and format of data provided • Providing effective training Findings

  14. Color Key 2006 Needs Improvement 6.5 or below 2004 2006 6.6 to 7.4 Good 2004 2006 7.5 or above Very Good 2004 Findings

  15. ERCOT Staff Performance: Corporate Objectives (10 point scale, means shown) Nondiscriminatory access to transmission/ distribution 2006 2004 Ensuring Reliability/Adequacy of Grid 2006 2004 Accurate accounting of electric production & delivery 2006 2004 Timely information about customer’s choice of REP 2006 2004 Findings * Significantly different from 2004 at 95% confidence level.

  16. ERCOT Staff Performance: Staff (10 point scale, means shown) 2006 Industry expertise 2004 2006 Attitude 2004 2006 Overall Performance 2004 2006 Responsiveness to Market Participants 2004 2006 Consistency 2004 Findings * Significantly different from 2004 at 95% confidence level.

  17. ERCOT Staff Performance: Officers & Directors (10 point scale, means shown) 2006 Industry expertise 2004 2006 Consistency 2004 2006 Attitude 2004 2006 Overall Performance 2004 Responsiveness to Market Participants 2006 2004 2006 Management of ERCOT organization 2004 Findings

  18. ERCOT Staff Performance: Account Managers (10 point scale, means shown) Attitude/Willingness to resolve problem 2006 2004 2006 Direction of inquiries 2004 2006 Overall expectations 2004 2006 Accessibility 2004 2006 Response accuracy 2004 2006 Timely response 2004 2006 Knowledge/ Industry Expertise 2004 Findings * Significantly different from 2004 at 95% confidence level.

  19. ERCOT Staff Performance: Functional Areas(10 point scale, means shown) Market Participant Registration 2006 2004 Retail Transaction Processing/ Customer Switching 2006 2004 2006 Systems Testing 2004 2006 Bi-Lateral Scheduling 2004 Grid Operations 2006 2004 Settlements Dispute Resolution 2006 2004 Settlements and Billing 2006 2004 Findings * Significantly different from 2004 at 95% confidence level.

  20. ERCOT Staff Performance: PRR (10 point scale, means shown) 2006 Administering the process 2004 2006 Timeliness 2004 Testing software changes 2006 2004 Effective communication 2006 2004 2006 Understanding the effects 2004 2006 Implementing projects 2004 Findings

  21. ERCOT Staff Performance: Metering* (10 point scale, means shown) Completing data edits Knowledge Accuracy of data edits Timeliness of data edits Identifying need for data edits *New Question in 2006 Findings

  22. ERCOT Staff Performance: Variance Disputes (10 point scale, means shown) Retail Transaction Variance 2006 Data Extract Variance 2006 Retail Variance 2004 Findings

  23. Staff Performance: Finance and Accounting Credit Standards* (10 point scale, means shown) Effectively communicates collateral requirements to market participants Responds in a timely manner to questions regarding collateral requirements Responds well or effectively to questions regarding collateral requirements Findings * New questions in 2006.

  24. Communications (10 point scale, means shown) 2006 Amount of information 2004 2006 Use of Appropriate Mailing Lists** 2004 2006 Written Communication 2004 2006 Verbal Communication 2004 Clarity of ERCOT Staff’s Messages 2006 2004 2006 Timeliness 2004 * Significantly different from 2004 at 95% confidence level. ** New question in 2006. Findings

  25. Communications: Functional Areas(10 point scale, means shown) 2006 Asset Registration 2004 2006 Systems Changes 2004 2006 Bidding 2004 2006 Systems Planning 2004 Findings * Significantly different from 2004 at 95% confidence level.

  26. Communications: Functional Areas* (cont.)(10 point scale, means shown) Collateral requirements Grid operations Changes to IT systems Congestion management * New questions in 2006. Findings

  27. Satisfaction with the Website (10 point scale, means shown) 2006 Look of the site 2004 2006 Timeliness of posted information 2004 2006 Search functionality** 2004 2006 Ease of finding documents and info. 2004 * Significantly different from 2004 at 95% confidence level. ** New question in 2006. Findings

  28. Timeliness of Data(10 point scale, means shown) 2006 Transmission Congestion Rights 2004 2006 Settlements Bill 2004 2006 Renewable Energy Credits 2004 2006 Data Extracts 2004 Findings * Significantly different from 2004 at 95% confidence level.

  29. Accuracy of Data(10 point scale, means shown) 2006 Transmission Congestion Rights 2004 2006 Settlements Bill 2004 2006 Renewable Energy Credits 2004 2006 Data Extracts 2004 Findings * Significantly different from 2004 at 95% confidence level.

  30. Format of Data*(10 point scale, means shown) Renewable Energy Credits Transmission Congestion Rights Settlements Bill * New questions in 2006. Findings

  31. Effectiveness of Training(10 point scale, means shown) 2006 Settlement and Dispute Seminars 2004 2006 Retail Training Seminars 2004 2006 System Operations Seminars 2004 Findings * Significantly different from 2004 at 95% confidence level.

  32. Overview of Areas for Improvement • Spending priorities • IT Technical Helpdesk • Finance & Accounting Credit Standards • Project Management Findings

  33. ERCOT Spending Practices (10 point scale, means shown) 2006 Spending Funds Equitably 2004 Spending Funds on Things that are Important to Your Company 2006 2004 2006 Spending Funds Cost Effectively 2004 Findings

  34. Usefulness of Technical Helpdesk (10 point scale, means shown) Findings * Significantly different from 2004 at 95% confidence level.

  35. Finance and Accounting Credit Standards* (10 point scale, means shown) Credit exposure and related collateral requirements are calculated consistent with protocols Collateral requirements and credit standards are fairly implemented across the market Collateral requirements are adequate to mitigate the credit exposure in the market Findings * New questions in 2006.

  36. ERCOT Project Management Office (10 point scale, means shown) Knowledge/Industry expertise 2006 2004 Communicating progress of projects 2006 2004 Implementing projects based on system changes 2006 2004 Implementing projects that meet expectations 2006 2004 2006 Overall performance 2004 2006 Consistently applying cost-benefit analysis** 2004 2006 Spending money effectively 2004 Findings ** New question in 2006.

  37. Conclusions Market perceived areas of strength: • Performance with respect to corporate objectives • Systems are providing timely and accurate data • ERCOT staff performance across virtually all functional areas • Performance of Account Managers Conclusions

  38. Conclusions (cont.) Specific areas for strategic consideration: • Market’s understanding of the role of committees / governance in setting spending priorities • Defining and communicating ERCOT staff’s market function • The market is critical of ERCOT’s project management -- More research on this issue is necessary. Conclusions

More Related