WECC Compliance Metrics Report March 2013

1. WECC Compliance Metrics Report March 2013

2. WECC Compliance Report 1 Index of Slides 2 Audit and Spot Check Reports 3 Violation Review and Validation-Data 4 Violation Review and Validation-Line Chart 5 Mitigation Plan Review-Data • Mitigation Plan Review-Line Chart • Completed Mitigation Plan Review-Data • Completed Mitigation Plan Review-Line Chart 9 Violations Received (Reporting Method) 10 Violations Received (Shown by Percent) Appendix (attached): Detailed explanation for each measure

3. Non-public Audit & Spot Check Reports Measure: Average days to produce audit report and file with NERC (“non-public” = violations are not yet processed to finality) Spot Check data includes only those since 1/1/2011 when reports were required to be sent to NERC. “Days Outstanding” measures average days that the incomplete reports are pending Goal: Per CMEP, “normally” 60 days for audit reports; 90 days for spot check reports if there are no violations) * Year-to-date

4. Violation Review and Validation Measure: Average days to complete reviews for validated (enforceable) violations reviewed during the referenced period. Includes technical review. Upon completion of review validated violations are sent to Enforcement staff for further processing and final disposition. This data does not include dismissed violations. Goal: 60 days (Internal; no CMEP requirements) # CIP pending review as of 2/28/2013: 44 # O&P pending review as of 2/28/2013: 28 * Quarter-to-date

5. Violation Review and Validation Shows information from the preceding slide (slide 3) Goal: 60 days (Internal; no CMEP requirements) # pending review as of 2/28/2013: 72 # pending review > 90 days as of 2/28/2013: 9 * Quarter-to-date

6. Mitigation Plan Review Measure: Average days to complete reviews for accepted Mitigation Plans reviewed during the referenced period Does not include rejected MPs or MPs with dismissed violations Goal: 60 days (Internal. CMEP: Region has 30 days to accept, reject, or extend review. If the MP is not reviewed within 30 days, WECC always extends the review within 30 days.) # CIP pending review as of 2/28/2013: 8 # O&P pending review as of 2/28/2013: 13 * Quarter-to-date

7. Mitigation Plan Review Measure: Shows information from the preceding slide (slide 5) Goal: 60 days (Internal. CMEP: Region has 30 days to accept, reject, or extend review. WECC always extends the review within 30 days, if necessary.) # pending review as of 2/28/2013: 21 # pending review > 90 days as of 2/28/2013: 0 * Quarter-to-date

8. Completed Mitigation Plan Review Measure: Average days to complete reviews for accepted Completed Mitigation Plans reviewed during the referenced period Does not include rejected CMPs or CMPs with dismissed violations Goal: 60 days (Internal; no CMEP requirements) # CIP pending review as of 2/28/2013: 67 # O&P pending review as of 2/28/2013: 20 * Quarter-to-date

9. Completed Mitigation Plan Review Measure: Shows information from the preceding slide (slide 7) Goal: 60 days (Internal; no CMEP requirements) # pending review as of 2/28/2013: 87 # pending review > 90 days as of 2/28/2013: 36 * Quarter-to-date

10. Violations Reporting Method Measure: Enforceable violations (Reviewed violations that are not dismissed) shown by source (CMEP monitoring method) * "Other" includes Spot Check, Exception Report, Periodic Data Submittal, Compliance Investigation and Complaint ** Year-to-date

11. Violation Reporting Method Measure: Enforceable violations (Reviewed violations that are not dismissed), with source expressed as a percentage of enforceable violations. * "Other" includes Spot Check, Exception Report, Periodic Data Submittal, Compliance Investigation and Complaint ** Year-to-date

12. Appendix to Compliance Report Detailed Explanation of Measures

13. Appendix - Explanation of Measures Slide 2 Explanation: Audit and Spot Check Reports Measure: This data chart measures the average number of days it takes for the WECC audit team to produce an audit report following an audit and file it with NERC. It also measures the number of completed reports, the number of incomplete reports and the total number of days incomplete reports are outstanding. Measurement Period: One year. (4 years’ worth of audit reports and 3 years’ worth of spot check reports are shown) Purpose: WECC measures this to gauge our compliance with the CMEP time suggested; to gauge the audit and audit support staff efficiency and efficiency of both staff and tools. Terms: “Non public” :refers to final audit reports send to NERC. Audit reports are not posted publically until all due process relating to disposing of any violations are complete, i.e. a Notice of Penalty is approved by FERC. “Days outstanding” measures the average number of days that the incomplete reports have been pending. Goal: 60 days for audit report; 90 days for spot check report. Per the Compliance Monitoring and Enforcement Program) CMEP, “normally” audit reports would be filed with NERC 60 days following conclusion of the audit. The CMEP indicates that for spot check reports, these should be filed within 90 days if there are no violations. Notes: Spot Check data includes only those since 1/1/2011 when reports were required to be sent to NERC.

14. Appendix - Explanation of Measures Slides 3-4 Explanation: Violation Review and Validation Report (Data and Line Charts) Measure: These charts measure the average number of days staff (generally SMEs, Subject Matter Experts) takes to complete reviews of all NPVs (New Possible Violations) which are validated and transferred to WECC Enforcement staff for processing and disposition. It also measures the number of violations reviewed and validated and the number of violations pending review > 90 days at the end of the measurement period. Also noted are the total number of CIP and O&P violations currently pending review. Measurement Period: Quarter (six most recent quarters are shown) Purpose: WECC measures this as a gauge of staffing sufficiency given workload, and efficiency of both staff and processes. Terms: “NPV” is New Possible Violation. NPVs are reported and entered into the tracking system within five days after initial indication of a violation . “CMEP” is the FERC-approved Compliance Monitoring and Enforcement Program: Goal: 60 days from entry of the NPV to the date it is either dismissed or referred for further processing by WECC Enforcement staff. This is an internal goal; the CMEP specifies no goals or requirements. Notes: NPV are reviewed by an appropriate WECC SME, who may review the available record , request further information from the entity, or issue data requests. At that point, the NPV is either: (1) Dismissed (meaning that upon review the SME determined that no violation in fact exists) or (2) “Validated” and becomes an “Alleged Violation” which is forwarded to the WECC Enforcement staff for appropriate disposition (for example, through FFT, Find, Fix and Track, or an Expedited Settlement Agreement, Notice of Penalty, or Spreadsheet NOP). This measure currently does not include violations that are dismissed. The dismissal data in WECC’s new webCDMS system includes a number of out-of-process outliers, which can skew the average days calculation. WECC is working with the webCDMS vendor to add the ability to flag these outliers for exclusion.

15. Appendix - Explanation of Measures Slides 5-6 Explanation: Mitigation Plan Review (Data and Line Charts) Measure: These charts measure the average number of days staff (generally SMEs, Subject Matter Experts) takes to complete reviews of Mitigation Plans (MPs) which were accepted during the referenced period. It also measures the number of MPs reviewed and accepted and the number of MPs pending review > 90 days at the end of the measurement period. Also noted are the total number of CIP and O&P MPs currently pending review. These measures do not include MPs that were rejected after review or that pertain to violations that ultimately were dismissed. Measurement Period: Quarter (six most recent quarters are shown) Purpose: WECC measures this as a gauge of staffing sufficiency given workload, and efficiency of both staff and processes. Terms: “MP” is Mitigation Plan. Entities are required to file these on a requirements (rather than Standard) level. Thus, for example, violations of two different requirements relating of a single standard would result in two MPs. “CMEP” is the FERC-approved Compliance Monitoring and Enforcement Program: Goal: 60 days from submittal of MP to the date it is either accepted or rejected. This is an internal goal; the CMEP specifies the region has 30 days to accept, reject, or extend the review period. WECC always extends the review period within 30 days, if necessary. Notes: WECC believes that 30 days for review is not a realistic goal. Under the CMEP, entities are not required to file MPs until after issuance of the NOAV (Notice of Alleged Violation), if not contested. Yet entities are encouraged to file MPs as quickly as possible once they believe they are in violation. It is not unusual for an entity to file a Self Report, or certify non-compliance, simultaneously (or nearly so) with a corresponding MP. The dilemma is that WECC cannot assess the sufficiency of the MP until it understands the violation by performing its review (initial validation of the New Possible Violation, then Enforcement staff assessing of the scope and risk of the Alleged Violation). The CMEP contains no measures for these violation review activities, but it’s only after doing performing them that staff assess whether it can accept the MP. All these activities easily can take longer than 30 days. Figures also include the federal cases on hold (pending legal resolution). In these cases MPs are not required and entities have been submitting voluntarily.

16. Appendix - Explanation of Measures Slides 7-8 Explanation: Completed Mitigation Plan Review (Data and Line Charts) Measure: These charts measure the average number of days staff (generally SMEs, Subject Matter Experts) takes to complete reviews of Completed Mitigation Plans (CMPs) which were accepted during the referenced period. It also measures the number of CMPs reviewed and accepted and the number of CMPs pending review > 90 days at the end of the measurement period. Also noted are the total number of CIP and O&P CMPs currently pending review. These measures do not include CMPs that were rejected after review or that pertain to violations that ultimately were dismissed. Measurement Period: Quarter (six most recent quarters are shown) Purpose: WECC measures this as a gauge of staffing sufficiency given workload, and efficiency of both staff and processes. Terms: “CMP” is Completed Mitigation Plan, which includes a Certification of Mitigation Plan Completion and supporting evidence. “CMEP” is the FERC-approved Compliance Monitoring and Enforcement Program: Goal: 60 days from submittal of CMP to the date it is either accepted or rejected. This is an internal goal; the CMEP specifies no goals or requirements. Notes: CMP reviews are generally more involved and time consuming compared to MP reviews. In addition to a Certification of Mitigation Plan Completion document, an entity must submit evidence, which demonstrates the mitigating activities outlined in a particular MP are complete. These CMP reviews not only require WECC SMEs to review the evidence provided by entities, but also SMEs often need to issue data requests in order to get the information necessary to verify MP completion. Figures also include the federal cases on hold (pending legal resolution). In these cases MPs are not required and entities have been submitting voluntarily.

17. Appendix - Explanation of Measures Slides 9-10 Explanation: Enforceable Violations and Enforceable Violations by Percent Measure: These charts measures the number of enforceable violations by reporting method: Self-Report; Self-Certification; Audit and Other, and shows the breakdown by percent as well. Measurement Period: One year (4 most recent years are shown) Purpose: This does not measure performance or efficiency. It is used to track the number of discovered violations relative to the various reporting methods, primarily to assess trends. This information occasionally is requested by stakeholders. Terms: "Other" includes Spot Check, Exception Report, Periodic Data Submittal, Compliance Investigation and Complaint. “Enforceable violations” are violations that are reviewed and validated. Goal: N/A Notes: Dismissed violations are not enforceable.