The Relationship between Reliability Standard Audit Worksheets and Data Requests

1. The Relationship between Reliability Standard Audit Worksheets and Data Requests April 10, 2019 Mike Wells Sr. Auditor O & P Eric Weston Cyber Security Audits & Investigations

2. Agenda • Intro • Tools of a NERC auditor • Familiarizing ourselves with Reliability Standard Audit Worksheets (RSAW) • Types of Data Requests (DR) • Improving RSAWs • Improving DR responses • Closing

3. Tools of a NERC Auditor • Inherent Risk Assessment (IRA) • RSAWs • Data Requests • Protection Systems Maintenance Summary (Pre-audit Request) • CIP Request for Information (RFI) document (Pre-audit Request) • CIP Data Set (Pre-audit) • Follow-up Data Requests • Interviews • Site Visits

4. Audit Assessment Steps

5. Familiarizing Ourselves with RSAWs Compliance Narrative

6. Familiarizing Ourselves with RSAWs Registered Entity Evidence

7. Familiarizing Ourselves with RSAWs Compliance Assessment Approach

8. Types of Data Requests O&P Documentation Instructions in the Notice of Audit The document outlines and clarifies the information, reports, and data submitted to show compliance with the Requirements of the audit scope. Requests information and documentation in addition to evidence necessary to demonstrate compliance.

9. Types of Data Request O&P Documentation Instructions inthe Notice of Audit Examples: • System single-line diagram(s) • Communication diagram(s) showing data and voice functionality • Processes, plans for performance-based requirements • Lists of system elements

10. Types of Data Request O&P Request For Information Document Protection Systems Maintenance Summary PRC Standards FAC-501-WECC-2

11. Types of Data Request CIP Request For Information Document

12. Types of Data Request CIP Request For Information Document • Provided in the Notice of Audit package • Requests: • Process documentation • Evidence required to perform assessments • Special instructions for requested evidence

13. Types of Data Request CIP Data Set

14. Types of Data Request CIP Data Set • Part of the Notice of Audit Package • Requests: • List of BES Cyber Systems • List of ESPs and information pertaining to the ESP • List of Cyber Assets split out into Impact Ratings • Decommissioned Assets • Personnel • Transient Cyber Assets / Removable Media

15. Types of Data Request Sampling Data Requests “Sampling is essential for auditing and compliance monitoring because it is not always possible or practical to test 100% of either the equipment elements or documentation artifacts.”1 1—http://www.nerc.com/pa/comp/Documents/Sampling_Handbook_Final_05292015.pdf

16. Types of Data Request Common Sampling Standards • CIP-011 • FAC-003 • FAC-008 • PRC-005 • CIP-002 • CIP-004 • CIP-007 • CIP-010

17. Types of Data Request Site Visit Data Requests • Site visit request will be request from the CIP team and O&P team • Site visits may include: • Control Centers • Substations • Generation Facilities • Security Operations Centers • Network Operations Centers • Data Centers

18. Types of Data Request Interview Requests • Interviews are held to corroborate evidence and clarify understanding of entity’s programs and implementation • Interviews may not be held for all Standards in scope of an audit • Multiple interviews on a single Standard may be held in cases where the auditors are having trouble gaining reasonable assurance or following evidence

19. Types of Data Request General Data Request Through the course of an audit, the audit teams will submit data requests. Requests may include: • Request for evidence not provided with RSAW and initial DRs • Clarification of submitted evidence • Request for corroborating evidence

20. Improving RSAWs RSAW compliance narratives are a key tool in the audit assessment process A good RSAW narrative: • Can reduce the number of DRs • Streamline the audit engagement

21. Improving RSAWs What is the difference between processes for Standards and RSAW narratives? • Processes are written for your employees • RSAW narratives describe how your entity meets the Requirement • Similar to an executive summary • Can help auditors understand your entity’s organization • Steer auditors through the evidence • Clarify how your company approaches compliance

22. Improving RSAWs What should you include in the RSAW narrative? • Explanation of process and tools unique to your organization • Describe how evidence is generated • If process changes were implemented, describe the change • Don’t over complicate the narratives

23. Improving RSAWs Registered Entity Evidence • Ensure page numbers are referenced accurately. • Do the processes reference other documents? If so, provide the referenced documents.  • Are the documents relevant? 

24. Improving RSAWs Compliance Assessment Approach • Why is this section important for an entity?

25. Improving DR Responses Protection Systems Maintenance Summary Provide a narrative explanation in the RSAW for any omission of requested testing dates.

26. Improving DR Responses Request For Information Document • How do I use this document? • Do I need to fill something out?  • What if all the information requested is in our RSAWs?  • What if a Requirement is out of scope or not applicable to our environment? 

27. Improving DR Responses CIP Data Set Document • Information MUST be complete and accurate. Errors may lead to additional data requests. • Common issues are: • Misidentification of Cyber Assets • Listing assets multiple times • Listing personnel not associated with CIP • Entities may submit this document early

28. Improving DR Responses • Discuss uncertainty as to what the DR is requesting with the ATL before preparing the response • Ensure responses are relevant to the Standard and the request • Be clear with responses. If the requested evidence cannot be given, don’t give something that will not address the request.

29. Post Audit Assessments Many entities perform a post audit assessment. Keep these things in mind during the assessment: • The number of DRs does not indicate how well your audit went • Look for trends in data requests: • Do DRs ask for items that should have been included in the initial data submissions? • Do DRs ask for the same thing several times? • Were multiple interviews held for the same Standard and Requirements?

30. Mike Wellsmwells@wecc.org Eric Weston eweston@wecc.org