ISASecure Device Test Development and Execution

1. ISASecure Device Test Development and Execution ISA99 Standards Committee Other Standards Organizations Marketplace Donors ISA99 Security Standards Other Standards, Regulations Market Donated IP ISA Security Compliance Institute Feedback on Gaps and Clarifications from Test Development and Execution (missing requirements) Feedback to ISA Security Compliance Institute ISASecure Test Specifications and Profiles Supplier Enhances Products/Systems (See details) Fail Test Execution Feedback to Supplier Pass ISASecure Compliant Products

2. Testing Profiles Testing Profiles Testing Profiles Device Device Device Device & System Device & System Device & System System System System ISASecure Device Conformance Test Development Path Standards Organizations ISA Security Compliance Institute ISA99 Standards ISA100 Standards IEC Standards DHS Requirements NERC Standards FERC Standards Other ISASecure Conformance Requirements ISASecure Test Kit Specification (includes test plan) WHAT ISASecure Test Kit (Test cases, procedures, tools) WHAT HOW with tools and procedures defined Tools and procedures

3. Harmonizing Market Supplied (Donated) IAC Security Conformance Requirements The harmonization process should follow the Conformance Test Development path with the benefit that specific work products should already exist as part of the donated IP; specifically the Conformance Requirements Document and the corresponding Test Kit ISCI TSC Issues a public call for input on ISASecure Conformance requirements (example: network attacks) Donated conformance requirements are evaluated for quality, format, completeness. Reject poorly constructed/ unusable requirements. Formally approved conformance requirements from TSC are sent to Governing Board for formal approval based on 2/3 majority of ALL voting Board Members. Approved conformance requirements submitted to ISA SP99 for consideration in standard. Donated conformance requirements are entered into a spreadsheet to identify duplications and gaps for analysis by TSC. TSC evaluates test kits against conformance requirements for approval as ISASecure test vendor. Forwards Recommendation to Governing Board. Test vendors update tests based on approved conformance requirements. TSC reviews Conformance requirements and gains consensus on requirements to include in ISASecure through a vetting process (2/3 majority). Governing board votes to approve test vendor for ISASecure (2/3 majority of all board members).

4. Testing Profiles Testing Profiles Testing Profiles Device Device Device Device & System Device & System Device & System System System System Harmonizing Market Supplied IAC Security Test Specifications Donor Organizations ISA Security Compliance Institute Evaluate whether the donated specifications include well-written Conformance Requirements (the ‘how’), Test Kit Specification and, the Test Kit For Example Network Attack Testing Mu Security Wurldtech Codenomicon Other ISASecure Conformance Requirements ISASecure Test Kit Specification (includes test plan) WHAT ISASecure Test Kit (Test cases, procedures, tools) WHAT HOW with tools and procedures defined Tools and procedures

5. ISASecure Logo Considerations • What does compliance mean? • Compliance by testing? • Compliance by verifiable/auditable process? • Other forms of compliance • Do we start with one with intent to evolve to something else?

6. ISASecure Compliance by Testing • Compliance Testing Approach • Works well for standard protocols • Fieldbus, OPC, TCP/IP • Can work for devices • Network connected only? • What about proprietary protocols? • What about open systems nodes? • What about systems?

7. ISASecure Compliance by Testing • Open systems node compliance • Testing for OS configuration • Testing for enabled services • What about systems that leverage additional services? • Testing OS security configuration • For Windows Systems • Compliance to Windows LOGO? • Which LOGO Standard? • Does this mean using VeriTest?

8. ISASecure Compliance by Testing • System Compliance • Network Infrastructure • Firewalls, routers, switches • Compartmentalization • Least privilege security configuration • Transferred risks • Role based security configurations • Application level security • …..

9. Conformance Testing Challenges • Approximately 50% of security issues are code bugs. • Compliance testing will uncover a majority of those bugs, but not all • Will also only find ones in 1st layer code not multiple layers down • Testing catches problems too late in the lifecycle • OK to start there but should drive behavioral change

10. Conformance Requirements • An additional area that causes security vulnerabilities is deployment errors • 30-40% of security compromises • Difficult to test deployment • Better to define deployment process and validate

11. Conformance Requirements • Process driven conformance • Similar to DO-178B for avionics products • Process conformance requirements • External audits for process conformance • IEC 61508 and 61511 also contain process conformance

12. Conformance by process • Conformance to Security standards • ISA SP99, others • Conformance for Security Assurance Levels • More objectives for higher assurance levels • DO-178B like • More objectives requiring independence • DO-178B like • Vendors must prove through evidence that required objectives have been met.