Boot Camp - Conformity

1. Boot Camp - Conformity July 19, 2010 Detroit, USA

2. Boot Camp - Conformity • Overview • What does this Working Group do? • Previous meetings – Knoxville, San Fran, DC • Guiding principles • Terms we use • NIST Activities • Our Activities • Re-organization Slide 2 Bruce Muschlitz, EnerNex

3. Boot Camp – ConformityGoals of the working group • Coordinate Task Forces • Edge Conformity • Enterprise Conformity • Security Conformity • Provide overall guidance • Propose/Review task force deliverables Slide 3 Bruce Muschlitz, EnerNex

4. Boot Camp – ConformityPrevious Meeting (Knoxville) • Organized Group • Introduced 61850-10 as one models • Stressed conformance != interoperabilty • Explained abstract vs. detailed tests • Introduced “virtual” test environments Slide 4 Bruce Muschlitz, EnerNex

5. Boot Camp – ConformityPrevious Meeting (San Francisco) • Continued Group Organization • Discussed “plug-fest” – won’t do this • Discussed how other do this: • ISO Guide 65 • IEC 17011 and 17025 • http://www.rabnet.com • Discussed Product Mark (logo) • Recognized: 61850 Testing, SGIP TCC Slide 5 Bruce Muschlitz, EnerNex

6. Boot Camp – ConformityPrevious Meeting (McLean) • Organized Security Conformity • Re-organized by Horizontal teams • Discussed interaction with SGIP TCC • Recognized Edge/ENT might work better as (Physical) Device/ (Middleware) Interface Slide 6 Bruce Muschlitz, EnerNex

7. Boot Camp – ConformityGuiding Principles • Detailed Tests are not defined by UCAIug • Testers shall adhere to the defined tests • Equivalence of testers (no easy testers) • Tester shall produce “full” test results • Testers are free to script the tests Slide 7 Bruce Muschlitz, EnerNex

8. Boot Camp – ConformityCommon Terms • Conformance – meets spec? • Interop – plays well with others? • Positive tests – does it work right? • Negative tests – recovers gracefully? • Black Box tests – no inside knowledge • White Box tests – view algorithms Slide 8 Bruce Muschlitz, EnerNex

9. Boot Camp – ConformityNIST Priority Action Plans Slide 9 Phil Beecher, PG&E

10. Boot Camp – ConformityNIST SGIP Standing Committees • Smart Grid Architecture Committee • Smart Grid Testing and Certification Committee • TCC committee will learn on our work • TCC will share some duties with out work Slide 10 Bruce Muschlitz, EnerNex

11. Conformity Activities • Common Glossary • Product Mark presentation • Templates – test cases, use cases • TISSUEs (Technical Issues) • “Conformity Requirements Document”

12. Re-Organization • Original organization based upon physical location of devices (distance from enterprise) • New organization based upon interface functionality (real or virtual) • Real – wired or wireless protocols • Virtual - APIs

13. Boot Camp – ConformityBackground Material • http://osgug.ucaiug.org • http://www.ucaiug.org • http://www.rabnete.com • http://www.iec.ch/helpline/sitetree/ conformity • http://collaborate.nist.gov/twiki-sggrid/bin/view/SmartGrid/WebHome Slide 13 Bruce Muschlitz, EnerNex

14. Boot Camp – ConformityQuestions? Slide 14 Phil Beecher, PG&E

15. Boot Camp Edge / Enterprise Conformity

16. Edge / Enterprise Conformity Activity • Certification Process Reference Manual • Define Abstract Test Cases

17. What is the CPRM? • Overview of device and system requirements • Identifies best practice for device and system protocol design • Defines the process used to define and maintain the quality of a Certification Program

18. Guiding Principles • Open standards based • Clean, layered architecture • Robust certification program • Focussed on application programming interface, not specific applications • Layered conformance testing • Performance testing considerations • Economically viable

19. Organisational Requirements • Identify organisational structure to support a robust certification and interoperability testing program • Program management • Test laboratory qualification • Logo management • Change control • Dispute resolution • Devices and systems

20. Program Overview

21. Context (systems)

22. Context (products)

23. OpenSG SG Conformity – Security Conformity July 19, 2010 Bobby Brown

24. Security Conformity Task Force

25. Security Conformity TF Charter Establish security conformance requirements for laboratories desiring to certify smart grid components and systems Establish clear scoping boundaries, perform research to identify existing models, and propose a high-level philosophy of approach Chair: Bobby Brown, EnerNex representing Consumers Energy, bobby@enernex.com Vice-Chair: needed

26. Work Plan Reporting & Communication High-level Conformity Requirements Testing Use Cases

27. Goals Provide testing laboratories with best practice for cyber security testing. Provide environmental and technical considerations for entities developing internal cyber security testing processes Develop cyber security tools list and resources for security testing. Develop test cases Support funded efforts conducting actual testing to vet and assist in the development of the testing method, and cyber security metrics. Develop guidance for suppliers outsourcing cyber security testing – technical, procedural, communication, reporting, and status.

28. Conformance Definitions “Is any activity to determine, directly or indirectly, that a process, product, or service meets relevant standards and fulfills relevant requirements.” ISO/IEC Guide 2:2004 Conforms if… “has not been proven to be non-conformant with standard x”

29. Meeting Logistics Every Friday at 2:00PM Eastern Time (changing to every other week) OPENSG-SGCONFORM-SEC@SMARTGRIDLISTSERV.ORG Contact bobby@enernex.com for dial-in information and to be added to ListServ

30. Detroit Face-to-Face Agenda 8-10 AM Thurs July 22nd in Founders B-3 Review Charter & Work Plan Old Business AMI Use Cases Identify Standards (AMI)

31. Thank you! Bobby Brown, EnerNex 865-740-2844 bobby@enernex.com

32. Boot Camp – Edge ConformityQuestions? Slide 34 Bruce Muschlitz, EnerNex