DOE O 414.1C, Quality Assurance “Improving Safety Software Quality”

1. DOE O 414.1C, Quality Assurance “Improving Safety Software Quality” Frank Russo Deputy Assistant Secretary Corporate Performance Assessment (EH-3)

2. Welcome

3. Video Conference Ground Rules • Mute remote locations • Silence cell phones • Hold questions until the end • Allow us to defer your question if it will be covered during the FAQs

4. DOE O 414.1C General Information Video Conference Agenda • Welcome (F. Russo) • DOE Expectations for Safety and Quality (R. Shearer) • Background - Order and Guide Development (F. Russo) • Safety Software Quality Responsibilities (F. Russo) • Safety Software Requirements (B. Danielson) • Field Experiences (C. Lagdon) • PSO Perspective (L. Dever, E. Schmidt, R. Singh, J. Poppiti, M. Janaskie) • Path Forward (R. Loesch) • Q&A (D. Sparkman, C. Lagdon, C. Thayer)

5. DOE Expectations for Safety and Quality Russell Shearer Principle Deputy Assistant Secretary Environment, Safety and Health

6. Background

7. Background • Importance of Software Quality Assurance as part of a Quality Assurance Program • New and revised Quality Assurance requirements in DOE directives • Majority of new requirements are for software quality assurance • First phase of a Department-wide rollout

8. Order and Guide Development • DOE EH-31 responsible for QA Order requirements • Surveyed applicable industry standards and guidance • Consulted DOE organizations, field staff, and SMEs • Working groups established for development of Guides • Wide spectrum of individuals • Selected based upon DOE organization, site location, and SME knowledge • Members authored sections in Guides • Members were SME reviewers for all sections

9. Informal and Formal Review Processes • Informal Reviews • SME Reviewers • Defense Nuclear Facility Safety Board • Formal Process • RevCom • Hundreds of valuable comments

10. Thank You • DOE G 414.1-2A Quality Assurance Writing Team

11. Thank You • DOE G 414.1-4 Safety Software Quality Assurance Writing Team

12. Roles and Responsibilities

13. Order Implementation Roles and Responsibilities • EH roles and responsibilities • DOE’s independent element responsible for safety of the public, worker and environment • Develops and maintains QA policy requirements, guides, and standards for all DOE work • Provides advise and assistance to DOE elements and contractors implementing DOE QA policy • Identifies and proposes resolutions for crosscutting QA issues • Manages DOE Safety Software Central Registry

14. Order Implementation Roles and Responsibilities continued • PSO roles and responsibilities • Ensure HQ, field elements, and contractors implement the requirements in the QA Order • Set priorities and schedule for compliance • Provide direction and resources, including prioritization, for implementing the QA requirements • Review and approve QAPs or other documents that include implementation approaches for the QA requirements

15. Safety Software Requirements Gustave E. (Bud) Danielson, Jr. Quality Policy Manager Office of Quality Assurance Programs

16. Significant Changes to DOE O 414.1C • Addition of SQA for safety software • Cancels DOE N 411.1, SQA Functions, Responsibilities and Authorities • Reflect existing requirements in 10 CFR 830 • Order references 1 updated and 2 new Guides • DOE G 414.1-2A, QA Management System • DOE G 414.1-3, Suspect/Counterfeit Items • DOE G 414.1-4, Safety Software • Inclusion of aviation safety into Corrective Action Management Program (CAMP)

17. Significant Changes to DOE G 414.1-2AQuality Management • Strengthened the use of a single management system or work process for similar requirements • Incorporated review guidance for use by DOE in evaluating contractor quality management systems • Discussed the use of third party management system validation • New generic QAP assessment plan

18. Significant Changes to DOE G 414.1-2AQuality Management continued • Updated information pertaining to the identification and control of suspect/counterfeit items (S/CIs) and links to expanded guidance for S/CIs • Expanded information on the grading process to include programmatic and mission‑critical considerations and a description of the steps in implementing the grading process • Expanded the description of identification, tracking, and resolution of quality problems • Clarified the guidance on procurement processes for nuclear safety applications

19. New Guidance with DOE G 414.1-3 Suspect & Counterfeit Items • Issued November 3, 2004 • Incorporates the new complex-wide S/CI Process • Updates S/CI information and resources • Supersedes DOE G 440.1-6, Suspect Counterfeit Items Guide for use with DOE O 440.1, Worker Protection Management; 10 CFR 830.120; and DOE O 5700.6C, Quality Assurance

20. Safety Software Changes to DOE O 414.1C • Scope of 10 CFR 830 and DOE O 414.1B Included software related to nuclear (including radiological) facilities • Establishes specific category of software, SAFETY SOFTWARE • Identification of Safety Software definitions, responsibilities and requirements

21. Safety Software Definitions • Safety System Software: Software for a nuclear facility that performs a safety function as part of a structure, system, or component and is cited in either (a) a DOE approved documented safety analysis or (b) an approved hazard analysis per DOE P 450.4, Safety Management System Policy, dated 10‑15‑96, and the DEAR clause.

22. Safety Software Definitions continued Safety and Hazard Analysis Software and Design Software: Software that is used to classify, design, or analyze nuclear facilities.  This software is not part of a structure, system, or component (SSC) but helps to ensure the proper accident or hazards analysis of nuclear facilities or an SSC that performs a safety function.

23. Safety Software Definitions continued • Safety Management and Administrative Controls Software: Software that performs a hazard control function in support of nuclear facility or radiological safety management programs or technical safety requirements or other software that performs a control function necessary to provide adequate protection from nuclear facility or radiological hazards. This software supports eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment as addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause.

24. Significant Changes to DOE O 414.1C continued • Invokes ASME NQA-1-2000,supplemented by other consensus standards or other consensus standards that provide an equivalent level of SQA requirements • Sites define grading levels and consensus standards in DOE approved QAP or other appropriate DOE approved document • Using the grading levels, select and implement the applicable software quality assurance work activities defined in the Order • Identification, documentation, and maintenance of safety software inventory

25. Significant Changes to DOE O 414.1C continued • Identifies 10 SQA work activities • Software project management • Software risk management • Software configuration management • Procurement and supplier management • Software requirements identification and management • Software design and implementation • Software safety • Verification and validation • Problem reporting and corrective action • Training of personnel in the design, development, use and evaluation of safety software

26. New Guidance in DOE G 414.1-4 Safety Software • Facilitates implementation of SQA responsibilities and requirements • Detail guidance for each of the10 SQA work activities for safety software • Procedures for updating, adding and removing tool box codes from Central Registry • Cross references from ASME NQA-1-2000 to 10 CFR 830 and DOE O 414.1C • Sample Criteria Review and Approach Document for DOE O 414.1C

27. Impact • Update and maintain an inventory of safety software • NA and EM have initial lists • Grade safety software • Many sites institutional SQA programs include graded approach • Implement 10 SQA work activities • Basic SQA practices • Consistent with consensus standards • Map very closely to most sites institutional SQA program practices

28. Field Experiences Chip Lagdon Chief of Nuclear Safety (Acting)Energy, Science and Environment

29. Lessons Learned • Software Requirement Specification (SRS) and Software Design Document (SDD) are essential for developing quality software and life cycle maintenance. • Majority of software projects did not have SRSs and SDDs • Sites using the SRSs and SDDs have clear understanding of what was needed to develop and maintain software quality. • The sites without SRSs and SDDs appeared to be relying heavily on the available experts to ensure software is developed or procured to meet the project needs. Related SQA Work Activities Software requirements identification and management (#5) Software design and implementation (#6) Verification and validation (#8)

30. Lessons Learned continued • Software procurement specifications should specify details of software requirements, not just catalog data. • Sites procuring PLC’s for process systems only specified the vendors’ catalog model information as procurement specifications • Supporting documentation for the suitability and applicability of the technical requirements not included Related SQA Work Activities Software requirements identification and management (#5)

31. Lessons Learned continued • Formal procedures for software problem reporting and corrective actions for software errors and failures need to be maintained and rigorously implemented. • Many sites resolve software errors and corrective actions at the project level and maintain informal coordination with vendors or other effected entities. Related SQA Work Activities Procurement and supplier management (#4) Problem reporting and corrective action (#9)

32. Lessons Learned continued • Appropriate qualifications and training on software use is essential for proper use of safety software. • Very sophisticated and complex software are being used without appropriate training in their use. Related SQA Work Activities Training of personnel in design, development, use and evaluation of safety software (#10)

33. Lessons Learned continued • Appropriate software control and configuration management are essential for safe use of the software. • Lack of proper controlhas resulted in multiple versions being available at the same time and even some with known errors. • Deficiencies have been noted with configuration control in terms of software version and documentation. Related SQA Work Activities Software configuration management (#3)

34. SC Perspective Leah Dever Associate Director for Laboratory Operations Office of Science

35. NNSA Weapons Perspective Ed Schmidt Director Office of Nuclear Weapon Surety and Quality

36. NNSA Perspective Rabi Singh NNSA QA Roadmap Leader National Nuclear Security Administration

37. EM Perspective James Poppiti Environmental Management

38. NE Perspective Mark Janaskie Office of Integrated Safety and Project Management Nuclear Energy, Science and Technology

39. Path Forward Robert Loesch Director (Acting) Office of Quality Assurance (EH-31)

40. Commitment and Accountability • The Department is committed to implementing SQA requirements as part of its overall Quality Assurance Program • We have worked extensively with NNSA (NA-121 & NA-124) and EM • Directives have been revised to reflect the SQA requirements along with roles and responsibilities • Central Registry established for “toolbox” codes • http://www.eh.doe.gov/sqa/central_registry.htm • QA and SQA web sites along with SQA List Server established to share information and solicit feedback

41. Upcoming Activities • Toolbox code upgrades • Other directive revisions • Minor changes to include reference to revised QA Order and new SQA Guide • QA and SQA Qualification Standards Update • Order Roll Out • Regional training and support • Site visits upon request • Newsletter series on 10 work activities • Safety software assessment support

42. Resources • EH QA Web Site • http://www.eh.doe.gov/qa/ • EH SQA Knowledge Portal • http://www.eh.doe.gov/sqa/ • SQA List Server • CENTREG@VM1.HQADMIN.DOE.GOV • EH S/CI Web Site • http://www.eh.doe.gov/sci/

43. Resources continued • EH Staff • QA - Bud Danielson 301-903-2954 bud.danielson@eh.doe.gov • SQA - Debra Sparkman 301-903-6888 debra.sparkman@eh.doe.gov • S/CI - Rick Green 301-903-7709 rick.green@eh.doe.gov

44. FAQs Debra Sparkman, Software Quality Engineer Gustave (Bud) Danielson, Jr. Quality Policy Manager Office of Quality Assurance Programs

45. FAQs 1. Why does the safety software definition include references to 10 CFR 835, DOE P 450.4, and the DEAR ISMS clause? 2. How were the 10 work activities determined? 3. Our site currently does not use NQA-1-2000. Will this be a big change for our programs?

46. FAQs continued 4. Our site’s SQA program is based on 10-CFR-830 , ASME NQA-1 2000, QC1, RW0333P, and DOE Orders. Our SQA / QA program and implementing procedures cover all software. Can we continue to use our  grading levels if they are different from those suggested in the Guide? 5. Facility design software used by a DOE contractor may be graded differently than the same software used by a supplier of design services to the DOE contractor. Why does DOE G 414.1-4 recommend different grading of the work activities?

47. FAQs continued 6. The Order is silent on software quality requirements for "non-safety software".  What software quality standards are required for "non-safety software"? 7. How do the safety software requirements in DOE O 414,1C apply to DOE weapons related work? 8. How do the safety software requirements in DOE O 414.1C differ from those in QC-1?

48. FAQs continued 9. Are there any changes in the way software users will be contacted on software bugs or major issues, especially with respect to software used by many contractors? 10. Is there a centralize list of safety analysis software used by DOE contractors? 11. How does the graded approach apply to safety software? Can you provide examples?

49. FAQs continued 12. Can a developer or contractor submit software to DOE to be considered a toolbox code and included in the Central Registry? 13. When will the contractor be required to comply with DOE O 414.1C?