DOE O 414.1C Regional Meeting

1. 1 Gustave E. (Bud) Danielson, Jr. Quality Policy Manager Office of Quality Assurance Programs Debra Sparkman, Software Quality Engineer Lawrence Livermore National Laboratory for Office of Quality Assurance Programs DOE O 414.1C Regional Meeting

2. 2 Agenda Roles and Responsibilities Significant Changes to Order Safety Software Requirements Implementation Guidance Summary & Resources Questions welcomed throughout presentation

3. 3 Roles and Responsibilities

4. 4 EH Roles and Responsibilities Serves the Secretary as DOE�s independent element responsible for safety of the public, worker and environment Develops and maintains QA policy, requirements, guides, and standards for all DOE work Provides assistance to DOE &contractors Conducts Nuclear Safety Regulation Enforcement Implements Safety SQA for EH work From the QA Order, EH has the following responsibilities. These roles and responsibilities are supported by DOE�s Implementation Plans for both the Recommendation 2002-1 and 2004-1. DOE�s independent element responsible for safety of the public, worker and environment Develops and maintains QA policy requirements, guides, and standards for all DOE work Provides advise and assistance to DOE elements and contractors implementing DOE QA policy Examples include: QAP review assistance with EM and working with NNSA on there Roadmap. Identifies and proposes resolutions for crosscutting QA issues Examples include: Work with EFCOG and sponsoring the ISM Working Group and QA Subgroup EFCOG Weld Alert issued Attributes of Effective Quality Improvement Process � in review Manages DOE Safety Software Central Registry Central Registry was established for "Toolbox" codes that are commonly used to perform calculations to support safety analyses. The Central Registry assists users in configuration control and as a POC in resolving user issues.From the QA Order, EH has the following responsibilities. These roles and responsibilities are supported by DOE�s Implementation Plans for both the Recommendation 2002-1 and 2004-1. DOE�s independent element responsible for safety of the public, worker and environment Develops and maintains QA policy requirements, guides, and standards for all DOE work Provides advise and assistance to DOE elements and contractors implementing DOE QA policy Examples include: QAP review assistance with EM and working with NNSA on there Roadmap. Identifies and proposes resolutions for crosscutting QA issues Examples include: Work with EFCOG and sponsoring the ISM Working Group and QA Subgroup EFCOG Weld Alert issued Attributes of Effective Quality Improvement Process � in review Manages DOE Safety Software Central Registry Central Registry was established for "Toolbox" codes that are commonly used to perform calculations to support safety analyses. The Central Registry assists users in configuration control and as a POC in resolving user issues.

5. 5 Significant Changes

6. 6 DOE O 414.1C Safety Software Changes Safety software established as specific category of software Scope of 10 CFR 830 and DOE O 414.1C includes software related to nuclear (including radiological) facilities Identification of Safety Software definitions, responsibilities and requirements Federal staff with SQA responsibilities must be qualified according to TQP and DOE-STD-1172 Establishes specific category of software, SAFETY SOFTWARE The new Order created a category specific for this software, safety software. The scope is software as govern by 10 CFR 830 and DOE O 414.1C. This includes software related to nuclear facilities. Nuclear facility as defined by the Rule includes radiological facilities. Identification of Safety Software definitions, responsibilities and requirements Identified responsibilities and requirements for this category. New and existing systems are included in the scope. Existing systems, including systems sometimes referred to as legacy systems must meet the same requirements. The graded approach should be applied. The reasoning for no grandfathering of legacy systems is that if the safety software is still being used (either in operations or part of validating/updating the safety basis documents), the quality level of the software impacts the safe operation of a facility in the same manner as new safety software. Risk levels and the graded approach can be used to address any reduction in risk for systems that have been in operation (without modification) for several years. Establishes specific category of software, SAFETY SOFTWARE The new Order created a category specific for this software, safety software. The scope is software as govern by 10 CFR 830 and DOE O 414.1C. This includes software related to nuclear facilities. Nuclear facility as defined by the Rule includes radiological facilities. Identification of Safety Software definitions, responsibilities and requirements Identified responsibilities and requirements for this category. New and existing systems are included in the scope. Existing systems, including systems sometimes referred to as legacy systems must meet the same requirements. The graded approach should be applied. The reasoning for no grandfathering of legacy systems is that if the safety software is still being used (either in operations or part of validating/updating the safety basis documents), the quality level of the software impacts the safe operation of a facility in the same manner as new safety software. Risk levels and the graded approach can be used to address any reduction in risk for systems that have been in operation (without modification) for several years.

7. 7 Safety Software Definitions Safety System Software: Software for a nuclear facility that performs a safety function as part of a structure, system, or component and is cited in either (a) a DOE approved documented safety analysis or (b) an approved hazard analysis per DOE P 450.4, Safety Management System Policy, dated 10-15-96, and the DEAR ISMS clause [48 CFR 970.5223-1]. Tie sw back to DSA & TSR but the software itself may not be listed in these documents EH�s Basis for the definition change is: By using the term �nuclear facility� both reactor and nonreactor nuclear facilities classified as hazard category 1, 2, and 3 will be addressed and linked to a DOE approved DSA. Because a DSA is not required for �below hazard category 3�, the second part of the definition �an approved hazard analysis per DOE P 450.4, Safety Management System Policy and the DEAR clause� was included to cover �below hazard category 3� nuclear facilities (i.e., radiological facilities) and link it to a contractor approved hazard analysis required by DOE P 450.4 Safety Management System Policy and the DEAR clause. This definition addresses #2 under the 10 CFR 830 definition of �hazard controls�. 10 CFR 830 definitions to support Safety Software definition. Nuclear facility means a reactor or a nonreactor nuclear facility where an activity is conducted for or on behalf of DOE and includes any related area, structure, facility, or activity to the extent necessary to ensure proper implementation of the requirements established by this Part. In Response to Comments on the Interim Rule, the following was provided to the question what is a �below hazard category 3� nuclear facility: In DOE-STD-1027, these facilities are categorized as having no potential for significant offsite, onsite, or localized consequences. A �below category 3� nuclear facility is a DOE facility or activity that meets the definition of a nuclear facility but does not meet the threshold in DOE-STD-1027 for a hazard category 3 nuclear facility. These facilities are sometimes referred to as �radiological facilities.� See also Table 1 in Appendix A to Subpart B of the rule. Here�s the original definition as per the IP Safety system software is Computer software and firmware that performs a safety system function as part of a SSC that has been functionally classified as Safety Class (SC) or Safety Significant (SS). This also includes computer software such as human-machine interface software, network interface software, PLC programming language software, and safety management databases, that are not part of an SSC but whose operation or malfunction can directly affect SS and SC SSC function.Tie sw back to DSA & TSR but the software itself may not be listed in these documents EH�s Basis for the definition change is: By using the term �nuclear facility� both reactor and nonreactor nuclear facilities classified as hazard category 1, 2, and 3 will be addressed and linked to a DOE approved DSA. Because a DSA is not required for �below hazard category 3�, the second part of the definition �an approved hazard analysis per DOE P 450.4, Safety Management System Policy and the DEAR clause� was included to cover �below hazard category 3� nuclear facilities (i.e., radiological facilities) and link it to a contractor approved hazard analysis required by DOE P 450.4 Safety Management System Policy and the DEAR clause. This definition addresses #2 under the 10 CFR 830 definition of �hazard controls�. 10 CFR 830 definitions to support Safety Software definition. Nuclear facility means a reactor or a nonreactor nuclear facility where an activity is conducted for or on behalf of DOE and includes any related area, structure, facility, or activity to the extent necessary to ensure proper implementation of the requirements established by this Part. In Response to Comments on the Interim Rule, the following was provided to the question what is a �below hazard category 3� nuclear facility: In DOE-STD-1027, these facilities are categorized as having no potential for significant offsite, onsite, or localized consequences. A �below category 3� nuclear facility is a DOE facility or activity that meets the definition of a nuclear facility but does not meet the threshold in DOE-STD-1027 for a hazard category 3 nuclear facility. These facilities are sometimes referred to as �radiological facilities.� See also Table 1 in Appendix A to Subpart B of the rule. Here�s the original definition as per the IP Safety system software is Computer software and firmware that performs a safety system function as part of a SSC that has been functionally classified as Safety Class (SC) or Safety Significant (SS). This also includes computer software such as human-machine interface software, network interface software, PLC programming language software, and safety management databases, that are not part of an SSC but whose operation or malfunction can directly affect SS and SC SSC function.

8. 8 Safety Software Definitions continued Safety and Hazard Analysis Software and Design Software: Software that is used to classify, design, or analyze nuclear facilities.� This software is not part of a structure, system, or component (SSC) but helps to ensure the proper accident or hazards analysis of nuclear facilities or an SSC that performs a safety function. The original Implementation Plan for Recommendation 2002-1 was changed during the comment resolution process to include hazards analysis. EH�s Basis for the definition change is: Again, the term �nuclear facility� is used to address both reactor and nonreactor nuclear facilities classified as hazard category 1, 2, or 3 nuclear facilities. Adding �hazard� analysis to the definition is intended to cover �below hazard category 3� nuclear facilities (i.e., radiological facilities). This definition addresses #1 under the 10 CFR 830 definition of �hazard controls�. 10 CFR 830 definitions to support Safety and Hazard Analysis Software and Design Software Nuclear facility means a reactor or a nonreactor nuclear facility where an activity is conducted for or on behalf of DOE and includes any related area, structure, facility, or activity to the extent necessary to ensure proper implementation of the requirements established by this Part. In Response to Comments on the Interim Rule, the following was provided to the question what is a �below hazard category 3� nuclear facility: In DOE-STD-1027, these facilities are categorized as having no potential for significant offsite, onsite, or localized consequences. A �below category 3� nuclear facility is a DOE facility or activity that meets the definition of a nuclear facility but does not meet the threshold in DOE-STD-1027 for a hazard category 3 nuclear facility. These facilities are sometimes referred to as �radiological facilities.� See also Table 1 in Appendix A to Subpart B of the rule. Hazard analysis means the determination of material, system, process, and plant characteristics that can produce undesirable consequences, followed by the assessment of hazardous situations associated with a process or activity. Largely qualitative techniques are used to pinpoint weaknesses in design or operation of the facility that could lead to accidents. The Safety Analysis Report hazard analysis examines the complete spectrum of potential accidents that could expose members of the public, onsite workers, facility workers, and the environment to hazardous materials. DOE-STD-3009-94." Accident Analysis means those bounding analyses selected for inclusion in the Safety Analysis Report. These analyses refer to design basis accidents (DBAs) only. DOE 5480.21" Here�s the original definition as per the IP Safety analysis and design software is software that is not part of an SSC but is used in the safety classification, design and analysis of nuclear facilities to: ensure the proper accident analysis of nuclear facilities; ensure the proper analysis and design of safety SSCs; and ensure the proper identification, maintenance, and operation of safety SSCs. The original Implementation Plan for Recommendation 2002-1 was changed during the comment resolution process to include hazards analysis. EH�s Basis for the definition change is: Again, the term �nuclear facility� is used to address both reactor and nonreactor nuclear facilities classified as hazard category 1, 2, or 3 nuclear facilities. Adding �hazard� analysis to the definition is intended to cover �below hazard category 3� nuclear facilities (i.e., radiological facilities). This definition addresses #1 under the 10 CFR 830 definition of �hazard controls�. 10 CFR 830 definitions to support Safety and Hazard Analysis Software and Design Software Nuclear facility means a reactor or a nonreactor nuclear facility where an activity is conducted for or on behalf of DOE and includes any related area, structure, facility, or activity to the extent necessary to ensure proper implementation of the requirements established by this Part. In Response to Comments on the Interim Rule, the following was provided to the question what is a �below hazard category 3� nuclear facility: In DOE-STD-1027, these facilities are categorized as having no potential for significant offsite, onsite, or localized consequences. A �below category 3� nuclear facility is a DOE facility or activity that meets the definition of a nuclear facility but does not meet the threshold in DOE-STD-1027 for a hazard category 3 nuclear facility. These facilities are sometimes referred to as �radiological facilities.� See also Table 1 in Appendix A to Subpart B of the rule. Hazard analysis means the determination of material, system, process, and plant characteristics that can produce undesirable consequences, followed by the assessment of hazardous situations associated with a process or activity. Largely qualitative techniques are used to pinpoint weaknesses in design or operation of the facility that could lead to accidents. The Safety Analysis Report hazard analysis examines the complete spectrum of potential accidents that could expose members of the public, onsite workers, facility workers, and the environment to hazardous materials. DOE-STD-3009-94." Accident Analysis means those bounding analyses selected for inclusion in the Safety Analysis Report. These analyses refer to design basis accidents (DBAs) only. DOE 5480.21" Here�s the original definition as per the IP Safety analysis and design software is software that is not part of an SSC but is used in the safety classification, design and analysis of nuclear facilities to: ensure the proper accident analysis of nuclear facilities; ensure the proper analysis and design of safety SSCs; and ensure the proper identification, maintenance, and operation of safety SSCs.

9. 9 Safety Software Definitions continued Safety Management and Administrative Controls Software: Software that performs a hazard control function in support of nuclear facility or radiological safety management programs or technical safety requirements or other software that performs a control function necessary to provide adequate protection from nuclear facility or radiological hazards. This software supports eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment as addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause [48 CFR 970.5223-1]. The original IP for Recommendation 2002-1 included safety management software with the safety system software definition. During the comment resolution process, the original definition was reevaluated. It was obvious that the original definition was including 2 separate distinct types of software. Thus safety management and administrative controls was separated and defined to be linked closer to the hazards and TSRs. These will be handled on a case-by-case basis determined by its use and impact on safety. EH�s Basis for the definition change is: This definition is intended to address software that does not meet the definition of �Safety System Software� or �Safety and Hazard Analysis Software and Design Software� but performs a �hazard control� function as part of a safety management or administrative controls function. This software does not directly affect a safety SSC function. 10 CFR 830 definitions to support Safety Management and Administrative Controls Software definition. Administrative controls means the provisions relating to organization and management, procedures, recordkeeping, assessment, and reporting necessary to ensure safe operation of a facility. Hazard means a source of danger (i.e., material, energy source, or operation) with the potential to cause illness, injury, or death to a person or damage to a facility or to the environment (without regard to the likelihood or credibility of accident scenarios or consequence mitigation). Hazard controls means measures to eliminate, limit, or mitigate hazards to workers, the public, or the environment, including (1) Physical, design, structural, and engineering features; (addressed by the definition of Safety and Hazard Analysis Software and Design Software) (2) Safety structures, systems, and components; (addressed by the definition of Safety System Software) (3) Safety management programs; (4) Technical safety requirements; and (5) Other controls necessary to provide adequate protection from hazards. Safety management program means a program designed to ensure a facility is operated in a manner that adequately protects workers, the public, and the environment by covering a topic such as: quality assurance; maintenance of safety systems; personnel training; conduct of operations; inadvertent criticality protection; emergency preparedness; fire protection; waste management; or radiological protection of workers, the public, and the environment. The original IP definition as it related to safety management software. This also includes computer software � safety management databases, that are not part of an SSC but whose operation or malfunction can directly affect SS and SC SSC function. The original IP for Recommendation 2002-1 included safety management software with the safety system software definition. During the comment resolution process, the original definition was reevaluated. It was obvious that the original definition was including 2 separate distinct types of software. Thus safety management and administrative controls was separated and defined to be linked closer to the hazards and TSRs. These will be handled on a case-by-case basis determined by its use and impact on safety. EH�s Basis for the definition change is: This definition is intended to address software that does not meet the definition of �Safety System Software� or �Safety and Hazard Analysis Software and Design Software� but performs a �hazard control� function as part of a safety management or administrative controls function. This software does not directly affect a safety SSC function. 10 CFR 830 definitions to support Safety Management and Administrative Controls Software definition. Administrative controls means the provisions relating to organization and management, procedures, recordkeeping, assessment, and reporting necessary to ensure safe operation of a facility. Hazard means a source of danger (i.e., material, energy source, or operation) with the potential to cause illness, injury, or death to a person or damage to a facility or to the environment (without regard to the likelihood or credibility of accident scenarios or consequence mitigation). Hazard controls means measures to eliminate, limit, or mitigate hazards to workers, the public, or the environment, including (1) Physical, design, structural, and engineering features; (addressed by the definition of Safety and Hazard Analysis Software and Design Software) (2) Safety structures, systems, and components; (addressed by the definition of Safety System Software) (3) Safety management programs; (4) Technical safety requirements; and (5) Other controls necessary to provide adequate protection from hazards. Safety management program means a program designed to ensure a facility is operated in a manner that adequately protects workers, the public, and the environment by covering a topic such as: quality assurance; maintenance of safety systems; personnel training; conduct of operations; inadvertent criticality protection; emergency preparedness; fire protection; waste management; or radiological protection of workers, the public, and the environment. The original IP definition as it related to safety management software. This also includes computer software � safety management databases, that are not part of an SSC but whose operation or malfunction can directly affect SS and SC SSC function.

10. 10 The 5 Main Requirements

11. 11 1. Federal SQA Technical Qualifications Must be qualified according to the Technical Qualification Program and DOE-STD-1172: Review & evaluate safety software plans and processes Verify safety software plans and processes are based upon hazard and risk analyses Verify that safety software is developed, procured, verified, validated, used and maintained according to nuclear safety requirements Assess & monitor safety software QA programs Provide technical support for accident & occurrence investigations as they relate to safety software These are just some of the duties and responsibilities of the federal staff that is responsible for safety software. Refer to DOE Std 1172 (Qual Std) for a complete list.These are just some of the duties and responsibilities of the federal staff that is responsible for safety software. Refer to DOE Std 1172 (Qual Std) for a complete list.

12. 12 2. Facility Design Authority Involvement Safety software cannot and should not be separated from the �safety system� Facility Design Authority needs to be involved in all aspects of the safety software Specifications Acquisitions Design and development V&V CM Maintenance activities Retirement Identification of the safety software specification Acquisition To ensure safety rqmts are included in procurement or other acquisition documents Safety software design and development Includes involvement in the selection of standards V&V CM Safety software maintenance activities Safety software retirement Identification of the safety software specification Acquisition To ensure safety rqmts are included in procurement or other acquisition documents Safety software design and development Includes involvement in the selection of standards V&V CM Safety software maintenance activities Safety software retirement

13. 13 3. Safety Software Inventory Identify, document, and maintain the safety software inventory DSAs, TSRs and other safety documentation will assist in the identification Key attributes include a unique identifier (name and version # or date) Must be available to PSOs, regulators, and any oversight organizations NA and EM have initials lists NA has updated and validated their list as part of Roadmap for Nuclear Facility Quality Assurance Excellence Milestone #11. Recommended key attributes Safety software name and version id (unique identifier) DOE and site contact information DSA, TSR or other safety document where referenced Safety software use (if classification allows)NA and EM have initials lists NA has updated and validated their list as part of Roadmap for Nuclear Facility Quality Assurance Excellence Milestone #11. Recommended key attributes Safety software name and version id (unique identifier) DOE and site contact information DSA, TSR or other safety document where referenced Safety software use (if classification allows)

14. 14 3. Safety Software Inventory continued Focuses the application of DOE O 414.1C requirements to safety software Assists in applying engineering and financial resources effectively by focusing on software that impacts safety

15. 15 4a. Define Grading Level Define grading levels Document in QAP or other appropriate document DOE reviews and approves DOE G 414.1-4 grading levels Are recommended but can use any grading level as noted above Will be used by DOE EH for their safety software The Order does not specify a particular graded approach. The Sites define the grading level Reasoning behind the requirement Safety software has 3 different types of use (i.e., the 3 definitions) Safety software may be used differently by each site and perhaps by each facility Impact on safety most likely will vary from application to application of the safety software Effective (cost and resource) implementation of SQA work activities should be based upon the software�s impact on safety These grading levels and consensus standards must be documented in an approved QAP or other appropriate DOE approved document. The grading levels and standards are approved for use through DOE�s QAP process or other appropriate process.The Order does not specify a particular graded approach. The Sites define the grading level Reasoning behind the requirement Safety software has 3 different types of use (i.e., the 3 definitions) Safety software may be used differently by each site and perhaps by each facility Impact on safety most likely will vary from application to application of the safety software Effective (cost and resource) implementation of SQA work activities should be based upon the software�s impact on safety These grading levels and consensus standards must be documented in an approved QAP or other appropriate DOE approved document. The grading levels and standards are approved for use through DOE�s QAP process or other appropriate process.

16. 16 4b. Define Consensus Standards Order invokes ASME NQA-1-2000 Implementing organization defines standards Document in QAP or other appropriate document DOE reviews and approves all standards Standards that provide an equivalent level of SQA requirements to NQA-1 may be approved Determination and documentation of equivalency is required Crosswalk is needed DOE G 414.1-4 Table 3 provides assistance Sites define the consensus standard & document that in a DOE approved QAP. Invokes ASME NQA-1-2000,supplemented by other consensus standards or other consensus standards that provide an equivalent level of SQA requirements ASME NQA-1-2000 is the foundation standard for safety software. It may be supplied by other consensus standards when additional detail is required. Other standards that provide an equivalent level of SQA requirements as ASME NQA-1-2000 can be used if approved by DOE/NNSA. Reasoning behind the requirement ASME NQA-1-2000 is the single most comprehensive consensus standard for the nuclear industry ASME NQA-1-2000 provides a solid baseline for SQA work activities Versions of ASME NQA-1 already in use at many DOE sites Thus if the consensus standard is NOT ASME NQA-1-2000, an equivalence needs to be performed. Sites define the consensus standard & document that in a DOE approved QAP. Invokes ASME NQA-1-2000,supplemented by other consensus standards or other consensus standards that provide an equivalent level of SQA requirements ASME NQA-1-2000 is the foundation standard for safety software. It may be supplied by other consensus standards when additional detail is required. Other standards that provide an equivalent level of SQA requirements as ASME NQA-1-2000 can be used if approved by DOE/NNSA. Reasoning behind the requirement ASME NQA-1-2000 is the single most comprehensive consensus standard for the nuclear industry ASME NQA-1-2000 provides a solid baseline for SQA work activities Versions of ASME NQA-1 already in use at many DOE sites Thus if the consensus standard is NOT ASME NQA-1-2000, an equivalence needs to be performed.

17. 17 5. Select and Implement Work Activities Basic SQA practices Consistent with consensus standards Map very closely to most sites institutional SQA program practices All work activities are not applicable to every type of software (i.e. acquired vs. custom developed)

18. 18 10 Work Activities Software project management and quality planning Software risk management Software configuration management Procurement and supplier management Software requirements identification and management Software design and implementation Software safety Verification and validation Problem reporting and corrective action Training of personnel in the design, development, use and evaluation of safety software Identifies 10 SQA work activities These activities are common software quality assurance practices identified in industry standards and DOE SQA programs. Identifies 10 SQA work activities These activities are common software quality assurance practices identified in industry standards and DOE SQA programs.

19. 19 Implementation of Requirements

20. 20 Federal Line Management Activities Approve QAP or other document(s) that include graded approach and consensus standard(s) to be used Understand basis for software included in safety software inventory list (DSA, TSR, etc) Perform oversight functions, including assessments/reviews of safety software Understanding of the software included in the safety software inventory The problem being solved or the purpose for the safety software being used. The limitations of that safety software. Specifically for safety system software, the diagnostics and controls implemented to check the �health� of the safety software. The contingencies or workarounds should the safety software failUnderstanding of the software included in the safety software inventory The problem being solved or the purpose for the safety software being used. The limitations of that safety software. Specifically for safety system software, the diagnostics and controls implemented to check the �health� of the safety software. The contingencies or workarounds should the safety software fail

21. 21 EH Specific Activities Implement SQA safety software requirements according to DOE G 414.1-4 Filter Test Facility contractor Radiological and Environmental Sciences Lab Central Registry Safety software used to validate nuclear facility design or safety analysis Other instances of safety software use with EH For EH-31, the Central Registry criteria aligns with DOE G 414.1-4. The procedures to add new software to the Central Registry are being �test driven� with the evaluation of IMBA (a dose calculation software application) this winter.For EH-31, the Central Registry criteria aligns with DOE G 414.1-4. The procedures to add new software to the Central Registry are being �test driven� with the evaluation of IMBA (a dose calculation software application) this winter.

22. 22 Identify Safety Software Organization applying the software is responsible for identifying, evaluating and designating the software as safety software Based upon the requirements in DOE O�414.1C, and 10�CFR�830 Subparts A and B Application of the software determines the grading level Made by the organization applying the software The determination of what constitutes safety software is made by the organization applying the software Based upon the requirements Based upon the requirements in DOE O�414.1C, and 10�CFR�830 Subparts A and B. Application of the software The application of the software determines whether it is safety related and how it should be graded. Responsibility for designating the software as safety software lies with the organization using the software Therefore, the organization applying the software is responsible for evaluating and designating the software as safety software and then ensuring that the software development and operations have followed the appropriate QA proceduresMade by the organization applying the software The determination of what constitutes safety software is made by the organization applying the software Based upon the requirements Based upon the requirements in DOE O�414.1C, and 10�CFR�830 Subparts A and B. Application of the software The application of the software determines whether it is safety related and how it should be graded. Responsibility for designating the software as safety software lies with the organization using the software Therefore, the organization applying the software is responsible for evaluating and designating the software as safety software and then ensuring that the software development and operations have followed the appropriate QA procedures

23. 23 Define Grading Levels 3 grading levels recommended Based upon the safety software�s impact on safety Use of the safety software is the key Provides safety control Reliance on safety software results in making safety decisions Consider the software type (e.g., custom developed, configurable, acquired, utility calculations, and commercial design and analysis) For EH, this has already been done through the DOE G 414.1-4 3 Grading Levels DOE Guide recommends 3 levels for safety software Some sites use 2 levels for designating safety software with another 3 levels for non-safety software Choice is up to the sites but more than 1 is needed to provide grading. 3 seems the best fit to allow for a high, low and somewhere in between impact on safety Base grading levels on impact on safety Risk which is probability of failure and its consequence determine the impact on safety. The grading definitions need to address the impact on safety. Use of safety software influences the impact on safety Software that provides safety control, such as safety system software, may have a more direct impact on safety and thus reflected in the definitions of the grading levels. The grading levels need to address software that is used as part of the decision making process for categorizing, design, accident analysis and even movement of material or decisions regarding training. This type of software that most likely is identified as safety and hazard analysis software and design software or safety management and administrative controls software probably spans across multiple grading levels. For EH, this has already been done through the DOE G 414.1-4 3 Grading Levels DOE Guide recommends 3 levels for safety software Some sites use 2 levels for designating safety software with another 3 levels for non-safety software Choice is up to the sites but more than 1 is needed to provide grading. 3 seems the best fit to allow for a high, low and somewhere in between impact on safety Base grading levels on impact on safety Risk which is probability of failure and its consequence determine the impact on safety. The grading definitions need to address the impact on safety. Use of safety software influences the impact on safety Software that provides safety control, such as safety system software, may have a more direct impact on safety and thus reflected in the definitions of the grading levels. The grading levels need to address software that is used as part of the decision making process for categorizing, design, accident analysis and even movement of material or decisions regarding training. This type of software that most likely is identified as safety and hazard analysis software and design software or safety management and administrative controls software probably spans across multiple grading levels.

24. 24 Fold in Grading For each of the 10 work activities, review ASME NQA-1-2000 sections1 associated with each work activity Identify the specific sub-activities or tasks Determine the impact of performing (or not performing) each of these sub-activities/tasks in producing, procuring, and/or using safety software Note: 1. If using an approved �ASME NQA-1-2000 equivalent standard(s) review the appropriate sections in that standard(s) For EH, this has already been done through the DOE G 414.1-4 Note: 1. If using an approved �ASME NQA-1-2000 equivalent standard(s) review the appropriate sections in that standard(s) For EH, this has already been done through the DOE G 414.1-4

25. 25 Fold in Grading continued Then identify the work activity as: Needing to be fully implemented Has some but not all of the sub-activities needing to be performed Not applicable The recommended grading of the 10 work activities is contained in DOE G 414.1-4 Table 4 and Section 5.2

26. 26 Recommended Level A Meets one or more of the following criteria: Could initiate a limiting condition for operation (LCO) Could cause a reduction in the safety margin Could result in nonconservative safety analysis, design, or misclassification of facilities or SSCs

27. 27 Recommended Level B Does not meet Level A criteria but meets one or more of the following criteria: Aids in decision making that could impact safety SSC operation Could result in incorrect analysis, design, monitoring, alarming, or recording of hazardous exposures to workers or the public Could comprise the defense in depth capability

28. 28 Recommended Level C Does not meet Level B criteria but meets one or more of the following criteria: Could cause a potential violation of regulatory permitting requirements Could affect environment, safety, health monitoring or alarming systems Could affect the safe operation of a non-safety SSC

29. 29 Implementing the 10 Work Activities 

30. 30 Safety Software Work Activities Reminders - Not all work activities will be applicable to all software types Must use the best judgment of the software quality engineering and safety systems staffs when applying the graded approach

31. 31 Work Activity 1 Software Project Mgmt & Quality Planning Foundation to ensure a quality software product and results Flow down from system level Defines and guides the processes Identifies how the graded approach will be applied Identifies specific tasks Establishes quality goals As with any system, project management and quality planning are key elements to establishing the foundation to ensure a quality product that meets project goals. For software, project management starts with the system level project management and quality planning. Software specific tasks should be identified and either included within the overall system planning or in separate software planning documents. The software project management and quality planning should include identifying all tasks associated with the software development and procurement, including procurement of services, estimate of the duration of the tasks, resources allocated to the task, and any dependencies. Software quality and software development planning identifies and guides the software phases and any grading of the SQA and software development activities performed during software development or maintenance. The software quality and software engineering activities and rigor of implementation will be dependent on the identified grading level of safety software and the ability of DOE or its contractors to build quality in and assess the quality of the safety software. As with any system, project management and quality planning are key elements to establishing the foundation to ensure a quality product that meets project goals. For software, project management starts with the system level project management and quality planning. Software specific tasks should be identified and either included within the overall system planning or in separate software planning documents. The software project management and quality planning should include identifying all tasks associated with the software development and procurement, including procurement of services, estimate of the duration of the tasks, resources allocated to the task, and any dependencies. Software quality and software development planning identifies and guides the software phases and any grading of the SQA and software development activities performed during software development or maintenance. The software quality and software engineering activities and rigor of implementation will be dependent on the identified grading level of safety software and the ability of DOE or its contractors to build quality in and assess the quality of the safety software.

32. 32 Work Activity 1 Sw Project Mgmt & Quality Planning continued The details should include: All tasks associated with software development and procurements (e.g., hw, sw, PLCs and services) Estimate duration of tasks, resources allocated, and any dependencies between tasks Description of all tasks Documentation should be evaluated for: The need to be separate from the system level documentation The need to have more detail planning documents associated with a particular work activity (e.g. SCMP, SV&VP ASME NQA-1-2000 References, Req. 1; Req. 5; Req. 18; SP 2.7, 201; SP 2.7, 400; SP 4.1, 201; Part 3 1A-1, 200; Part 3 2A-2, 300 ASME NQA-1-2000 References, Req. 1; Req. 5; Req. 18; SP 2.7, 201; SP 2.7, 400; SP 4.1, 201; Part 3 1A-1, 200; Part 3 2A-2, 300

33. 33 Work Activity 2 Software Risk Mgmt Proactive decision making that continually assesses what can go wrong Identifies the important risks Includes risks associated with: Software development and procurement process (e.g., using unproven technology or supplier) Unsuccessful software program/project completion (e.g., high turnover of software staff, staff unfamiliar with developing safety software, software funding being cut) Software risk management provides a disciplined environment for proactive decision making to continuously assess what can go wrong, determine what risks are important to address, and implement actions to address those risks. Although sometimes associated with safety analysis of potential failures, software risk management focuses on the risks to the successful completion of the software project. Risk assessment and risk control are two fundamental activities required for project success. Risk assessment addresses identification of the potential risks, analysis of those risks, and prioritizing the risks to ensure that the necessary resources will be available to mitigate the risks. Risk control addresses risk tracking and resolution of the risks. Identification, tracking, and management of the risks throughout all phases of the project�s life-cycle should include special emphasis on tracking the risks associated with costs, resources, schedules, and technical aspects of the project. Several risk identification techniques are described and detailed in standards and literature. Risk resolution includes risk avoidance, mitigation, or transference. Even the small risks during one phase of the safety software application�s life have the potential to increase in some other phase of the application�s life with very adverse consequences. In addition, mitigation actions for some risks could create new (secondary) risks. Examples of potential software risks for the safety software application might include� incomplete or volatile software requirements; specification of incorrect or overly simplified algorithms or algorithms that will be very difficult to address within safety software; hardware constraints that limit the design; potential performance issues with the design; a design that is based upon unrealistic or optimistic assumptions; design changes during coding; incomplete and undefined interfaces; using unproven computer and software technologies such as programming languages not intended for the target application; use of a programming language with only minimal experience using the language; new versions of the operating system; unproven testing tools and test methods; insufficient time for development, coding, and/or testing; undefined or inadequate test acceptance criteria; and potential quality concerns with subcontractors or suppliers.Software risk management provides a disciplined environment for proactive decision making to continuously assess what can go wrong, determine what risks are important to address, and implement actions to address those risks. Although sometimes associated with safety analysis of potential failures, software risk management focuses on the risks to the successful completion of the software project. Risk assessment and risk control are two fundamental activities required for project success. Risk assessment addresses identification of the potential risks, analysis of those risks, and prioritizing the risks to ensure that the necessary resources will be available to mitigate the risks. Risk control addresses risk tracking and resolution of the risks. Identification, tracking, and management of the risks throughout all phases of the project�s life-cycle should include special emphasis on tracking the risks associated with costs, resources, schedules, and technical aspects of the project. Several risk identification techniques are described and detailed in standards and literature. Risk resolution includes risk avoidance, mitigation, or transference. Even the small risks during one phase of the safety software application�s life have the potential to increase in some other phase of the application�s life with very adverse consequences. In addition, mitigation actions for some risks could create new (secondary) risks. Examples of potential software risks for the safety software application might include� incomplete or volatile software requirements; specification of incorrect or overly simplified algorithms or algorithms that will be very difficult to address within safety software; hardware constraints that limit the design; potential performance issues with the design; a design that is based upon unrealistic or optimistic assumptions; design changes during coding; incomplete and undefined interfaces; using unproven computer and software technologies such as programming languages not intended for the target application; use of a programming language with only minimal experience using the language; new versions of the operating system; unproven testing tools and test methods; insufficient time for development, coding, and/or testing; undefined or inadequate test acceptance criteria; and potential quality concerns with subcontractors or suppliers.

34. 34 Work Activity 2 Software Risk Mgmt continued Addresses how to avoid, mitigate or transfer unacceptable risks Monitors those risks Takes necessary steps to bring risks to an acceptable level ASME NQA-1-2000 References, Part 3 2A-2, 301, 502; SP 2.7, 400; SP 4.1, 101, 200, 404, 406 ASME NQA-1-2000 References, Part 3 2A-2, 301, 502; SP 2.7, 400; SP 4.1, 101, 200, 404, 406

35. 35 Work Activity 3 Software Configuration Mgmt Includes all functions and tasks to: Identify the configuration items Control the items Establish configuration baselines Perform status accounting Perform configuration audits & reviews2 Note: 2. ASME NQA-1-2000 does not include this sub-activity/task SCM activities identify all functions and tasks required to manage the configuration of the software system, including software engineering items, establishing the configuration baselines to be controlled, and software configuration change control process. The following four areas of SCM should each be addressed when performing configuration management: (1)�configuration identification, (2)�configuration control, (3)�configuration status accounting, and (4)�configuration audits and reviews. Audits or reviews should be conducted to verify that the software product is consistent with the configuration item descriptions in the requirements and that the software, including all documentation, being delivered is complete. Physical configuration audits and functional configuration audits are examples of audits or reviews that should be performed. SCM work activities should be applied beginning at the point of DOE�s or its contractor�s control of the software. ASME NQA-1-2000 References, Req. 3, 802; Req. 6; SP 2.7, 201; SP 2.7, 203; SP 4.1, 201; SP 4.1, 203; SCM activities identify all functions and tasks required to manage the configuration of the software system, including software engineering items, establishing the configuration baselines to be controlled, and software configuration change control process. The following four areas of SCM should each be addressed when performing configuration management: (1)�configuration identification, (2)�configuration control, (3)�configuration status accounting, and (4)�configuration audits and reviews. Audits or reviews should be conducted to verify that the software product is consistent with the configuration item descriptions in the requirements and that the software, including all documentation, being delivered is complete. Physical configuration audits and functional configuration audits are examples of audits or reviews that should be performed. SCM work activities should be applied beginning at the point of DOE�s or its contractor�s control of the software. ASME NQA-1-2000 References, Req. 3, 802; Req. 6; SP 2.7, 201; SP 2.7, 203; SP 4.1, 201; SP 4.1, 203;

36. 36 Work Activity 4 Procurement & Supplier Mgmt Most projects have procurements or acquired software Commercial software (e.g., safety analysis, facility design applications) Embedded software (e.g., PLCs) Development tools (e.g., compilers, sw design, test tools) Developed by other DOE sites Include technical and quality requirements Specifications for features, including safety, security, and performance/timing Steps used to develop and validate safety software Documentation and test results to be delivered Requirements for supplier notification of defects, new releases or other issues Mechanism for users to report defects and request assistance Most software projects will have procurement activities that require interactions with suppliers regardless of software grading level. Procurement activities may be as basic as the purchase of compilers or other development tools for custom developed software or as complicated as procuring a complete safety system software control system. ASME NQA-1-2000 References, Req. 3, 802; Req. 4; Req. 7,; SP 2.7, 203; SP 2.7, 300; SP 4.1, 203; SP 4.1, 300; Most software projects will have procurement activities that require interactions with suppliers regardless of software grading level. Procurement activities may be as basic as the purchase of compilers or other development tools for custom developed software or as complicated as procuring a complete safety system software control system. ASME NQA-1-2000 References, Req. 3, 802; Req. 4; Req. 7,; SP 2.7, 203; SP 2.7, 300; SP 4.1, 203; SP 4.1, 300;

37. 37 Work Activity 4 Procurement & Supplier Mgmt continued Requires a variety of approaches based upon: Level of control DOE or its contractors have on the quality Complexity of the safety software 4 major approaches Assess the supplier Self-declaration by the supplier Accepting safety software �as is� based upon key characteristics Supplier certification or accreditation (e.g., ISO, UL, SEI CMMi) There needs to be a variety of approaches for software procurement and supplier management based upon� the level of control DOE or its contractors have on the quality of the software or software service being procured and the complexity of the software. There are four major approaches for this assessment: performing an assessment of the supplier, requiring the supplier to provide a self-declaration that the safety software meets the intended quality, accepting the safety software based upon key characteristics (e.g., large user base), and verifying the supplier has obtained a certification or accreditation of the software product quality or software quality program from a third party (e.g., the International Organization for Standardization, Underwriters Laboratories, and Software Engineering Institute). It is important to note that while all grading levels in DOE G 414.1-4 are required to fully meet this work activity, the implementation detail and assessment method of the supplier can vary based on the complexity of the software and its importance to safety. There needs to be a variety of approaches for software procurement and supplier management based upon� the level of control DOE or its contractors have on the quality of the software or software service being procured and the complexity of the software. There are four major approaches for this assessment: performing an assessment of the supplier, requiring the supplier to provide a self-declaration that the safety software meets the intended quality, accepting the safety software based upon key characteristics (e.g., large user base), and verifying the supplier has obtained a certification or accreditation of the software product quality or software quality program from a third party (e.g., the International Organization for Standardization, Underwriters Laboratories, and Software Engineering Institute). It is important to note that while all grading levels in DOE G 414.1-4 are required to fully meet this work activity, the implementation detail and assessment method of the supplier can vary based on the complexity of the software and its importance to safety.

38. 38 Work Activity 5 Sw Rqmts Identification & Mgmt System requirements provide the foundation for the sw requirements Requirements should be complete, correct, consistent, clear, verifiable, and feasible Include software requirements for: Safety system requirements provide the foundation for the requirements to be implemented in the software. These system requirements should be translated into requirements specific for the software. safety software requirements may be documented in: system level requirements documents, software requirements specifications, procurement contracts, and/or other acquired software agreements. These requirements should identify functional; performance; security, including user access control; interface and safety requirements; and installation considerations and design constraints where appropriate.Safety system requirements provide the foundation for the requirements to be implemented in the software. These system requirements should be translated into requirements specific for the software. safety software requirements may be documented in: system level requirements documents, software requirements specifications, procurement contracts, and/or other acquired software agreements. These requirements should identify functional; performance; security, including user access control; interface and safety requirements; and installation considerations and design constraints where appropriate.

39. 39 Work Activity 5 Sw Rqmts Id & Mgmt continued Manage requirements to: Minimize conflicts with other safety software or system requirements Maintain accuracy of requirement for later validation Trace requirement throughout the software life cycle The requirements should be complete, correct, consistent, clear, verifiable, and feasible. The requirements should be managed to minimize conflicting requirements and maintain accuracy for later validation activities to ensure the correctness of the software, and should be traceable throughout the software life-cycle. ASME NQA-1-2000 References, Req. 1; Req. 2, 100; Req. 3, 800; Req. 5; Req. 6; SP 2.7, 201; SP 2.7, 400; SP 4.1, 201; SP 4.1, 400; The requirements should be complete, correct, consistent, clear, verifiable, and feasible. The requirements should be managed to minimize conflicting requirements and maintain accuracy for later validation activities to ensure the correctness of the software, and should be traceable throughout the software life-cycle. ASME NQA-1-2000 References, Req. 1; Req. 2, 100; Req. 3, 800; Req. 5; Req. 6; SP 2.7, 201; SP 2.7, 400; SP 4.1, 201; SP 4.1, 400;

40. 40 Work Activity 6 Sw Design & Implementation Key sub-activities Developing (designing & coding) Documenting Verifying (reviews, traceability, developer testing) Controlling (SCM) Applies primarily to custom safety software Design needs to be complete & sufficient to meet software requirements Design needs to be adequate to fully describe: Interfaces with other system components Software structure and process flow Key sub-activities During software design and implementation the software is developed, documented, verified, and controlled. Applies primarily to custom safety software But is applicable in a graded method for any software product that requires customization such as configurable PLC and utility calculations developed by the user. Design needs to be complete & sufficient to meet software requirements The software design should be complete and sufficient to meet the software requirements. It should identify the operating system, function, interfaces, performance requirements, installation considerations, design inputs, and design constraints. Design activities and documentation should be adequate to fully describe how the software will interface with other system components and how the software will function internally. Data structure requirements and layouts may be necessary to fully understand the internal operations of the software. ASME NQA-1-2000 References, Part 3 400; Req. 3, 800; SP 2.7, 400; SP 4.1; Key sub-activities During software design and implementation the software is developed, documented, verified, and controlled. Applies primarily to custom safety software But is applicable in a graded method for any software product that requires customization such as configurable PLC and utility calculations developed by the user. Design needs to be complete & sufficient to meet software requirements The software design should be complete and sufficient to meet the software requirements. It should identify the operating system, function, interfaces, performance requirements, installation considerations, design inputs, and design constraints. Design activities and documentation should be adequate to fully describe how the software will interface with other system components and how the software will function internally. Data structure requirements and layouts may be necessary to fully understand the internal operations of the software. ASME NQA-1-2000 References, Part 3 400; Req. 3, 800; SP 2.7, 400; SP 4.1;

41. 41 Work Activity 7 Software Safety Sw failures WILL affect the overall safety 2 primary activities Performing safety & hazard analysis of software throughout the life cycle Implementing design methods to promote safe operation of system Identify & evaluate potential failures for consequences of failure and the probability of occurrence (software level hazard analysis) Software is only one component of the overall safety system. It may be embedded in an I&C system, it may be a custom control system for hardware components, or it may be standalone software used in safety management or support decisions. Methods to mitigate the consequences of software failures should be an integral part of the software design. Specific software analysis and design methods for ensuring that safety functions are well thought out and addressed properly should be performed throughout the software development and operations life-cycles. Potential failures need to be identified and evaluated for their consequences of failure and probability of occurrence. Some potential problems are: complex or faulty algorithm, lack of proper handling of incorrect data or error conditions, buffer overflow, and incorrect sequence of operations due to either logic or timing faults. Hazard analysis techniques including failure modes and effects analysis, fault-tree modeling, event-tree modeling, cause-consequence diagrams, hazard and operability analysis, and interface analysis should be performed. The design of the software is critical to ensuring safe operation of the system. Software is only one component of the overall safety system. It may be embedded in an I&C system, it may be a custom control system for hardware components, or it may be standalone software used in safety management or support decisions. Methods to mitigate the consequences of software failures should be an integral part of the software design. Specific software analysis and design methods for ensuring that safety functions are well thought out and addressed properly should be performed throughout the software development and operations life-cycles. Potential failures need to be identified and evaluated for their consequences of failure and probability of occurrence. Some potential problems are: complex or faulty algorithm, lack of proper handling of incorrect data or error conditions, buffer overflow, and incorrect sequence of operations due to either logic or timing faults. Hazard analysis techniques including failure modes and effects analysis, fault-tree modeling, event-tree modeling, cause-consequence diagrams, hazard and operability analysis, and interface analysis should be performed. The design of the software is critical to ensuring safe operation of the system.

42. 42 Work Activity 7 Software Safety continued Evaluate software design for effectiveness in mitigating or eliminating the hazards identified Assess impact of partial sw failures that may degrade system performance Implement safety design techniques Simplicity Decoupling Isolation Redundancy Reduction in complexity (software and data inputs) Fault detection Self diagnostics The software design should consider principles of simplicity, decoupling, and isolation to eliminate the hazards. Complexity of the software design, including the logic and number of data inputs, has proven to increase the defect density in software components. The safety features should be separate from non-safety modules, minimizing the impact of failure of one module on another. Separation of the safety features also allows for more rigorous software development and verification practices to be applied to the safety components while providing the appropriate and cost effective level of SQA applied to the non-safety components. Other design techniques, such as building fault detection and self-diagnostics into the software, should be implemented. Using external monitors (safety bag) for the software safety functions, n-version programming, and Petri nets are examples of techniques that can ensure the software design adequately addresses safety issues and minimizes failure modes by adding fault tolerant concepts. Self-diagnostics detect and report software faults and failures in a timely manner and allow actions to be taken to avoid an impact on the system operating safety. Some of these techniques include memory functionality and integrity tests, such as checksums and watch dog timers for software processes, including operating system processes. ASME NQA-1-2000 References, Req. 2; Req. 3, 800; SP 2.7, 402; SP 4.1, 100; The software design should consider principles of simplicity, decoupling, and isolation to eliminate the hazards. Complexity of the software design, including the logic and number of data inputs, has proven to increase the defect density in software components. The safety features should be separate from non-safety modules, minimizing the impact of failure of one module on another. Separation of the safety features also allows for more rigorous software development and verification practices to be applied to the safety components while providing the appropriate and cost effective level of SQA applied to the non-safety components. Other design techniques, such as building fault detection and self-diagnostics into the software, should be implemented. Using external monitors (safety bag) for the software safety functions, n-version programming, and Petri nets are examples of techniques that can ensure the software design adequately addresses safety issues and minimizes failure modes by adding fault tolerant concepts. Self-diagnostics detect and report software faults and failures in a timely manner and allow actions to be taken to avoid an impact on the system operating safety. Some of these techniques include memory functionality and integrity tests, such as checksums and watch dog timers for software processes, including operating system processes. ASME NQA-1-2000 References, Req. 2; Req. 3, 800; SP 2.7, 402; SP 4.1, 100;

43. 43 Work Activity 8 V&V Largest work activity Verification is performed throughout the software life cycle Verifies that the �requirements or conditions� of the previous phase were fulfilled �Did we build the widget right?� Validation is performed at the end of software development or acquisition process Validates that the software meets the intended software requirements �Did we build the right widget?� V&V is the largest area within the SQA work activities. Verification is performed throughout the life-cycle of the safety software. Validation activities are performed at the end of the software development or acquisition processes to ensure the software meets the intended requirements. V&V is the largest area within the SQA work activities. Verification is performed throughout the life-cycle of the safety software. Validation activities are performed at the end of the software development or acquisition processes to ensure the software meets the intended requirements.

44. 44 Work Activity 8 V&V continued Activities include: Reviews (e.g., software, documentation, procedures, users manuals) Inspections (e.g., Fagan, walkthroughs, desk checks) Assessments & Audits (e.g., product, process, supplier) Observations Testing (e.g., developer, acceptance, installation, in-use) Continually monitor system to estimate software reliability and safety Evaluate defects discovered for trends Perform periodic testing (in-use) if possible Monitor operational parameters (e.g., timing, file and/or data size) V&V activities include reviews, inspections, assessments, observations, and testing. This Guide expands on ASME NQA-1-2000 acceptance testing activities to include more extensive V&V activities of reviews, inspections, assessments, and observations as described in other consensus standards. Reviews and inspections of software deliverables requirement specifications, procurement documents, software design, code modules, test results, training materials, user documentation, and processes that guide the software development activities should be performed. Observations and testing can be performed during the development, factory or site acceptance testing, installation, and operation (i.e., in-use testing) of the software. Acceptance testing should include functional testing, performance testing, security testing, stress testing, and load testing. Users� guides, use cases, and operational profiles are instrumental in identifying and detailing the positive test cases and procedures. Failure mode analyses can be used for defining negative test cases and procedures. Testing strategies that may be appropriate for acceptance testing include equivalence class testing, branch and path testing, statistical-based and boundary value testing. Additionally, the system should continually be monitored to estimate its continuing reliability and safety. Periodic testing of the operational system should be performed to detect any degradation If testing is not possible, monitoring using quantitative measurements should be performed. ASME NQA-1-2000 References, Part 3 1A-1, 303; 402.1, 404; Req. 3, 801.4, 801.5; Req. 11, 400; SP 2.7, 402.1, 404; SP 4.1; SP 4.1, 101.3; ASME SP 4.1, 101.1 of particular interest for utility calculationsV&V activities include reviews, inspections, assessments, observations, and testing. This Guide expands on ASME NQA-1-2000 acceptance testing activities to include more extensive V&V activities of reviews, inspections, assessments, and observations as described in other consensus standards. Reviews and inspections of software deliverables requirement specifications, procurement documents, software design, code modules, test results, training materials, user documentation, and processes that guide the software development activities should be performed. Observations and testing can be performed during the development, factory or site acceptance testing, installation, and operation (i.e., in-use testing) of the software. Acceptance testing should include functional testing, performance testing, security testing, stress testing, and load testing. Users� guides, use cases, and operational profiles are instrumental in identifying and detailing the positive test cases and procedures. Failure mode analyses can be used for defining negative test cases and procedures. Testing strategies that may be appropriate for acceptance testing include equivalence class testing, branch and path testing, statistical-based and boundary value testing. Additionally, the system should continually be monitored to estimate its continuing reliability and safety. Periodic testing of the operational system should be performed to detect any degradation If testing is not possible, monitoring using quantitative measurements should be performed. ASME NQA-1-2000 References, Part 3 1A-1, 303; 402.1, 404; Req. 3, 801.4, 801.5; Req. 11, 400; SP 2.7, 402.1, 404; SP 4.1; SP 4.1, 101.3; ASME SP 4.1, 101.1 of particular interest for utility calculations

45. 45 Work Activity 9 Prblm Rpting & Corrective Action Addresses documenting, evaluating and correcting defects Defines roles and responsibilities For custom built sw, should be part of the overall software change control & management system as described in the SCM work activity Integrated with system level problem reporting and corrective action system Coupled with the configuration management of the software system, the problem reporting and corrective action process should address the appropriate requirements of the corrective action system. The reporting and corrective action system will cover (1)�methods for documenting, evaluating and correcting software problems; (2) an evaluation process for determining whether a reported problem is indeed a defect or an error; and (3) the roles and responsibilities for disposition of the problem reports, including notification to the originator of the results of the evaluation Procurement documents should identify the requirements for suppliers to report problems to the supplier, any required supplier response, and the method for the purchasers to report problems to the supplier. ASME NQA-1-2000 References, Req. 15; Req. 16; SP 2.7, 204; SP 4.1, 204; Coupled with the configuration management of the software system, the problem reporting and corrective action process should address the appropriate requirements of the corrective action system. The reporting and corrective action system will cover (1)�methods for documenting, evaluating and correcting software problems; (2) an evaluation process for determining whether a reported problem is indeed a defect or an error; and (3) the roles and responsibilities for disposition of the problem reports, including notification to the originator of the results of the evaluation Procurement documents should identify the requirements for suppliers to report problems to the supplier, any required supplier response, and the method for the purchasers to report problems to the supplier. ASME NQA-1-2000 References, Req. 15; Req. 16; SP 2.7, 204; SP 4.1, 204;

46. 46 Work Activity 10 Training More than training the user/operator Training of personnel in Safety software analysis Safety software design techniques Human factors/User interface design Testing techniques using emulators Commensurate with scope, complexity, and importance of the task Consider education, experience, and proficiency of the individual Training personnel in designing, developing, testing, evaluating, or using the safety software application is critical for minimizing the consequences of software failure. Training may be necessary for the analyst, development and test teams, application users, and operations staff. The analyst and developers may need training in fault tolerant methodologies, safety design methodologies, user interface design issues, testing methodologies, or configuration management to ensure delivery of a robust software application. Meanwhile, the software application users and operations staff may need training specific to the software to ensure that proper data are entered, that proper options and menus are selected, and that the results of the software can be interpreted correctly. Training should be commensurate with the scope, complexity, and importance of the tasks and the education, experience, and proficiency of the individual. Personnel should also participate in continuing education and training as necessary to improve their performance and proficiency and ensure that they stay up-to-date on changing technology and new requirements.Training personnel in designing, developing, testing, evaluating, or using the safety software application is critical for minimizing the consequences of software failure. Training may be necessary for the analyst, development and test teams, application users, and operations staff. The analyst and developers may need training in fault tolerant methodologies, safety design methodologies, user interface design issues, testing methodologies, or configuration management to ensure delivery of a robust software application. Meanwhile, the software application users and operations staff may need training specific to the software to ensure that proper data are entered, that proper options and menus are selected, and that the results of the software can be interpreted correctly. Training should be commensurate with the scope, complexity, and importance of the tasks and the education, experience, and proficiency of the individual. Personnel should also participate in continuing education and training as necessary to improve their performance and proficiency and ensure that they stay up-to-date on changing technology and new requirements.

47. 47 In Summary

48. 48 Important Things to Remember Scope of 10 CFR 830 and DOE O 414.1C includes software related to nuclear (including radiological) facilities Involve the facility design authority Maintain a safety software inventory In an approved DOE document, define grading levels & consensus standards Select and implement the SQA work activities in accordance with ASME NQA-1-2000 or equivalent level of SQA If not using ASME NQA-1-2000, a crosswalk or other documentation must be able to show equivalency

49. 49 Important Things to Remember - continued For Federal Staff Staff responsible for SQA activities must be qualified according to DOE Technical Qualification Program and DOE-STD-1172 Staff must be knowledgeable of alternative standards and ASME NQA-1-2000 to approve the use as an alternate standard(s)

50. 50 Resources EH QA Web Site http://www.eh.doe.gov/qa EH SQA Knowledge Portal http://www.eh.doe.gov/sqa EH SQA Discussion Forum DOE-STD-1172-2003, Safety Software Quality Assurance Functional Area Qualification Standard http://www.eh.doe.gov/techstds/standard/std1172/std11722003.pdf

51. 51 Resources continued QA - Bud Danielson 301-903-2954 bud.danielson@eh.doe.gov SQA - Debra Sparkman 301-903-6888 debra.sparkman@eh.doe.gov

52. 52