240 likes | 466 Views
JLab Software Assurance Program. A Risk Based Approach to Software Management. Outline. Software Assurance vs. Software Quality Assurance QA Order Requirements Processes that address software assurance JLab’s SW Assurance Effort Risk Based Model Assessment Method Preliminary Results
E N D
JLab Software Assurance Program A Risk Based Approach to Software Management
Outline • Software Assurance vs. Software Quality Assurance • QA Order Requirements • Processes that address software assurance • JLab’s SW Assurance Effort • Risk Based Model • Assessment Method • Preliminary Results • Path Forward
DOE 414.1C QA Order 414.1C QA PLAN GENERAL REQUIREMENTS SUSPECT/COUNTERFEIT ITEM PROCESS CORRECTIVE ACTION MANAGEMENT (SAFETY) SOFTWARE QUALITY SOFTWARE ASSURANCE PROCEDURE • Applies to ALL Software activities • Ten Criteria for Safety SW QA Program • Requirements flow-down through CRD
JLab Software Assurance Procedure Implementation of Requirements of Quality Assurance Plan Implements a process for identifying and classifying the impact SW may have on multiple subject areas, including safety Adaptable to all software activities important to facility mission and goals Implements consistent tiered approach
Software Quality Assurance Software Assurance NASA-STD-8739.8 (w/Change 1) July 28, 2004 “Software assurance consists of the following disciplines: • Software Quality • Software Quality Assurance • Software Quality Control • Software Quality Engineering • Software Safety • Software Reliability • Software Verification and Validation (V&V) • Independent Verification and Validation (IV&V)” NASA-GB-8719.13 NASA Software Safety Guidebook. • Implementation guidance for high consequence software
JLab Process • SW Assurance team chartered by CIO • Representatives from across site: • Scientific • Experimental • Theoretical • Computing • Business • Controls • Cyber Security • HR • Safety Systems
JLab SWAP - Table of Contents 1 Purpose 1.1 Structured Approach 2 Scope 2.1 Exemptions 3 Responsibilities 4 Software Control Procedure 4.1 Software Risk Assessment Process Steps & Expectations 4.1.1 Software critical to JLab safety, operations, and mission 4.1.2 Software important to JLab safety, operations, and mission 4.1.3 Software Risk Assessment Assumptions: 4.1.4 Software Risk Assessment Tool
Table of Contents - Continued 4.2 Software Assurance 4.2.1 Graded Approach 4.2.2 Software Assurance Program Requirements 4.2.2.1 Acquirer Software Assurance 4.2.2.2 Basic Requirements for Software Assurance Processes: 4.2.2.2.1 Software Lifecycle 4.2.2.2.2 Software Quality 4.2.2.2.3 Competence 4.2.2.2.4 Sustainability 4.2.2.2.5 Configuration Management 4.2.2.2.6 Assessment 4.3 Metrics and Continuous Improvement 5.0 DEFINITIONS 6.0 REFERENCES 7.0 REVISION SUMMARY
JLab SW Assurance Procedure • Clarifies scope: • Applies to all projects, programs, facilities, and activities that may impact JLab mission and goals • Applies to all software developed, or modified for use at JLab. • Compliments Cyber Security Program • Reflects SW Enclaves • Applies to security software configuration items only where ineffective security software controls may directly affect operations and safety • Cyber Security Risk Assessment incorporated in to overall risk assessment
JLab SW Assurance • Defines SW Risk Assessment Procedure: • Identifies pre- and post-mitigation Risk • Applicable to ALL SW within scope of process • Defines Requirements for Owner Organization SW • Requirements for lifecycle model • Requirements for
Structured Approach • Identify important JLab software configuration items and activities • Identify roles and responsibilities for software control activities within the context of this procedure • Perform a software risk assessment on applicable configuration items • Apply a risk-based graded approach to software assurance activities • Apply a value added continuous improvement process to the software assurance processes
Exemptions “This procedure does not apply to unmodified general purpose computing software, unmodified enterprise software, and general purpose desk-top software managed under the IT/CIO Division. Examples include office productivity software, public web pages, and LAN/WAN networking software.”
Roles and Responsibilities • Defines roles, responsibilities, and authority for • COO • CIO • ESH&Q AD • Division Management • Line Management • Software Owner • Oversight committees
Scope • internal software development • software used to collect and manage data • startup and configuration scripts • incorporation of open source software • modified off the shelf (MOTS) software used to design, analyze, or control safety or mission essential aspects of JLab operations • commercial off the shelf (COTS) software used to design, analyze, or control safety or mission essential aspects of JLab operations • programs and firmware for monitoring or control, including IOCs and PLCs • modifiable embedded software and firmware including PICs and PC104 type SBCs • programs and development software for field programmable integrated circuits such as Field Programmable Gate Arrays • Other software as defined by the JLab Chief Information Officer
SW Risk Assessment Software is scored (1-5) in each of six areas of impact Direct Risk of Financial Loss Direct Risk of Loss of Tangible Equipment/Property Direct Risk of Harm to People Direct Risk of Harm to the Environment Direct Risk of Loss of Continuity of Operations/Organization/Mission Direct Risk of Enforcement Action Scoring is reviewed at both the individual and cumulative level
Types of Software Assessed • Lattice QCD Modeling • Experiment Data Management • MIS Accounting • MIS Procurement • Corrective Action Tracking System • Facilities Work Order System • Travel • -Timesheets • Facilities Work Order System • Accelerator Physics Model • Machine Protection • Personnel Safety System • PLC Firmware • PLC Program • Radiation Instrumentation • Beam Position Monitor
SW Assurance as a Mitigation Each Owning Organization responsible for tailoring requirements in to their own SA program Procedure provides consensus standard references for generally accepted good practice Procedure provides references for incorporation in to organization’s individual process
Metrics Recommended for Acceptable and Tolerable Risk Required for Intolerable and Unacceptable Risk Assessments go back to central repository for analysis Allows comparison of mitigations vs. claimed effectiveness Refers to existing SW metric processes Guidance refers to CMMI process
Pilot project complete In process of implementing procedure lab wide Feedback (mostly) positive Expanding Risk Assessment data Status
Conclusions JLab is implementing a risk based software assurance process Consensus based procedure with buy-in from all SW enclaves Tools, e.g. risk assessment spreadsheet, are integrated in to the process Provides minimum requirements for SW lifecycle Incorporates resources and guidance for application Process incorporates metrics