160 likes | 182 Views
NASA OSMA SAS ‘03. Software Requirements Analysis As Fault Predictor Dolores R. Wallace SRS Technologies Software Assurance Technology Center http://satc.gsfc.nasa.gov dwallac@pop300.nasa.gov William M. Wilson SRS Technologies/SATC wilsonw@pop300.nasa.gov.
E N D
NASA OSMA SAS ‘03 Software Requirements Analysis As Fault Predictor Dolores R. Wallace SRS Technologies Software Assurance Technology Center http://satc.gsfc.nasa.gov dwallac@pop300.nasa.gov William M. Wilson SRS Technologies/SATC wilsonw@pop300.nasa.gov SAS03/FaultPrediction
Software Requirements Analysis As Fault Predictor Presentation Outline • Research Rationale & Objective • Requested Project Data • Planned Approach • Data Acquisition & Analysis Obstacles • Results & Findings • Conclusions • Recommendations SAS03/FaultPrediction
Research Rationale & Objective Rationale • The earlier in the lifecycle that software faults can be found the less expensive it is to correct them. • The earliest opportunity to preclude software faults is during the development of the system's requirements. Objective • To determine if software requirements analysis results can be used as a predictor of software faults. SAS03/FaultPrediction
Requested Project Data • Requirements Specification • Accessible in electronic media in MS Word format for use with the ARM tool • Conforming to NASA-STD-2100 • Specification statements identified by hierarchical numbers SAS03/FaultPrediction
Requested Project Data – Cont. • Verification Test Matrix • Requirements to Test or vice versa - OR - • Traceability Matrix • Requirements to Design Elements • Design Element To Software Component - AND - • Test Plans • Tests vs. Software Components. SAS03/FaultPrediction
Requested Project Data – Cont. • Failure Reports/Data - Preferably in DDTS format • Including: test id, module id, CSCI id, release/build id, suspected cause of failure and resolution (e.g., actual cause of failure - what was fixed) • Configuration Control Reports • Baseline item changed -e.g., Requirement, Design, Code • Reason/Source of change – e.g., RID, Failure Report, PR, etc. SAS03/FaultPrediction
Planned Approach • Use ARM tool to analyze projects’ software requirements. • Collected projects’ failure data in a DDTS database. • Organize the data to represent same components for each project. • Conduct correlation analysis. • Study results of step 4 to prove or disprove hypothesis. SAS03/FaultPrediction
ARM TOOL Reports Identifies and Counts: • Size of Requirements Document • Lines of Text • Numbered Paragraphs • Number of Specification Statements • Number of Unique Specification Subjects • Number of Specifications Containing • Weak, Optional and/or Incomplete Phrases. • Compound and/or Complex Statements. • Requirements Document Structure • Depth and Distribution of Specifications SAS03/FaultPrediction
Data Acquisition & Analysis Obstacles • Common Problems • No commonality or standardization of data or formats across project. • Current projects are reluctant to provide data to outside organizations. • Data from completed projects is incomplete, not in a usable form or is not accessible. SAS03/FaultPrediction
Data Acquisition & Analysis Obstacles Cont. • Specific Problems • No project staff available to guide through the mass of documents in the collection • Documents locked on a WEB site • Documents in “.pdf” format • Variation of format and media within project • Complete set of data for a specific Build/Release not available • Data held by support contractor and released only on a “Project Need-To-Know” basis SAS03/FaultPrediction
Results & Findings • Data provided by four projects • Projects A and B • Major subsystems of operational information processing system • Projects C and D • Flight instrumentation packages SAS03/FaultPrediction
Results & Findings Cont SAS03/FaultPrediction
Conclusions • Data from many more projects is required to improve the possibility of finding a number of data sets with sufficient commonality to support analysis. • Analysis approach needs to be expanded to include determining if failures attributed to documentation and design flaws are in reality related to deficiencies in requirements. SAS03/FaultPrediction
Recommendations Advocate and support: • The use of NASA-STD-2100 documentation standard • The use of a format to report problems and failures that is compatible with DDTS • The creation and use of centralized Center repositories for data from completed projects. SAS03/FaultPrediction
Summary • Lessons • Getting data requires strong, interested Civil servant advocate • Analyzing data requires tedious effort, even with automated tools • Future • Analyze latest set of data • Attempt correlations for the 4 data sets SAS03/FaultPrediction