Technology Infusion of CodeSonar into the Space Network Ground Segment (RII07)

1. Technology Infusion of CodeSonar into the Space Network Ground Segment (RII07) Software Assurance Symposium Executive Summary

2. Relevance Problem Approach Capabilities Challenges Measures Relevance to NASA (1) The NASA Space Network is a fleet of nine geosynchronous satellites that provide continuously available “bent pipe” communications to customers at and below low earth orbit as well as three ground stations that control the satellites and act as a communications conduit to mission operations centers Image credit: Canadian Space Agency educational resource web site; http://www.space.gc.ca/asc/eng/educators/resources/orbital/geosynchronous.asp

3. Relevance Problem Approach Capabilities Challenges Measures Relevance to NASA (2) Human Spaceflight, Space Science Earth Science and other government agencies (National Science Foundation, for example) rely on the Space Network for many forms of communication including launch support, tracking, telemetry, and command, voice communication, and delivery of science data

4. Relevance Problem Approach Capabilities Challenges Measures Problem Statement • The Space Network requires availability of all communications equipment at minimum 97.00% of all time and requires 99.90% proficiency in delivery of scheduled communications events. The standard of excellence in availability and proficiency are more stringent (98.00% and 99.97%, respectively). • The Space Network equipment is controlled by over eight million lines of software, which cannot feasibly be exhaustively tested*. Software Engineering, A Practitioner’s Approach, 4th edition, Chapter 16: Software Testing Techniques provides an explanation and example of this phenomenon, which is applicable to software in general.

5. Relevance Problem Approach Capabilities Challenges Measures Approach • Space Network software maintenance performs formal demonstration, analysis, test, and manual product inspections • The approach of this technology infusion initiative is to reduce loss of availability and proficiency caused by software via insertion of an automated software analysis tool into the software maintenance lifecycle

6. Relevance Problem Approach Capabilities Challenges Measures Current Project Capability • Software has historically accounted for an annual average of 28% of the Space Network loss of availability and proficiency (low of 11% and high of 57% annually) Percentages reflect the portion of losses attributed to software with respect to the total loss within the control of the Space Network. For example, if the annual average loss attributable to the Space Network was 500 out of 130k total hours (99.996% proficiency), then the portion caused by software would be 500 x 0.28 = 140 hours.

7. Relevance Problem Approach Capabilities Challenges Measures Planned Project Capability • Technology infusion of static code analysis is expected to improve the existing baseline (reduce losses due to latent defects) and prevent certain classes of defects from entering the operational system (new losses not otherwise eliminated). • Automated static code analysis is expected to find defects sooner to prevent rework, which permits better use of resources. Image credit: CrossTalk: TheJournal on Defense Software Engineering; Determining Return on Investment Using Software Inspections; Don O’Neill; http://www.stsc.hill.af.mil/crosstalk/2003/03/ONeill.html

8. Relevance Problem Approach Capabilities Challenges Measures Future Project Capability • GrammaTech’s CodeSonar was selected as the source code analysis tool on two Space Network C++ software systems because of the past success of GrammaTech’s CodeSurfer product in application to other NASA software and the beta capabilty of CodeSonar to analyze the Ada language, which constitutes over 25% of the Space Network software lines of code • Success in this project will lead to automated code analysis becoming an integral part of the Space Network software maintenance processes Image credit: GrammaTech; http://www.grammatech.com

9. Relevance Problem Approach Capabilities Challenges Measures Technical Challenges (1) • The Space Network uses a variety of compilers and operating systems, some up to twenty years old. Effort required to apply the source code analysis tool across the many platforms and languages is a challenge. • Mitigation for platforms not supported by the tool is to perform static analysis on a modern, sterile* configuration management system. Image credit: GrammaTech; http://grammatech.com/products/codesonar/tour.html CodeSonar is advertised as working best by “watching” compilation but can also be used on a system other than the development system. A “sterile” system hosts the source code but does not have all tools needed to build the code.

10. Relevance Problem Approach Capabilities Challenges Measures Technical Challenges (2) • Source code analysis tools have historically produced a large number of false positives. Differentiating false positives from real defects can require too much time from developer staff to be cost effective. • Formal training and support from the vendor in the tools capabilities to narrow false positives and availability of staff familiar with automated source code analysis tools will streamline the filtering process to overcome this challenge.

11. Relevance Problem Approach Capabilities Challenges Measures Acceptance Measures (1) Measures are meant to aid in determining the impact of the technology on the project both in the expenditure of effort and the technical results of the technology Average time to correct defect Average data loss per defect

12. Relevance Problem Approach Capabilities Challenges Measures Acceptance Measures (2) The decision to make permanent the use of static source code analysis as part of the maintenance lifecycle is based on weighing factors influenced by use of the tool Time to correct discrepancy Time to review discrepancy report Data loss caused by defect Defect count in operations Time to operate tool Time to inspect code