Product-Line Verification of Safety-Critical Software

1. National Aeronautics and Space Administration Elliott, JPL: MSAP NASA, GRAIL Product-Line Verification of Safety-Critical Software NASA OSMA Software Assurance Symposium September 8-12, 2008 Robyn Lutz, JPL & ISU This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, and at NASA Ames Research Center, under a contract with the National Aeronautics and Space Administration. The work was sponsored by the NASA Office of Safety and Mission Assurance under the Software Assurance Research Program led by the NASA Software IV&V Facility. This activity is managed locally at JPL through the Assurance and Technology Program Office SAS08_Product_Line_Verif_Safety_Lutz

2. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical SoftwareProblem • Industrial experience shows that it is the verifiable conformance of each system to the product-line specifications that makes or breaks the product-line practice • Verification that the software for each project satisfies its intended product-line constraints is thus essential • How should we verify that delivered software conforms to the product-line and how do we document that conformance? • How should we verify that safety-critical software built using product-line assets is safe? SAS08_Product_Line_Verif_Safety_Lutz

3. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical Software Approach • Identify candidate suite of product-line verification techniques, focusing on safety • Tailor as appropriate to NASA needs • Apply, evaluate, and report results in collaboration with project • Produce examples and data for training and transfer of techniques to projects SAS08_Product_Line_Verif_Safety_Lutz

4. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical Software Relevance to NASA • NASA is reusing more product-line, inherited, & reused software assets • We need better ways to verify compliance of delivered software against the constraints levied on all systems in that product line • We need better ways to verify that problems with similar, earlier systems won’t recur in our new system • Work focuses on identifying product-line verification techniques that can help solve problems now for current missions SAS08_Product_Line_Verif_Safety_Lutz

5. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical Software Accomplishments FY’07 • Surveyed effective product-line verification techniques that exist in industry • Report 1: “Survey of Product-Line Verification and Validation Techniques” • Best practices • Verification challenges & verification enablers identified for NASA • Resources assembled (conferences, industrial and defense industry experiences, annotated bibliography) • Report 2: “Tool-Support Survey for Product-Line Verification and Validation Techniques” • Commercial and academic tools for product-line development • Configuration management and change management tools for product lines • Tool-supported testing in product lines (both domain testing and application testing) • Advice from product-line experts regarding tool support SAS08_Product_Line_Verif_Safety_Lutz

6. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical SoftwareAccomplishments FY’08 • Recommended effective product-line verification techniques at NASA based on industrial practices • Paper, “Enabling Verifiable Conformance for Product Lines”, accepted to 12th Int’l Software Product Line Conference • Delivered report evaluating the top three software product-line support tools (identified in FY07 report) for project applications (authored by Simon Chong) • Downloaded and experimented with each • FORM (from Pohang University of Science & Technology), Pure::Variants (from PureSystems, GmbH), and Gears (from BigLever) • Emphasis on practical use: e.g., setup, support, user interface, support for commercial tools, functionality • Recommendation for application plan based on measured criteria: Gears SAS08_Product_Line_Verif_Safety_Lutz

7. Product-Line Verification of Safety-Critical SoftwareCurrent capability of the research National Aeronautics and Space Administration • MSAP Project: Worked this year with Multi-Mission System Architecture Platform project at JPL • to identify (1) “negative lessons learned” from industrial use of product lines (risks) and (2) relevant mitigation strategies • findings for MSAP managers to use in product-line business case • MSAP collaboration slowed when key personnel moved • GRAIL Project: Now working with Gravity Recovery and Interior Lab (Discovery Lunar mission, 2011; twin spacecraft; inheritance from GRACE’s earth-study techniques and from Lockheed Martin’s product-line spacecraft, especially MRO) • to perform trend-tracking for GRAIL of software changes and anomalies across product line (beginning with MRO): on-going in FY09 • to evaluate use of DDP to customize product-line risks/mitigations to GRAIL: on-going in FY09 SAS08_Product_Line_Verif_Safety_Lutz

8. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical SoftwareTechnical challenges of approach Challenges and techniques for verifiable conformance of a product line SAS08_Product_Line_Verif_Safety_Lutz

9. National Aeronautics and Space Administration Product-Line Verification of Safety-Critical Software Planned capability of research Customize industrial, product-line techniques to verify correctness of a new system’s software inheritance, and apply to NASA systems: • GRAIL (FY09 focus) • JUNO (same heritage) • Future NASA spacecraft with same LMA product-line heritage • Exploration systems reusing same baseline and inherited software SAS08_Product_Line_Verif_Safety_Lutz