Software Research and Technology Infusion June 2007 Presented by

1. Software Research and Technology Infusion June 2007Presented by Pavan RajagopalPavan.Rajagopal@ivv.nasa.gov Wes Deadrickwesley.w.deadrick@nasa.gov

2. 2007 Infusion Team • Representation from GSFC, MSFC, JSC, JPL, and the NASA IV&V Facility • Allen Nikora (JPL) - Allen.P.Nikora@jpl.nasa.gov • Sally Godfrey (GSFC) - Sara.H.Godfrey@nasa.gov • Ben Kobler (GSFC) - Ben.Kobler@nasa.gov • Ken Chen (JSC) - Ken.K.Chen@nasa.gov • Andy Young (MSFC) - Andy.Young@nasa.gov • Wesley Deadrick (IVVF) – Wesley.W.Deadrick@nasa.gov • Pavan Rajagopal (IVVF) – Pavan.Rajagopal@ivv.nasa.gov

3. Outline • Background • Goal & Approach • Collaboration concept • Funding for Collaboration • Selected Technologies • Collaboration Roles • Selected software engineering and assurance technologies • SW Developers Assistant • SPACE • CodeSurfer/CodeSonar • Klockwork Inspect • SpecTRM • Perspective Based Inspections • Requirements Assistant • Reactis • SAVE Toolset • Software Cost Reduction

4. Infusing Software Research and Technologies • Goal: Transfer mature Software Engineering and Software Assurance technologies into practice • Provide a reduced risk approach to evaluating: • Technologies derived from NASA-sponsored research • Other new and innovative SE/SA tools and technologies • Approach • Present selected SW technologiesto theNASA software development/assurance community • Encourage and support collaborationsbetween the technology providers andNASA software developersand assurance personnel.

5. Collaborations • Initiated by a individual involved with SW development/assurance who wants to bring on board a candidate technology • Purpose • benefit the software development project • validate the technology • generate empirical data to assess adoption • Not: further develop the research • Funding available for— • training and consulting in the use of the technology • license fees in the case of commercial technologies • applying the technology • collecting & analyzing data • reporting results

6. Funding for Collaborations • Funding for 5 - 7 collaborations available via the Software Assurance Research Program (SARP). • History: 15+ projects in the range $15K - $45K • Competition for SARP funds is among the NASA Centers and JPL. Proposals must come from a civil servant or a contractorwho has a contractual vehicle in place with NASA. • Scope and POP of contract must be able to support the collaboration • NO NEW CONTRACTS WILL BE AWARDED • Proposal template and instructions on the Research Infusion website • www.nasa.gov/centers/ivv/research/research_infusion_index.html • Proposals Due: Monday, July 23 5:00 PM EDT • Collaborations Start: Oct - Dec 2007

7. Funding for Collaborations (cont.) • Mechanization • The Principal Investigator (PI) represents the organization which plans to apply the new technology. PI can be a civil servant orcontractor. • Proposals must identify a NASA CS Point of Contact (POC) responsible for managing the collaboration • If PI is a contractor, often the POC is the COTR or technical manager on the PI’s contract • POC is responsible for coordinating the mechanization of the funding • Either the PI or the POC can pay the technology provider • In-kind funding is welcome!

8. Selected Technologies • Identified from • NASA-sponsored software engineering and assurance research • Leading edge commercial tools • Research in DoD, institutes and industry suggested by NASA projects. • Reviewed by researchers experienced in tech transfer of software engineering research • Send us suggestions for next time. • SE & SA development problem areas • SE & SA technologies • Send suggestions to researchinfusion@ivv.nasa.gov

9. Selected Technologies (continued) • Technology Selection Criteria • Focus on Software Development or Software Assurance • Address a known need/requirement • Robust and mature with good user documentation • Demonstrated successes outside of a single domain or application • Not currently in widespread use within the agency • Promise of support from technology providers

10. Collaboration Roles • Principal Investigator • During proposal preparation: • Work with technology provider to plan collaboration and select suitable application • Must have buy in from the technology provider • Write and submit the proposal • If proposal is selected: • Coordinate training course with developer • Identify software artifacts to which the technology will be applied • Apply the technology (may require multiple iterations) • Collect data & evaluate performance • Write final report

11. Collaboration Roles (continued) • Technology provider : • During proposal preparation • Help plan collaboration, including assisting in the selection of a suitable application • If Principal Investigator’s proposal is accepted • Provide any necessary training course (preferably on-site) • Provide tutorial and other user documentation • Provide customer support throughout collaboration

12. Selected Technologies Pavan RajagopalPavan.Rajagopal@ivv.nasa.gov

13. Perspective Based Inspections (PBI)Dr. Forrest Shull, Fraunhofer Center - Marylandfshull@fc-md.umd.edu, 301-403-8970 • What is it? • An inspection technique that improves the cost-effectiveness of inspections (and teams' participation) by enabling reviewers to focus on topics most relevant to their expertise. • Features • Each participant looks at the work product from a focused perspective, more related to their expertise or job needs. • Can be applied to requirements, design, code or other types of documents. (E.g. test plans) • Benefits • Identifies defects (missed, incomplete, or wrong information) early • Has been shown to be more effective than other inspection approaches

14. Perspective Based Inspections (PBI) • Successes • At GSFC: • Dr. Shull worked with development team members to tailor the perspective-based approach for their requirements inspections. • The approach was both piloted and reviewed by key personnel. The results of both were very positive. • The final technique was written up for inclusion in branch-level standards. • At JSC: • Dr. Shull worked with a team at United Space Alliance to tailor the approach for their specific requirements and code, and provided training in the technique. • Defects were found during practice inspections, in reused software that was thought to be defect free. • PBI also found a major defect which had escaped previous inspections. • Less inspection time was required per inspector than usual. • Contexts for best use • The only challenges found so far for the technique have been with very small development teams (i.e. 3 to 4 people on the entire project team). However, a tailoring kit is being developed to make insertion easier in these contexts.

15. Requirements Assistant (RA)Herman Driessen, Sunny Hills Consultancyhbdriessen@cs.comValerie Jones, IV&V ContactValerie.S.Jones@ivv.nasa.gov • What is it? • Toolset designed to help ensure that requirements are complete, consistent, feasible, and unambiguous, using text in natural language as input. • Features • Runs on desktop • Employs a lessons learned knowledge base gathered from analysis of numerous requirements reviews. • Uses natural language parsing, checklists, templates • Can be customized to specific domains. • Benefits • Reduces much of the tedium involved in document reviews. • Supports continuous improvement by enhancing lessons learned data base.

16. Requirements Assistant (RA) • Successes • NASA IV&V Analysts performed an evaluation of several Requirements Analysis tools and RA was judged the best. • Contexts for best use • during requirements and design reviews • during requirements generation • as an input check before requirements are put into a traceability matrix • tool can also be used on use cases, test specifications, and contracts.

17. ReactisPoC: Steve Sims, CTO, Reactive Systemssims@reactive-systems.com 703-534-6458 • What is it • Tools to automate white-box test case generation and validation of Simulink and Stateflow models. • Features • Runs on Desktop. • User can specify structural-coverage criteria (such as branch and MC/DC, as required by FAA DO-178B) in selecting test data. • Generates test data automatically from models to achieve specified coverage. • Check whether model violates requirements. • Advanced debugging: breakpoints, reverse execution, track model coverage metrics. • Benefits • Construct better models more quickly through requirements checking and debugging, and better code through coverage testing.

18. Reactis • Successes • Reported 25 – 75% reduction in testing costs in automotive industry. • Contexts Best Used • Any project using Simulink/Stateflow in a model-based design process to develop software • Use Reactis to debug models • Use Reactis to compare code against model • Scales well to large models • Automatic test-generation even for very big models • Manual hand-tuning possible to enhance automatically generated tests

19. Software Architecture Visualization and Evaluation (SAVE)Dr. Mikael Lindvall – Fraunhofer Center Marylandmikli@fc-md.umd.edu, 301-403-8972 • What is it? • Toolset and process that allows a user to capture and compare a planned software (SW) architecture with the actual SW architecture evidenced in source code (e.g. C/C++, Java, ADA, Fortan) • Features • Analyzes source code to enable visualization and high-level understanding of the architecture • Identifies architecture changes and potential improvements • Supports bringing architecture into alignment with plan • Benefits • Used to identify architecture violations • Used to simplify complex architectures

20. Software Architecture Visualization and Evaluation (SAVE) • Successes • Successful application of tool set and process with: • Common Ground SW at JHU/APL • Space Network Access System at NASA • Digital Camera at Ricoh • Engine Control System at Hitachi • Contexts for best use • Systems that are developed, maintained by different people • Systems that are difficult to understand, costly to maintain • Systems that need to be restructured/reengineered

21. Software Cost Reduction (SCR)Connie Heitmeyer, Naval Research Laboratoryheitmeyer@itd.nrl.navy.mil, 202-767-3596 • What is it? • Toolset that facilitates the application of formal methods - model checking, theorem proving, simulation and automated test case generation – for defining and validating requirements specifications. • Features • Runs on desktop • Enables conversion of natural language requirements into lightweight formal specifications • Permits simulation/animation (i.e., symbolic execution) of the system based on the specifications • Identifies incompleteness (e.g., missing cases) and inconsistencies (e.g., unwanted non-determinism) in specs • Generates tests satisfying various coverage criteria; Criteria include Branch Coverage and MCDC • Scalable to large systems

22. Software Cost Reduction (SCR) • Successes • Lockheed Martin • User of tools since 1999 • Tools used to specify autopilot logic, flight navigation, flight control/management, and airborne traffic/collision avoidance • Tools currently being used on the Joint Strike Fighter • ISS Fault Detection, Isolation Recovery (FDIR) of Thermal Radiator Recovery Subsystem at IV&V Facility. • Error detected in the original requirements documents • Implementation bias in the requirements exposed • ISS Bus FDIR at IV&V Facility • Consistency checker detected missing and incorrect requirements

23. Software Cost Reduction (SCR) • Contexts for best use • System/software with following characteristics: • Systems with modes • Systems with complex control logic • Safety-critical, Fault-tolerant systems • Prerequisites for effective use of the SCR method: • Precise knowledge of the system/software requirements • Domain experts who can answer questions about the requirements

24. Software Developer’s Assistant (SDA)Stewart Bush: Tietronixstewart.bush@tietronix.com, 281.224.4408 • What is it? • Process enactment platform that guides NASA software teams through their project-specific standards, processes, and procedures. • Features • Web based toolset • Helps ensure process tasks are completed in the correct sequence. • Provides all of the tools, instructions, reference materials, and supportive artifacts to enable process compliance. • Supports NASA Flight Software Development Standard • Tailorable and Customizable • Flexible, Robust, Extensible • Provides real time status reporting of multiple projects

25. Software Developer’s Assistant (SDA) • Benefits • Enhances productivity by shortening process learning curve • Automates tedious aspects of project management and process compliance • Reduces effort for CMMI compliance • Supports continuous improvement by sharing best practices and enabling comparison between projects. • Makes process and data more easily visible and accessible. • Successes • Tietronix received SBIR Phase II grant to mature the tool for multiple NASA organizations. Phase III expected in July. • SDA currently implements NASA/JSC Engineering Flight Software process

26. Software Developer’s Assistant (SDA) • Contexts for best use • Currently able to support implementation of NPR 7150.2 and RUP. • Also may soon be able to support IEEE 12207, OpenUP, SCRUM

27. Software Process Assurance for Complex Electronics (SPACE)Richard Plastow : SAICRichard.A.Plastow@nasa.gov, 216-433-3431 • What is it? • Overall approach that includes process and product assurance activities that can be used on complex electronics. • Provides examples on how to use Fault Tree Analysis. Traceability Analysis, etc. on complex electronic devices. • Features • Guide implemented via a web-based interface. • Assists in defining, planning and managing the assurance process. • Includes document templates, techniques, and checklists • Benefits • Provides a process checklist for each phase of the life cycle • Includes Best Practices from NASA and Industry

28. Software Process Assurance for Complex Electronics (SPACE) • Successes • Checklists used to do spot checks on several projects • Currently being used by several projects in their initial stages of development • Currently being evaluated by multiple projects for use in development of their FPGAs • Contexts for best use • Gives the best results when used starting with the initial project development but provides value to projects as far along as the test phase.

29. CodeSurferPoC: Mark Zarins, GrammaTech, Inc.mzarins@grammatech.com, 408-246-9100 • What is it: Intelligent C/C++ source code browser for: • Program understanding • Manual code inspection • Safety/Security auditing • Features • Commercially supported product • Code navigation • Graphical reports (including call graphs) • Advanced control flow and dataflow analysis • Powerful searching

30. CodeSurfer • Benefits • CodeSurfer makes it easy to analyze and understand code. • Successes • Used by NASA, Mitre, MIT, Thales, Airbus, and many other organizations • Successful 2004 Research Infusion collaboration at JSC: • “Our group analyzes many mission-critical software projects to reduce defects and ensure that the software meets requirements. We conducted a formal study to see if CodeSurfer could improve our software inspections. We found that CodeSurfer reduced inspection time by an average of 30%. In addition, when using CodeSurfer the number of defects found more than doubled.“ • Contexts for best use • Manual code inspection/reviews • Works on compilable C source code • Best applied smaller applications (< 250K LOC).

31. CodeSonarPoC: Mark Zarins, GrammaTech, Inc.mzarins@grammatech.com, 408-688-1243 • What is it: • Automated code analysis tool that identifies serious defects in C/C++ code. • Can be used on a whole program or program fragments. • Features • Commercially supported product • Finds over 50 types of problems, including buffer overruns, null pointer dereferences, uninitialized variables, and memory/resource leaks. • Users can extend the analysis with custom checks. • Some checkers enforce coding recommendations for software reliability and safety developed by Gerard Holzmann’s group at NASA JPL. • HTML reports show details of bugs and paths upon which they occur.

32. CodeSonar • Benefits • Identification of many types of defects, including interface defects. • More coverage than test suites alone. • Easy to apply; no user input is required. • Successes • Many commercial customers, including Smiths Aerospace/GE: • "CodeSonar does a better job of finding the more serious problems, which are often buried deep in the code and sometimes hidden by unusual programming constructs that are hard for other static analysis tools to parse. CodeSonar is not easily fooled." • Contexts for best use • Need compilable C source code; build application with a standard C/C++ compiler; CodeSonar monitors build and captures information to identify defects. • Can handle both small and huge (millions of lines of code) applications.

33. Klocwork InspectPoC: Tim Oliver, Klocwork, Inc.tim.oliver@klocwork.com,770.529.7111 • What is it? • Automated source code analysis tool for detecting logical code problems, metric violations and architectural violations in C and C++, and Java code. • Features • Detects over 125 pre-defined defect rules out of the box • Customizable to include user-defined defect rules • Can filter to view defect by severity, status, type, and scope • Can identify new defects and fixed defects in current build • Can be integrated into organization’s build & issue-reporting process • Relatively high accuracy (identifies real defects with relatively few false positives or false negatives) compared to most other source code analysis tools • Includes trend analysis, architectural analysis, and metrics. • Benefits • Identification of defects early in the development lifecycle  reduced cost to fix. • High accuracy  less time spent on filtering false alarms

34. Klocwork Inspect • Successes • Positive evaluation by Gerard Holzmann at JPL on several example applications including 400 K non-comment source code flight application. • Used by HP and other large software developers. (Largest application in use is over 60 MLOC). • Also used by NSA, DOD, DISA, DOA, IRS, SPAWARS, US Missile Defense Command, LMCO, Boeing, Motorola, Sysco and others. • Contexts for best use • Integrated into the development environment. Best used at the developer desktop • Target applications should follow good programming practices guidelines

35. SpecTRM (Specification Tools and Requirements Methodology)PoC: Grady Lee, Safeware Engineering sales@safeware-eng.com, 206-328-4880 • What is it • Environment for creating & analyzing state-machine based requirements models, with particular emphasis on mission-critical and safety-critical systems. • Features • Emphasis on construction of software requirements models that can be easily read, reviewed, simulated visually, traced and analyzed • Direct support for capturing “intent specifications” to document rationale—not only what and how but also why. • Tool support for completeness in hazard analysis

36. SpecTRM • Benefits • Find consistency/completeness errors at the requirements level, where resolving errors is least costly and most effective • Easily-learned notation enables use by domain experts • Assist in satisfying new NASA safety standard. • Successes • Derived from Prof. Nancy Leveson’s work on TCAS II with NASA and FAA support • Adopted and used by Japanese Manned Space Systems Corporation • SpecTRM-based services provided to automotive, aerospace and medical devices industry by Safeware • Contexts for best use • Software-intensive, mission-critical and safety-critical systems. • Software with complex decision-making algorithms, such as mode and state transition logic, benefit more than systems where complexity is in numerical calculations.

37. Next Step • If you’re interested in a collaboration involving a Research Infusion technology, check out the collaboration proposal process at http://www.nasa.gov/centers/ivv/research/research_infusion_proposal.html • We will help broker matches oftechnology and softwaredevelopers.