Software Research and Technology Infusion 23 January 2008 Presented by

1. Software Research and Technology Infusion 23 January 2008Presented by Lisa Montgomery, NASA Lisa.P.Montgomery@nasa.gov P. Luigi Long, RI Contractor Pier.L.Long@ivv.nasa.gov

2. Overview • Background • Goal & Approach • Collaboration Concept • Funding for Collaboration • Selected Technologies • High-Level Descriptions of each of the 25 Technologies • Collaboration Roles • Next Steps

3. Background Materialized as a collaborative effort between Office of the Chief Engineer and the Software Assurance Research Program (SARP). Goal: Transfer mature technology into practice …and reduce the risk of doing so NOT – further develop the technology January 2008 3

4. Background • As a part of the SARP, Research Infusion (RI) seeks to support NASA’s missions. To do that, we look to the Centers both to propose work and evaluate those proposals. • Selection recommendations are made by a group representing most, if not all Centers. This group will be reconfigured this year to ensure balance. • Final approval is given by the SEB so that an Agency perspective is maintained.

5. FY07 Research Infusion Initiatives • Infusion of Perspective-Based Inspection in NASA IV&V • Infusion of Requirements Assistant (RA) into CEV IV&V Validation Activities • Supporting Model-Based Systems and Software Engineering with SpecTRM • Technology Infusion of CodeSonar into the Space Network Ground Segment • Technology Infusion of SAVE into STRS Architecture Compliance Verification at GRC • Technology Infusion of SDA into the MOD Software Development Process January 2008

6. Previously completed Research Infusion Initiatives • Technology Infusion of SAVE into the Common Ground Software Development Process for NASA Missions at JHU/APL • Application of SCR to ISS Biological Research Project On-Orbit Crew Displays at ARC • Application of SpecTRM at JPL's Advanced Project Design Team (TeamX) • Infusion of CodeSurfer into TCMS Sustaining • Infuse CodeSurfer into NASA Code S IV&V Process GSFC FSB Application of Perspective-Based Inspections • Visit http://sarpresults.ivv.nasa.gov for the deliverables from these efforts that have been cleared for public release January 2008

7. Infusing Software Research and Technologies • Intent of RI is to support increased software assurance and technical excellence • By providing an opportunity for NASA project teams to evaluate new technologies • While mitigating some of the risks • Approach • The RI Team identifies technologies to solve Software Development and Assurance challenges • Surveys new SW engineering research areas • Identifies promising technologies which could be adopted by NASA • The Team also surveys the commercial marketplace for potential technologies not already in widespread use in NASA

8. Infusing Software Research and Technologies • Approach (continued) • Offer selected technologies to the NASA software development/assurance community • Foster collaborations between the technology developers and NASA software developers and SQA • Provide funding to reduce the risk of applying a new technology • Generate empirical data to support good engineering decisions about the value of adopting these technologies. January 2008 8

9. Collaborations • How • Initiated by a individual involved with software development or assurance who wants to bring on board a candidate technology • Purpose • Benefit the software development project • Validate the technology • Generate empirical data to assess adoption • Not intended to develop the research • Funding available for— • Training and consulting in the use of the technology • License fees in the case of commercial technologies • Applying the technology • Collecting & analyzing data • Reporting results

10. Funding for Collaborations • Funding for 5 - 7 collaborations available via the Software Assurance Research Program (SARP). • History: 15+ projects in the range $15K - $45K • Competition for SARP funds is among the NASA Centers and JPL. Proposals must come from a civil servant or a contractorwho has a contractual vehicle in place with NASA. • Scope and POP of contract must be able to support the collaboration • Note: NO NEW CONTRACTS WILL BE AWARDED • Proposal template and instructions on the Research Infusion website • www.nasa.gov/centers/ivv/research/research_infusion_index.html • Proposals Due: By 5:00 PM ET Friday, 21st March 2008 • Collaborations Start: 9th June 2008

11. Funding for Collaborations (cont.) • Mechanization • The Principal Investigator (PI) represents the organization which plans to apply the new technology. PI can be a civil servant orcontractor. • Proposals must identify a NASA CS Point of Contact (POC) responsible for managing the collaboration • If PI is a contractor, often the POC is the COTR or technical manager on the PI’s contract • POC is responsible for coordinating the mechanization of the funding • Either the PI or the POC can pay the technology provider • In-kind funding is welcome!

12. Selected Technologies • Identified from • NASA-sponsored software engineering and assurance research • Leading edge commercial tools • Center input • Reviewed by researchers experienced in tech transfer of software engineering research • Send us suggestions for next time. • SE & SA development problem areas • SE & SA technologies • Send suggestions to researchinfusion@ivv.nasa.gov

13. Selected Technologies (cont) • Technology Selection Criteria • Focus on Software Development or Software Assurance • Address a known need/requirement: • Software Architecture Specification and Analysis • Model based software development and assurance • Improvement of SW development processes • Enhanced SW verification • Robust and mature with good user documentation • Demonstrated successes outside of a single domain or application • Not currently in widespread use within NASA • Assurance of user support from technology providers

14. Selected Technologies (cont) • List and detailed description of offered technologies provided on RI Website: http://www.nasa.gov/centers/ivv/research/research_infusion_technologies.html • Over 40 technologies reviewed • Twenty-five technologies selected for 2008 Infusion: • 21st Century Effort Estimator (2CEE) • Architectural Analysis and Design Language (AADL) • Architecture Tradeoff Analysis Method (ATAM) • Attribute Driven Design (ADD) • CONFIG Hybrid Simulator (CHS) • Defect Detection and Prevention (DDP) • Java PathFinder (JPF) • Model Checking Artificial Intelligence-Based Planners (MAP) • ODC Defect Analysis Technology • PathMATE Transformation Engine (PMTE)

15. Selected Technologies (cont) • List of 25 technologies selected for 2008 Infusion: • Quality Attribute Workshops (QAW) • RAPID RMA (RRMA) • Reactis • ReaGeniX Programmer • Reconciler Text Analysis Tool and Aerospace Ontology (RTATAO) • Requirement Assistant • Safety-Critical Application Development Environment (SCADE) Suite • Software Architecture Visualization and Evaluation (SAVE) Toolset • Software Process Assurance for Complex Electronics (SPACE) • Software Reuse Analysis Environment (SRAE) • Systems Testing and Operations Language (STOL) Analysis Tool • Testability And Engineering Maintenance System (TEAMS) • UML Dynamic Specification (SARA Tool) • Unit Testing (CTA++) • Views and Beyond Approach to Software Architecture Documentation (V&B)

16. High Level Description of Technologies 1. 21st Century Effort Estimator (2CEE) • Result of four years of research using machine learning technique to study model calibration and validation techniques • Probabilistic • Key Features: • Dynamic calibration of models using variable reduction and nearest neighbor search • Can be used as either a model analysis tool, calibration tool, and/or an estimation tool • Can estimate with partial inputs • Uses median not mean to evaluate model performance • COCOMO II parameters (can be partial set) • Runs in Windows, coded in Visual Basic • Will be running it in parallel with core tools over next year • NASA Centers and Universities can obtain a copy by sending a request to • Softwarerelease@jpl,nasa,gov

17. High Level Description of Technologies 2. Architectural Analysis & Design Language (AADL) • Existing AADL and supporting open-source tool (OSATE) supports precise modelling and analysis of real-time embedded software architectures. • Research products provide a framework for utilizing AADL/OSATE in assurance environment: both within development (V&V) and independent (IV&V) contexts. • The toolset operates in eclipse framework, so it is compatible with any host system running eclipse • Models can be defined at various levels of abstraction, supporting model development and analysis early in the development life-cycle and identifying issues that might otherwise go undetected • First year of current effort (2007) utilized AADL to study an architecture framework under development by JPL (Mission Data System – MDS). Second year (2008) will continue to elaborate the MDS modelling – supporting current development of the system at JPL. • This research focuses on leveraging an existing technology to develop a comprehensive analysis framework for embedded software architectures.

18. High Level Description of Technologies 3. Architecture Trade-off Analysis Method (ATAM) • ATAM exposes architectural risks that potentially inhibit the achievement of an organization's business goals. • It also provides insight into how those quality goals interact with each other-how they trade off against each other • Clarified quality attribute requirements • Improved architecture documentation • Documented basis for architectural decisions • Identified risks early in the life-cycle • Increased communication among stakeholders • Used on the Earth Observing System Data Information System (EOSDIS) Core System (ECS) • The SEI is currently looking for organizations that would like to incorporate the ATAM as one of their routine software development practices and for partners to be certified to perform SEI-authorized ATAM evaluations.

19. High Level Description of Technologies 4. Attribute Driven Design (ADD) What is it? • It is a method for designing the software architecture of a software system(s) to ensure that the resulting products have the desired qualities. • It is based on the explicit description of both functional and quality requirements • ADD is iterative in that first an overall pattern is chosen and subsequently the overall pattern is refined to achieve a final architectural design. How would it benefit? • The use of ADD would benefit software design at NASA by ensuring that the design satisfies quality attribute requirements. IV&V would be simplified in the area of quality attribute requirement satisfaction since on step of ADD explicitly tests the quality attribute requirements and recording the results of that testing will yield an IV&V trail through the evolution of the design. Where used? • It has been used in the design of geographic information systems and Automated Guided Vehicle (AGV) systems; Also used by Robert Bosch to design automotive components

20. High Level Description of Technologies 5. CONFIG Hybrid Simulator (CHS) • What is the Technology • Combines discrete event and discrete time simulation • Accepts abstract or approximate design models • Dynamic model reconfiguration during simulation to accommodate changes in energy flows in the architecture • SARP use • Assess plausibility and severity of hazard and failure scenarios where system and software interact • Analyze event cascades, evaluate system-software safety cases • Previous NASA use: • Validate automation software • CONFIG hardware models interfaced to the control software

21. High Level Description of Technologies 6. Defect Detection & Prevention (DDP) • It is Allows users to perform a variety of system engineering/risk management activities • FDPP PACT Effectiveness ‘pre-canned’ information or previous DDP evaluations • Existing schedules, preliminary risk elements and mitigation options • Requirements trees, fault trees, etc. at various levels of importability • Information can be entered prior to sessions or in ‘real time’ • Project Requirements and their relative weights • Article Trees (breakdown of system into subsystems into assemblies, etc.) • Failure Modes and Risk Elements (from high-level categories to low-level mechanisms) • PACT options (from high-level types to specific activities) • Additional information: http://ddptool.jpl.nasa.gov/docs/DDP-Seminar-mar23-2001.ppt

22. High Level Description of Technologies 7. Java PathFinder (JPF) • Goal: “get confidence where testing doesn’t work” • Automatically finds hard-to-test defects in concurrent and highly modal programs • Gets complete information to analyze and reproduce defects • Automatically compute interesting data values for test cases • Approach: “execute program in all possible ways” • Builds “Virtual Machine” (VM) to explore all scheduling sequences and data choices systematically • Keeps track of program states explored already to avoid expensive re-execution • Solution: “JPF - the Swiss Army knife of Java program verification” • Java Pathfinder (JPF) - a fully backtrack able, state matching Java VM that can be configured and extended in many ways (search strategies, execution modes, library abstractions and many more)

23. High Level Description of Technologies 8. Model-Checking Artificial Intelligence-Based Planners (MAP) • MAP converts ASPEN Planner input models to Promela, the language of the Spin Model Checker • Can be tested very thoroughly against a set of formalized correctness properties (i.e. safety or liveness requirements) to ensure that certain high risk plans to not exist. • This tool provides the means to improve the thoroughness of testing autonomous planning systems, in particular, verifying input models to the ASPEN planner • MAP was used to test the Earth Observer 1 input model. No errors were found in this model after exploring millions of paths. • Java sources are available and can be compiled for any platform that supports Java – i.e. PCs, Linux, etc.

24. High Level Description of Technologies 9. Orthogonal Defect Classification (ODC) Defect Analysis Technology • What is the Technology? • Defect Analysis technology • Platform independent • Informal but systematic, and it’s comprehensive • (so low effort/bug allows you to classify all software bugs) • What does a project need to do? • Either manually or automatically capture bugs (e.g., MS Access) and classify them according to the ODC categories; output these classifications to a reporting tool (e.g., MS Excel) • Additional Information: • Work has successfully characterized safety-critical, post-launch (operational) software anomalies using data from seven spacecraft

25. High Level Description of Technologies 10. PathMATETransformation Engine (PMTE) • What is the Technology • PathMATE transforms MDA Platform-Independent Models (PIMs) directly into high-performance embedded C, C++ & Java. • Integrated with leading UML modelling environments including Rational Rose, RSM/RSD/RSA, Rhapsody and Topcased • Benefits • Highest performance generated code: 1/3 code size, 7X faster • Much higher reuse of large grained components, even across varying platforms (implementation language, target RTOS, target topology) • Automated production of custom system documentation • Successes • Portable Scalable (Radar) Signal Processor • Nuclear Plant Control System – Embedded Central Controller • SLAMRAAM JDNS Translation/Routing

26. High Level Description of Technologies 11. Quality Attribute Workshops (QAW) • What is the Technology • QAWs provide a method for identifying a system’s architecture critical quality attributes, such as availability, performance, security, interoperability, and modifiability, that are derived from mission or business goals. • Benefits • QAW contributes to software assurance by providing quality attribute scenarios • A concrete response measure that can be used to guide development to ensure the system achieves important quality attribute goals. • Successes • It has been used by the SEI mostly in command and control application domains  • QAW complements the Architecture Tradeoff Analysis Method (ATAM)

27. High Level Description of Technologies 12. RAPID Rate Monotonic Analyses (RRMA) • What is the Technology • RRMA analyses for predictable worst case response times rather than traditional simulation based- average response statistics. • Also provides what-if analysis to help pinpoint potential bottlenecks or resource contention problems while in the design phase. • Potential applications include Performance Critical, Mission Critical, Safety Critical deployed systems where timing failure results in unacceptable harm. • Benefits • Benefits include early detection of possible architecture flaws. • Has peen applied to Datalink multi-aircraft communications project and other classified programs – Raytheon Tucson and LA, NGC Florida, Mitre Bedford, Mass., Aerospace LA. • Well described and limited execution and deployment resources. • Meaningful timing expectations. • Semi-formal design process using a tool such as Telelogic Rhapsody or IBM/Rational Rose Real-Time

28. High Level Description of Technologies 13. Reactis • What is the Technology • Reactis is a testing and validation package for Simulink/Stateflow models. It includes three components: • Reactis Tester - automatically generates tests to exercise as much of the model as possible • Reactis Simulator - an advanced debug environment for models in which you can execute the automatically generated tests. Also does reverse execution and detailed coverage tracking. • Reactis Validator - search for executions of the model that violate the requirement. If it finds a violation it returns a test that can be executed in Simulator to demonstrate the problem. • Benefits • Fewer bugs in software, lower costs to develop software, assist engineers to develop higher quality software more quickly • Successes • Over 40 major companies use Reactis in the automotive and commercial aerospace industries.

29. High Level Description of Technologies 14. ReaGeniX Programmer • What is the Technology • ReaGeniX is an automated modelling and development methodology for real-time and embedded systems. • For developers of control oriented software • Benefits • Automates, removes or helps complex issues • Speeds up development and improves quality • Quick, compact and easy to test, with re-usable software components • Additional Information • Easy software assurance • Easy maintenance, with scope of change easy to locate • Impact of change simple to verify • Has been used to develop commercial products • Uses MS Visio inputs, producing ready-to-use C Code

30. High Level Description of Technologies 15. Reconciler Text Analysis & Tool Aerospace Ontology (RTATAO) • Reconciler Text Analyzer • Semantic parsing for text analysis, word/phrase classification and tagging • Aerospace Ontology • Taxonomy of types of objects, functions/actions and problems for aerospace Extensive problem nomenclature for hazards (hardware, software and human) • Thesaurus with synonym lists, to accommodate varying terminology • SARP Project Use • Extracts model descriptions from Orion requirements and design • For semi-automatic generation of system models for software safety analysis • Additional Information • Current NASA use: Semantic text mining to classify engineering Discrepancy Reports (DRs) for trend analysis Trend analysis of problem reports – groups reports with similar problems • Discrepancy Reports: mechanical, electrical, software and process Browse, graph, sort and trend similar problem reports

31. High Level Description of Technologies 16. Requirements Assistant (RA) • What is the Technology • An analysis tool that is designed to help ensure that requirements are complete, consistent, feasible, and unambiguous, using text in NL as input • uses heuristics derived from analysis of hundreds of requirements reviews to enhance the natural language processing of the requirements and identify incorrect, inconsistent, ambiguous, or missing requirements • Benefits • A proven solution for the problem of poorly written requirements, combining the knowledge of many reviews with research to find defects in requirements. • Additional information • When new error types are found during reviews the Requirements Assistant™ can add this knowledge, as a new rule. This update capability of the tool enables the user to incorporate the lessons learned.

32. High Level Description of Technologies 17. Safety-Critical Application Development Environment (SCADE) Suite • A model-based development and auto code generation environment. It contains qualified development tools and verification tools to either automate, simplify or remove the costly V&V activities required by the DO-178B standard. • Potential applications include a development environment for safety-critical software. • Benefits include: • Removal of human coding errors, improving system quality, improve cycle time for design changes by 3-4x, and improve time to market by 40-50%. • Supported platforms include MS Windows XP. • This technology is currently being applied in the United States on 11 different projects, 3 evaluation programs and 1 research project. SCADE has been audited by the FAA, EASA (European), Transport Canada and CAAI (Israel). SCADE has been deployed on programs with Boeing, Airbus, Lockheed Martin, and GE.

33. High Level Description of Technologies 18. Software Architecture Visualization and Evaluation (SAVE) Toolset • What is the Technology • Automatically extracts, analyzes and visualizes the architecture of source code. • Can also compare the source code architecture with user-specified architectural models and rules • Benefits • The comparison of source code architecture with user-specified architectural models and rules. • The analysis of C/C++ and Java code, but ADA and Fortran parsers are also available. SAVE can also be used to analyze the product-line potential and deviations of the software in terms of architectural commonalities and differences between different software products. • Successes • This technology has been applied to a number of different real systems including digital cameras, automotive software, ground systems, flight software and CAD software.

34. High Level Description of Technologies 19. Software Process Assurance for Complex Electronics (SPACE) • What is the Technology • Takes proven tools and techniques and applies them to complex electronic devices. • Defines a process for both the software and the hardware cycle of development. • Benefits • Can be used by any Quality Assurance Engineer • Complexity and Safety Guidelines • Has process checklists for each stage of the devices’ life cycle • Product checklists for code reviews, requirements reviews, etc. • Additional Information • Templates for a Complex Electronics Assurance Plan. • Website available for product information & techniques http://www.hq.nasa.gov/office/codeq/software/index.htm

35. High Level Description of Technologies 20. Software Reuse Analysis Environment (SRAE) • What is the Technology • Web-application capable of analyzing spacecraft SW • Aids developers and analysts in accurately estimating software reuse based on context variables. • Can monitor projects based on reuse calculations, performing project planning w.r.t. SW reuse, validating software reuse claims, and aiding in the development of reusable software components. • Benefits • Only web-based toolset capable of evaluating software reuse from inception to decommission. • Additional Information • Platforms: MS Windows: IE 5.5+, Firefox 2.0+; on Linux Firefox 2.0+. • Has been applied to ground and space SW at JPL to determine the percentages of software reuse and the accuracy of early estimates by the developer. • Applied to specific spacecraft subsystems at GSFC to determine the reusability and the feasibility for developing plug-and-play reusable SW components based on legacy subsystems..

36. High Level Description of Technologies 21. Systems Testing & Operations Language (STOL) Analysis Tool • What is the Technology • Automated tool to help analyze STOL test scripts and their associated logs for test coverage • Benefits • Improve the quality and timeliness of test verification of STOL- based systems. • Ongoing standards activities could help unify, advance and broaden the use of a common STOL-like language in future NASA projects. • Additional Information • Infusion partners would be encouraged to participate in an Open Source style web-based feature/bug tracking system and discussion forums • Visual displays and annotations only, no printable reports or graphics. Internal model representations are exportable as XMI.

37. High Level Description of Technologies 22. Testability And Engineering Maintenance System (TEAMS) • What is the Technology • Model based Analysis (DFT, FMECA) and real-time diagnostics, guided troubleshooting utilizing reasoner technology. • Benefits • Can be used for V&V of contingency procedures and FDIR. • Produces a variety of analysis reports, real-time health status, step by step guided troubleshooting instructions. • Additional Information • Supported platforms include PC, Linux, Solaris. • This technology has been used by ARC and MSFC for design analysis of the Orion, being considered for ground support systems by them and KSC. • JSC, Lockheed-Martin. Honeywell utilizing it for in-flight Vehicle Health Determination of Orion.

38. High Level Description of Technologies 23. Software Architecture Risk Assessment (SARA) Tool • What is the Technology • The V&V of dynamic specifications • A utility that provides SW engineers and developers the ability to compute and analyze different architectural risk factors of SW architectures modelled using UML. • Benefits • Include defining and investigating metrics for domain architectures. • The ability to define metrics so as to reflect relevant qualities of domain architectures, and to alert the software architect to risks in the early stages of architectural design • Additional Information • Supported platforms include PC, Windows • Website: http://www.csee.wvu.edu/swarch/public_html/SARATool/

39. High Level Description of Technologies 24. CTA++ • What is the Technology • A tool for unit testing C++ classes, libraries and subsystems • The testing process becomes efficient, visible and organized - as required in a professional testing environment. • Benefits • Works fully in C++ .Supported platforms include Windows, Linux, Solaris, HPUX. • IDE integration to MS Visual Studio. • Multithread support: in code under test, in using multiple test drivers. • Additional Information • Automated, repeatable tests, trace file => visibility, regression tests. • Can be used with code coverage tools, e.g. with Testwell CTC++. Website: http://www.testwell.fi/ctadesc.html

40. High Level Description of Technologies 25. Views and Beyond Approach to Software Architecture Documentation (V&B) • What is the Technology • V&B includes a method for choosing the relevant views based on the structures that are inherent in the software architecture and on the needs and concerns of the architecture documentation's stakeholders. • Benefits • Decreased development costs due to less backtracking • Decreased life cycle costs due to more focused maintenance efforts • Increased likelihood of architecture (and therefore the system) meeting its requirements • Additional Information • This technology has been applied and taught to government and civilian organizations, and has been adopted by many of them. • The US Army's Future Combat Systems project uses an architecture documentation organization whose roots come from V&B.

41. Collaboration Roles • Roles of the Principal Investigator • During proposal preparation: • Works with technology provider to plan collaboration and select suitable application • Must have buy-in from the technology provider • Writes and submits the proposal • Should proposal be selected: • Coordinates training course with developer • Identifies software artifacts to which the technology will be applied • Applies the technology (may require multiple iterations) • Collects data & evaluates its performance • Writes final report

42. Collaboration Roles (cont) • Roles of the Technology Provider : • During proposal preparation • Helps to plan the collaboration, including assisting in the selection of a suitable application • If Principal Investigator’s proposal is accepted • Provides any necessary training course (preferably on-site) • Provides tutorial and other user documentation • Provides customer support throughout the collaboration

43. Next Steps • If you’re interested in a collaboration involving a Research Infusion technology, check out the collaboration proposal process at:- http://www.nasa.gov/centers/ivv/research/research_infusion_proposal.html • We will help broker matches oftechnology and softwaredevelopers.

44. Next Steps for FY08 (and beyond) Proposal Template released Friday, 25th January Solicitation closes Friday, 21st March Initial recommendations made Friday, 18th April SEB meets Friday, 2nd May Work for FY 08 initiatives should begin 2nd June FY 09 Research Infusion begins Telecons for the FY09 Research Infusion activities should be held in July January 2008

45. Final Thoughts Research Infusion should be an opportunity to try an approach that you and your team thinks will help you do your work better We are here to help If you need more information, If you need access to previous work not yet published, If you need help making contact, If you need additional support, contact us January 2008

46. Summary • Background • Goal & Approach • Collaboration concept • Funding for Collaboration • Selected Technologies • High-Level Descriptions of each of the 25 Technologies • Collaboration Roles • Next Steps

47. Contact Information • RI Team Email: • researchinfusion@ivv.nasa.gov • Lisa Montgomery, RI NASA Lead • Lisa.P.Montgomery@nasa.gov • Pavan Rajagopal, RI Contractor (Proj. Mgr.) • Pavan.Rajagopal@ivv.nasa.gov • P. Luigi Long, RI Contractor • Pier.L.Long@ivv.nasa.gov • Tools Lab. toolslab@ivv.nasa.gov • Telephone: (304) 367-8304

48. Questions?