Automated STOL Analysis Tool for Script Analysis

1. Automated Systems Test and Operations Language (STOL) Analysis Tool Jason G. Cooper jason.cooper@matricresearch.com July 20, 2006 SAS_06_STOL_Tool_Cooper

2. Outline • Problem/Approach Overview • General Discussion • Technology Readiness Level • Potential Applications • Data Availability • Barriers • In-Depth Project Plans SAS_06_STOL_Tool_Cooper

3. The IV&V community has expressed a need for automated assistance in regards to STOL script analyses activities Currently, practitioners have no automated methods to assist in analysis triage for STOL scripts To research algorithms and methodologies most suitable for STOL test script analysis Develop and evaluate an automated analysis tool to assist practitioners with STOL-related tasks Phase One Activities: Initial Knowledge Discovery Tool Software Development Verification and Test Evaluation Recommendations Problem / Approach SAS_06_STOL_Tool_Cooper

4. Outline • Problem/Approach Overview • General Discussion • Technology Readiness Level • Potential Applications • Data Availability • Barriers • In-Depth Project Plans SAS_06_STOL_Tool_Cooper

5. Technology Readiness Level (TRL) • This project begins at TRL 2 • TRL 2 (Technology Concept Formulated) is focusing theories and principles on the specific application area • In this project’s case, the team will be determining complexity and behavioral metrics to be used for automated analysis techniques SAS_06_STOL_Tool_Cooper

6. Potential Applications • The development of such an automated analysis tool has the potential for usage in STOL script activities across multiple domains. • A commonly phrased mantra is “test like you fly, fly like you test.” In following this testing framework, an analysis tool could offer benefits not only to verification of flight software test activities, but also to command scripts used for mission operations. SAS_06_STOL_Tool_Cooper

7. Data Availability • The research team is currently identifying which STOL version to focus initial efforts, as well as most commonly-occurring STOL script errors in the IV&V domain • Several potential projects, which utilized STOL, are available for retrospective analyses • The specific project for test and evaluation will be determined in the latter stages of the software development phase SAS_06_STOL_Tool_Cooper

8. Barriers • STOL Version: determining the most common version of the scripting language will limit the applicability of the initial tool. In the second phase, this may be mitigated by supporting multiple versions of the scripting language. • Metrics: standard complexity metrics will apply to portions of this scripting language. However, certain aspects of the language require new metrics to be assessed. SAS_06_STOL_Tool_Cooper

9. Outline • Problem/Approach Overview • General Discussion • Technology Readiness Level • Potential Applications • Data Availability • Barriers • In-Depth Project Plans SAS_06_STOL_Tool_Cooper

10. Initial Knowledge Discovery • Determine behavioral knowledge to extract from STOL constructs • Determine complexity metrics to apply • Capture most commonly occurring STOL errors in IV&V project for domain knowledge SAS_06_STOL_Tool_Cooper

11. Tool Software Development • Requirements Definition • Architecture Definition • Implementation • Unit Testing and “Seed” Testing SAS_06_STOL_Tool_Cooper

12. Verification and Test Evaluation • Select project and underlying STOL scripts • IV&V practitioner and tool analyze 50% of selected scripts; results comparison • Practitioner utilizes tool for remaining 50% of scripts; feedback and usability captured SAS_06_STOL_Tool_Cooper

13. Questions? jason.cooper@matricresearch.com SAS_06_STOL_Tool_Cooper