SQA & Reuse

1. SQA & Reuse Katerina Goseva-Popstojanova, WVUAaron Wilson, NASA IV&VKalynnda Berens & Richard Plastow, GRCJoanne Bechta Dugan, UVaDavid Gilliam JPL

2. Projects • Real-time Linux Evaluations Kalynnda Berens & Richard Plastow, GRC • Performability of Web-based applications Katerina Goseva-Popstojanova, WVU • Reducing Software Security Risk through an Integrated Approach, David Gilliam & John Powel, JPL • Software Assurance of Web-based Applications Tim Kurtz, GRC • Software Quality & Safety Assessment Using Bayesian Belief Networks, Joanne Bechta Dugan, UVa

3. Real-time Linux Evaluations • Performance benchmarking on flight-like hardware: • RTLinux (free version) V3.2 pre3 • RTLinux Pro (commercial) V2.0 • RTAI V24.1.11 • Linux 2.6.7 Kernel (future) • Jaluna (future) • RTLinux and RTAI are • Stable • Support many processors • Require a learning curve

4. Which Real-Time Linux is best?

5. Web measurement and modeling framework User session characterization Web access log analysis Realistic workload Session layer (user view) Performance model Software/hardware resource utilization Service layer (software architectural view) Performability model Application & hardware resource monitoring System layer (deployment view) Software/hardware failure/recovery characterization Reliability/ availability model Resource layer (hardware device view) Web error log analysis Request-based and session-based error characterization

6. Cost effective way to improve quality 10-35% of the total number of errors are due to only 3 files Fixing the errors with the highest frequency of occurrence is the most cost effective way to improve Web quality

7. Reducing Software Security Risk Through an Integrated Approach NASA • Software Vulnerabilities Expose IT Systems and Infrastructure to Security Risks • Goal: Reduce Security Risk in Software and Protect IT Systems, Data, and Infrastructure • Security Training for System Engineers and Developers • Software Security Checklist for end-to-end life cycle • Software Security Assessment Instrument (SSAI) • Security Instrument Includes: • Model-Based Verification • Property-Based Testing • Security Checklist • Vulnerability Matrix • Collection of security tools

8. Womb-to-Tomb Process • Coincides with Organizational Polices and Requirements • Security Risk Mitigation Process in the Software Lifecycle • Software Lifecycle Integration • Training • Software Security Checklist • Phase 1 • Provide instrument to integrate security as a formal approach to the software life cycle • Requirements Driven • Phase 2: • External Release of Software • Release Process • Vulnerability Matrix – NASA Top 20 • Security Assurance Instruments • Early Development – Model Checking / FMF • Implementation – Property Based Testing • Security Assessment Tools (SATs) • Description of available SATs • Pros and Cons of each and related tools with web sites • Notification Process when Software or Systems are De-Commissioned / Retired

9. Software Assurance of Web-based Applications • How should NASA SA assure web-based applications? • Solution • Implement the same types of controls on web-apps development that are used on other types of software development • Audit and review projects web-app development activities using a set of checklists • Pilot the guidebook/checklists • Deliverables • Best Practices guidebook • Checklists

10. How can we investigate and document the decision process that is used to go from... to… Is the system good enough to release? I have an acceptable level of belief that the system will operate as specified. Test Results Personal and Team CMM Quality Assurance Formal Methods Requirements Review Prototype Performance Risk Assessment Code Inspection Engineering Judgment for a computer-based system Software Quality & Safety Assessment Using BBN GETR Decision

11. BBN model of Software Development Process

12. Technology Readiness Level Reducing software security risk Web performability Software Quality & Safety

13. Brief description of the field • Quality attributes: reliability, performance, security, maintainability, and reusability • Techniques • Testing: property testing, performance testing • Real system, real workload • Analysis & Modeling: model checking, statistical & probabilistic analysis, BBN • Process & product

14. Potential benefits • Improved decision support, prioritization, better allocation of resources • Better product in a cost effective way through integrated approaches • Increased fidelity without increasing complexity

15. Directions • Increased coordination through unified approaches • Infusion of improved techniques into current processes • Improving the state of practice

16. Why • Potential benefits to NASA • Fewer mission failures • Reduced complexity • Greater reuse of software artifacts and process improvements • Transference of best practices and lessons learned

17. Why not • Standard traps • “There is no silver bullet” • “Teaching to the test” • Deadline vs. quality driven development • Tunnel vision • Dependencies on hardware and OS • Poor documentation and quality of data

18. Who is using this technology • NASA projects that are using this technology • Security checklist at JPL • RT Linux Pro at Glenn • Web performability at NASA IV&V • Web-based process assurance at Glenn • Seal of Approval Process for PRA tools at NASA HQ • Other projects outside of NASA that are using these tools/approaches • Web performability at LDCSEE • Formal security verification at Patchlink

19. Questions/Issues • Reliability, availability, performance, security • Integrated approaches needed • What are the interactions & tradeoffs? • Process & product • Better, Cheaper, Faster • Can we have it all? • Should we pick (any) two?