1 / 10

Maintaining System & SW Verification Test Consistency

This research program aims to analyze, test, and verify the requirements, build models, simulate, prototype, and integrate the IV&V facility maintenance system software. By closely coupling the system and software development loops, this initiative aims to discover software faults early and reduce project costs.

gcurrin
Download Presentation

Maintaining System & SW Verification Test Consistency

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. IV&V Facility Maintaining System & SWVerification Test Consistency Testing Flight SW Response to Enhanced Failure Modes OSMA Software Assurance Research Program FY03: Initiative 583 FY04: CSIP2004-77 By Ted Bennett & Paul Wennberg 2004 Software Assurance Symposium TRIAKIS Corporation July 2004

  2. Analyze/Test/V&V Analyze/Test/Verify Requirements Build Model,Simulate,Prototype,ES, etc. SYSTEM SW Interpretation Integration Testing Design/Debug Design/Debug Problem Most embedded SW faults found at integ. test traceable to Rqmts. & interface misunderstanding Disconnect exists between System and software development loops

  3. Analyze/Test/V&V Analyze/Test/Verify Requirements Build ES-BasedFull System Simulation (VSIL) SYSTEM SW Integration Testing • Simulate Embedded Controller HW Design/Debug Design/Debug • Replace ES Controller Part • Verify SW Using Unmodified System Tests ROM CPU I/O • Load Object Software RAM Approach • Test Results

  4. Accomplishments • Unmodified ES verification tests give same results when testing object SW running on simulated PowerPC-based SRMS • 131 system-level verification tests written • ES- and DE-based system simulations pass 129 tests • 2 failed tests due to simulator bug

  5. Importance/Benefits • Early discovery of SW faults prior to HW integration testing Unmodified system-level tests can be used to verify embedded object SW Promotes SW assurance through close coupling of system and SW V&V Lower development $$ by finding more faults early, w/o lab or integration HW

  6. Relevance to NASA • Potential to reduce project costs • Improve project-level assurance • Improve IV&V through SW testing • Verify executable SW • Manually generated software • Auto-generated software • Reused/modified software • Real-time operating systems

  7. Problem: FMEA Limitations Expensive & time-consuming List of possible failure modes extensive Focuses on prioritized subset of failure modes Approach: Test SW w/sim’d Failures Create pure virtual simulation of Mini-AERCam HW & flight environment running on PC Induce realistic component/subsystem failures Observe flight SW response to induced failures IV&V Facility Mini-AERCam Empirical Assurance of Embedded SWUsing Realistic Simulated Failure Modes • Can we improve coverage by testing SW resp. to sim’d failures? • Compare results with project-sponsored FMEA: #Failure modes evaluated? #Issues uncovered? Effort involved?

  8. Mini-AERCam Project Importance/Benefits • Virtual environment testing requires no HW • HW integration lab expensive, scarce • Simulate more failure modes than analysis permits • Beyond practical capability of HW integration labs • More failure modes tested  more bugs found • SMA use of simulator for orthogonal testing • System & SW implementation tested – not just design

  9. Mini-AERCam Project Relevance to NASA • Addresses failure mode growth due to increasing system complexity & autonomy • Mini-AERCam TBU for spacecraft inspections • Space shuttle, ISS, CEV, et al • Applicable to all NASA embedded systems • Multiple uses for simulator • Systems & SW Development • Project-level Safety & Mission Assurance • Independent Verification & Validation • Post-launch support

  10. Mini-AERCam Project Accomplishments • Project begun in earnest April ’04 • Well into simulator development Next Steps • Complete Mini-AERCam simulator • Write failure mode tests • Test SW response to enhanced failures • Compare empirical results with FMEA results

More Related