1 / 21

Software Reliability Assurance for Real-time Systems

This presentation provides an overview of system development, testing problems, solution approach, and results and conclusions for ensuring software reliability in real-time systems. It includes practical applications and strategies for achieving reliability and safety.

whitsett
Download Presentation

Software Reliability Assurance for Real-time Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Reliability Assurance for Real-time Systems Joel Henry, Ph.D. University of Montana NASA Software Assurance Symposium September 4, 2002

  2. Overview • System development • Testing problems • Solution approach • Results and conclusions • Practical application

  3. System DevelopmentExample: Wind Tunnel Software • Controls devices that control: • Wind generator • Model support • Tunnel atmosphere • Emphasizes reliability and safety • Utilizes multiple development strategies • Based on simple structure

  4. System DevelopmentSimple Structure External Devices Controlling Computers ALGORITHM Sample Inputs Run Software Update Outputs

  5. System Development • Engineer builds graphical model in MATLAB • Models enter simulate-debug-simulate-debug phase • Engineer auto-generates source code • Source code is compiled, linked, and then deployed • Hardware/software integration begins

  6. Testing ProblemsTest Size • Recall simple model • Input variables – sampled over time • Outputs variables – produced over time • Sample time – variable or set frequency • Consider test requirements • Input file/matrix • Output file/matrix • Analysis tools

  7. Testing ProblemsTest Size • Consider an example • 100 input variables • 50 output variables • 100 millisecond sample time • Assume you want to test a one hour operation period • 100 inputs*10 per second*3600seconds = 3,600,000 values • 50 outputs*10 per second*3600seconds = 1,800,000 values • Ignore issues of useful inputs and defect detection

  8. Testing ProblemsDomain Coverage • Domain determinants • Input variable – minimum, maximum, and accuracy • Output variable – minimum, maximum, and accuracy • Consider test requirements • Input file/matrix with all possible values for input • Output file/matrix much more complex problem

  9. Testing ProblemsDomain Coverage • Consider an example for input variable • Input variable for pressure in a tank • Min – 0 • Max – 999.999 • Accuracy – 3 (decimal places) • 1000*1000 = 1,000,000 possible values • Ignore issues of legal sequencing and combinations

  10. Solution ApproachOverview • Automation to: • Generate large input matrices/files • Perform simulation and/or test autogenerated code • Analyze output matrices/files • Methods to: • Evaluate domain coverage • Aid debugging • Evaluate results

  11. Solution Approach Suite of testing tools Generate Tests Simulate Model Test Auto-code Detect Faults Evaluate Results Command and Control Algorithm Verification and Validation Methodology MATLAB/Simulink Environment Executable Code Source Code

  12. Solution Approach Model Information Test Data Test Results Command and Control Algorithm MATLAB/Simulink Environment Suite of testing tools Generate Tests Test Data Simulate Model Test Auto-code Detect Faults Executable Code Test Results Evaluate Results Verification and Validation Methodology

  13. Solution Approach Data Graphs Suite of testing tools Generate Tests Simulate Model Raw Value Files Test Auto-code Detect Faults Evaluate Results Verification and Validation Methodology Completeness, MTTF, Reliability File

  14. Results and Conclusions • Execute multiple tests • Evaluate testing effectiveness • Track trends in model reliability • Automate and evaluate • Specify effectiveness and reliability goals • Evaluate on a per test case basis • Track through testing phase over all tests

  15. Results and ConclusionsExample – Multiple Tests

  16. Results and ConclusionsExample – Multiple Tests Bucket Coverage (%)

  17. Results and ConclusionsExample – Multiple Tests

  18. Results and ConclusionsExample – Multiple Tests

  19. Practical ApplicationWhat? • Can do domain testing supported with automation • Can set quantitative goals • Can evaluate progress toward goals • Can measure MTTF, domain coverage, confidence percentages, and reliability • Can create an organizational history

  20. Practical ApplicationHow? • Invest in automation • Integrate domain coverage with application specific testing • Establish goals and collect data • Calculate MTTF, domain coverage, and reliability • Use common sense with quantitative data

  21. Questions and Contact Info • Joel Henry • henryj@cs.umt.edu • MATT and RATT • http://www.cs.umt.edu/RTSL/design992/links/index.htm • MATLAB users -

More Related