Bayesian Framework for Reliability Prediction of Component Based Software Systems

1. Bayesian Framework for Reliability Prediction of Component Based Software Systems Bojan Cukic, Harshinder Singh, Erdogan Gunel, Chip Smith, Vijay Bharadwaj West Virginia University Vittorio Cortellessa Universita di L’Aquila, Italy

2. Overview • Introduction and motivation. • UML based architectural software reliability prediction methodology. • Reliability maximization based on component cost constraints. • ERA tool development. • Summary and further work.

3. Introduction • Utilization of off-the-shelf software components for rapid application development. • Such applications are expected to have high reliability as a result of deploying trusted components. • Software reliability techniques treat the program as a monolith, while it might be a collection of (potentially known and/or reused) components.

4. NASA Interests • New NASA projects tend to specify complex software systems using UML notation. • Revolutionary airframe concepts research and system studies (RACRSS). http://mdob.larc.nasa.Gov/summary01/a12_SALAS.pdf. • Expendable launch vehicle integrated support (ELVIS). • Advanced engineering environment (AEE). • Earth observation system clearinghouse (ECHO) system. http://www.blueprinttech.Com/html/CaseStudies.Asp. • Reusable space exploration software assets (components) under development at JPL. • Significant industrial evidence exists pointing to the cost and quality benefits of product-line engineering in software development.

5. Contributions • Applicability at the architectural level and throughout the system development life-cycle (demonstrated in FY 2001). • Inclusion of probabilistic uncertainties in component reliabilityestimates (FY 2001). • Ability to analyze distributed systems and hierarchical designs (FY 2002). • Consider sensitivity of the system with respect to changes in the subsystems (FY’01 & ’02)

6. Contributions (2) • Reliability optimization with respect to the given cost constraints (FY 2002). • Development of a support tool fully integrated with UML (FY ’02). • Integration with software fault tree analysis to tie in reliability assessment and overall system safety (FY ’03).

7. Process • Provide annotations for the application’s UML diagram(s). • Use annotations as inputs to reliability calculations. • Design level analysis (prediction): • The algorithm predicts expected system reliability from provided (assumed, hoped for) component reliabilities. • Algorithm supports system-wide cost-benefit analysis (what if I provide more reliable components and/or connectors?).

8. UML annotations Annotated Sequence Diagram Annotated Use Case Diagram (interaction of components within a use case) Annotated Deployment Diagram

9. UML annotations • Component failure probability within scenario j • Connector failure probability within scenario j • System failure probability

10. Assumptions • Component failure rates available. • Failure Independence. • A component’s failure probability does not depend on the failure probabilities of the other components. • Regularity. • A component’s and connector’s failure probabilities are the same across all the busy periods. • Pessimism. • Component failure always results in a system failure.

11. A Case Study A WEB-based transaction processing system (WBTPS)

12. From Annotations to Reliability C2R2: Component - Connector Reliability Record

13. Component Reliabilities Component Failure Probabilities as PDFs (Beta Distributions)

14. Reliability Prediction 95% confidence interval of system failure probability is(0.13 , 0.17). Reliability range (0.83 , 0.87) Plot of Prior Probability Density Function of the System Failure Probability S fitted to the normalized histogram from simulation observations

15. Sensitivity Analysis • Change reliabilities of individual components (and/or connectors) and observe the impact on overall system reliability. Improve Web servers C5: 0.009 -> 0.002 C6: 0.005 -> 0.001 QS:0.13 -> QS’: 0.11 Worse remote serversC11: 0.005 -> 0.02 C12: 0.007 -> 0.025 QS’’: about 2% worse

16. Reliability Optimization • Component reliability is an increasing function of its cost. • Optimization problem: • Find out component cost allocations so as to maximize system reliability subject to the fixed total budget. • The methodology has been developed and applied on a case study. • Outstanding Issues: Expressing component reliability as a function of cost, discrete cost functions.

17. ERA Minimum Requirements • MatLab 6.1.0.450 • Rational Rose Professional 2001 • Unisys Rose XMI Tool (Plug-in for Rational Rose) • Windows 98

18. Process Overview

19. Rational Rose Model File Exported Using Unisys XMI Plug-in Failure Probability and 95% Confidence Intervals of Processes Probability of an Actor using the System Probability of an Actor using a Specific Part of the System XML File Containing Simulation Settings Dat File Containing Simulation Results Histogram of Results including Calculation of System Failure Probability and 95% Confidence Interval Line Plot of Histogram and Simulation Beta Curve Inputs Outputs

20. Rational Rose Sequence Diagram

21. Probability of Actor Using a Use Case Process

22. Verifying Busy Periods and Connecting Nodes to Processes in Deployment Diagram

23. Specifying Process Failure and Confidence Intervals for a Processor

24. Specifying Failure and Confidence Intervals for Processor Connections

25. ERA Results

26. Summary • New framework for reliability prediction in component based systems. • Fully integrated with UML. • The methodology will be included (as a proposal) in the standardization effort by OMG in fall 2002. • Tool support (V 1.0) has been released. • Current/future work. • Model improvement and validation. • Identifying NASA project for collaboration. • Integration of fault tree analysis.