1 / 24

Software Reliability Engineering: A Roadmap

Future of Software Engineering ICSE ’ 2007 Minneapolis, Minnesota May 24, 2007. Software Reliability Engineering: A Roadmap. Michael R. Lyu Dept. of Computer Science & Engineering The Chinese University of Hong Kong. Introduction.

sacha-cotton
Download Presentation

Software Reliability Engineering: A Roadmap

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Future of Software Engineering ICSE’2007 Minneapolis, Minnesota May 24, 2007 Software Reliability Engineering: A Roadmap Michael R. Lyu Dept. of Computer Science & Engineering The Chinese University of Hong Kong

  2. Introduction • Software reliability is the probability of failure-free operation with respect to execution time and environment. • Software reliability engineering (SRE) is the quantitative study of the operational behavior of software-based systems with respect to user requirements concerning reliability. • SRE has been adopted by more than 50 companies as standards or best current practices. • Creditable software reliability techniques are still in urgent need.

  3. Historical SRE Techniques: Fault Lifecycle • Fault prevention: to avoid, by construction, fault occurrences. • Fault removal: to detect, by verification and validation, the existence of faults and eliminate them. • Fault tolerance: to provide, by redundancy and diversity, service complying with the specification in spite of manifested faults. • Fault/failure forecasting: to estimate, by statistical modeling, the presence of faults and occurrence of failures.

  4. Fault Lifecycle Technique Fault Manifestation and Modeling Process Reliability Fault Prevention Fault Removal Fault Tolerance Fault/Failure Forecasting

  5. Fault Lifecycle Technique Fault Manifestation and Modeling Process Reliability Availability Safety Security Fault Prevention Fault Removal Fault Tolerance Fault/Failure Forecasting

  6. Software Reliability Modeling  R = e -t Testing Time

  7. Current SRE Process Overview

  8. Current Trends and Problems • The theoretical foundation of software reliability comes from hardware reliability techniques. • Software failures do not happen independently. • Software failures seldom repeat in exactly the same or predictable pattern. • Failure mode and effect analysis (FMEA) for software is still controversial and incomplete. • There is currently a need for a creditable end-to-end software reliability paradigm that can be directly linked to reliability prediction from the very beginning.

  9. Future Direction 1: Reliability-Centric Software Architectures • The product view – achieve failure-resilient software architecture • Fault prevention • Fault tolerance • The process view – explore the component-based software engineering • Component identification, construction, protection, integration and interaction • Reliability modeling based on software structure

  10. Future Direction 2: Design for Reliability Achievement • Fault confinement • Fault detection • Diagnosis • Reconfiguration • Recovery • Restart • Repair • Reintegration

  11. Fault Confinement Offline Online Fault Detection Fault Detection Failover Diagnosis Repair Recovery Reconfiguration Restart Reintegration

  12. Future Direction 3: Testing for Reliability Assessment • Establish the link between software testing and reliability • Study the effect of code coverage to fault coverage • Evaluate impact of reliability by various testing metrics • Assess competing testing schemes quantitatively

  13. Positive vs. negative evidences for coverage-based software testing

  14. I II III IV V VI RSDIMU test cases description

  15. The correlation: various test regions • Linear modeling fitness in various test case regions • Linear regression relationship between block coverage and fault coverage in the whole test set Fault Coverage

  16. The correlation: normal operational testing vs. exceptional testing • Normal operational testing • very weak correlation • Exceptional testing • strong correlation

  17. The correlation: normal operational testing vs. exceptional testing • Normal testing: small coverage range (48%-52%) • Exceptional testing: two main clusters Fault Coverage Fault Coverage

  18. Coverage Based Testing Time Based Models The Spectrum in Software Testing and Reliability - user oriented - tester oriented - more physical meaning - less physical meaning - abundant models - lack of models - easy data collection - hard data collection - less relevance to testing - more relevance to testing New Model Software Reliability Growth Models Coverage-Based Analysis • A new model is needed to combine execution time and testing coverage

  19. Dependency factors A New Coverage-Based Reliability Model • λ(t,c): joint failure intensity function • λ1(t): failure intensity function with respect to time • λ2(c): failure intensity functionwith respect to coverage • α1,γ1, α2, γ2: parameters with the constraint of α 1 + α2 = 1 joint failure intensity function failure intensity function with time failure intensity function with coverage

  20. Estimation Accuracy

  21. Future Direction 4: Metrics for Reliability Prediction • New models (e.g., BBN) to explore rich software metrics • Data mining approaches • Machine learning techniques • Bridging the gap of the one-way function: feedback to building reliable software • Continuous industrial data collection efforts – demonstration of cost-effectiveness

  22. Future Direction 5: Reliability for Emerging Software Applications • “The Internet changes everything” • On-demand customizable software • Service oriented architecture, composition, integration • Customization by middleware – from metadata to metacode • A common infrastructure delivers reliability to all customers

  23. Replication Manager 6. Invoke Web service Web Service Web service selection algorithm • Create Web services • Select primary Web • service (PWS) Web Service Web Service IIS Application IIS IIS Database WatchDog Application Application Database Database • Keep check the availability of the PWS • If PWS failed, reselect the PWS. Client 3. Register 9. Update the WSDL Port Application UDDI Database Registry 4. Look up WSDL 5. Get WSDL A Paradigm for Reliable Web Service

  24. Conclusions • Software reliability is receiving higher attention as it becomes an important economic consideration for businesses. • New SRE paradigms need to consider software architectures, testing techniques, data analyses, and creditable reliability modeling procedures. • Domain specific approaches on emerging software applications are worthy of investigation. • Still a long way to go, but the directions are clear.

More Related