1 / 26

Achieving High Software Reliability Using a Faster, Easier and Cheaper Method

Achieving High Software Reliability Using a Faster, Easier and Cheaper Method. The Software Measurement Analysis and Reliability Toolkit. Taghi M. Khoshgoftaar. NASA OSMA SAS '01. September 5-7, 2001. NASA OSMA SAS '01. Outline. September 5-7, 2001. Introduction Overview of SMART

elkinsd
Download Presentation

Achieving High Software Reliability Using a Faster, Easier and Cheaper Method

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Achieving High Software Reliability Using a Faster, Easier and Cheaper Method The Software Measurement Analysis and Reliability Toolkit Taghi M. Khoshgoftaar NASA OSMA SAS '01 September 5-7, 2001

  2. NASA OSMA SAS '01 Outline September 5-7, 2001 • Introduction • Overview of SMART • Data Analysis and Modeling Features • Current Utilization of SMART • Case-Based Reasoning • Empirical Study • Conclusions

  3. NASA OSMA SAS '01 Introduction September 5-7, 2001 • Necessity of an integrated tool for efficient empirical software quality research • Commercial packages are available but expensive and don’t always match our exact needs • In house development gives us availability, flexibility and possibility to evolve

  4. NASA OSMA SAS '01 Overview of SMART September 5-7, 2001 • First version back in 1998 • Current version 2.0 • Written in Microsoft Visual C++ • Runs on Microsoft Windows based platforms • User friendly GUI

  5. NASA OSMA SAS '01 SMART GUI September 5-7, 2001

  6. NASA OSMA SAS '01 SMART GUI September 5-7, 2001

  7. NASA OSMA SAS '01 Features included in SMART September 5-7, 2001 • Data management • Multiple Linear Regression (MLR) • Case-based reasoning (CBR), • Case-based reasoning with two group clustering • Case-based reasoning with three group clustering • Module order modeling (MOM)

  8. NASA OSMA SAS '01 Organizational Flowchart September 5-7, 2001

  9. NASA OSMA SAS '01 Current Utilization of SMARTat the ESEL Laboratory September 5-7, 2001 • Empirical research: • Comparative studies of software quality models • Case studies based on real world systems

  10. NASA OSMA SAS '01 Case Based Reasoning September 5-7, 2001 • Based on automated reasoning processes • Easy to use • Results are easy to understand and to interpret • Looks at past cases that are similar to the present case in an attempt to predict or classify an instance

  11. NASA OSMA SAS '01 Case Based Reasoning: Additional Advantages September 5-7, 2001 • The ability to alert users when a new case is outside the bounds of current experience • The ability to interpret the automated classification through the detailed description of the most similar case • The ability to take advantage of new or revised information as it becomes available • The ability for fast retrieval as the size of the library scales up

  12. NASA OSMA SAS '01 Case Based Reasoning September 5-7, 2001 • Working hypothesis for software quality modeling: • Current cases that are in development will more than likely be fault-prone if past cases having similar attributes were fault-prone

  13. NASA OSMA SAS '01 Case Based Reasoning: Comparing the Cases September 5-7, 2001 • Similar cases to a new module or nearest neighbors are determined by similarity functions: • Absolute Distance • Euclidean Distance • Mahalonobis Distance

  14. NASA OSMA SAS '01 Case Based Reasoning: Prediction Methods September 5-7, 2001 • The value of the dependent variable is estimated using the values of the dependent variables of the nearest neighbors and a solution algorithm: • Unweighted Average • Inverse-Distance Weighted Average

  15. NASA OSMA SAS '01 Case Based Reasoning: Classification Methods September 5-7, 2001 • Used to classify a software module into a particular class (fault-prone, not fault-prone). • The types of classification methods include: • Majority Voting • Data Clustering

  16. NASA OSMA SAS '01 Case Study: System Description September 5-7, 2001 • Two data sets were obtained from two large Windows-based applications used primarily for customizing the configuration of wireless products. The data sets were obtained from the initial release of these applications. The applications are written in C++, and they provide similar functionality.

  17. NASA OSMA SAS '01 Case Study: System Description September 5-7, 2001

  18. Case Study:Data Collection Effort • Data collected by engineers over several months using the available information in: • Configuration Management Systems • Problem Reporting Systems

  19. NASA OSMA SAS '01 Case Study:Independent Variables September 5-7, 2001

  20. NASA OSMA SAS '01 Case Study:Accuracy Evaluation September 5-7, 2001 • Average Absolute Error: • Average Relative Error:

  21. NASA OSMA SAS '01 Case Study:Prediction Results September 5-7, 2001

  22. NASA OSMA SAS '01 Case Study:Classification Evaluation September 5-7, 2001

  23. NASA OSMA SAS '01 Case Study:Classification Results September 5-7, 2001 • Entire data set: • Fit and Test data set:

  24. NASA OSMA SAS '01 Case Study:Return On Investment September 5-7, 2001 • Classification using CBR

  25. NASA OSMA SAS '01 Conclusion September 5-7, 2001 • A tool that matches our needs • Used for our extensive empirical work • Proved useful on large scale case study • Faster • Easier • Cheaper • Ready for future enhancement

  26. NASA OSMA SAS '01 Reminder September 5-7, 2001 We will be presenting the tool on Friday, Please feel free to visit us!

More Related