1 / 58

CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1

CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1. oshana@airmail.net. Industry facts. 30-40% of errors detected after deployment are run-time errors [U.C. Berkeley, IBM’s TJ Watson Lab].

kaspar
Download Presentation

CSE 7314 Software Testing and Reliability Robert Oshana Lecture # 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSE 7314 Software Testing and Reliability Robert Oshana Lecture #1 oshana@airmail.net

  2. Industry facts • 30-40% of errors detected after deployment are run-time errors [U.C. Berkeley, IBM’s TJ Watson Lab] • The amount of software in a typical device doubles every 18 months [Reme Bourguignon, VP of Philips Holland] • Defect densities are stable over the last 20 years : 0.5 - 2.0 sw failures / 1000 lines [Cigital Corporation] • Software testing accounts for 50% of pre-release costs,and 70% of post-release costs [Cigital Corporation]

  3. Critical SW Applications Critical software applications which have failed : • Mariner 1 NASA 1962Missing ‘-’ in ForTran code Rocket bound for Venus destroyed • Therac 25 Atomic Energy of Canada Ltd 1985-87Data conversion error Radiation therapy machine for cancer • Long Distance Service AT&T 1990A single line of bad code Service outages up to nine hours long • Patriot Missiles U.S. military 1991Endurance errors in tracking system 28 US soldiers killed in barracks • Tax Calculation Program InTuit 1995Incorrect results SW vendor payed tax penalties for users

  4. Introduction Chapter 1

  5. History • STEP originally developed out of frustration! • STEP does not establish rules that must be followed • guidelines

  6. History of testing definitions

  7. Preventive testing • Philosophy that testing can actually improve the quality of the software tested if it occurs early enough • The process of writing test cases to test a requirement can identify the flaw in the requirement

  8. Preventive testing • “Withdraw $200 from an account with $165 in it” • “Withdraw $168.46 from an account with $200 in it”

  9. Waterfall model

  10. Why is testing so difficult? • Ambiguous and incorrect requirements • Tight time schedules • Many other !!!

  11. STEP • Introduced in 1985 • Built on IEEE 1008-1987 • Covers a broad activity of software evaluation

  12. Views of testing

  13. CSE 7314 Software Testing and Reliability Robert Oshana End of Lecture oshana@airmail.net

  14. CSE 7314 Software Testing and Reliability Robert Oshana Lecture #2 oshana@airmail.net

  15. Introduction Chapter 1

  16. Views of testing

  17. Elements of STEP

  18. STEP architecture

  19. STEP activities

  20. Activity timing of various levels of test

  21. Activity timing of various levels of test

  22. Parallel Mutually Supportive Development

  23. Roles and responsibilities in STEP

  24. Summary of STEP

  25. Chapter 2

  26. Risk analysis • No way to guarantee a software system will be perfect • Small number of projects meet criteria for success • Most of us realize its impossible to test everything in even the most trivial of systems • Shortage of trained testers doesn’t help

  27. CSE 7314 Software Testing and Reliability Robert Oshana End of Lecture oshana@airmail.net

  28. CSE 7314 Software Testing and Reliability Robert Oshana Lecture #3 oshana@airmail.net

  29. Chapter 2

  30. Risk analysis • No way to guarantee a software system will be perfect • Small number of projects meet criteria for success • Most of us realize its impossible to test everything in even the most trivial of systems • Shortage of trained testers doesn’t help

  31. Domain of all possible test cases

  32. What is risk? • “the chance of injury, damage, or loss; a dangerous chance; a hazard” • Everyone subconsciously performs risk analysis hundred of times a day • Risk management; risk analysis, avoidance, and control • IEEE standard defines a section for “Risk and contingencies”

  33. Risk analysis activities

  34. Software risk analysis • Purpose is to determine what to test, the testing priority, and the depth of testing • Also determining what not to test • Should be done by a team of experts from various groups in the organization

  35. Software risk analysis • Done as early as possible • First cut after high level requirements • Results of the analysis should be reviewed periodically because things change

  36. Software riskanalysisprocess

  37. Process • 1. Form a brainstorming team to increase the number of ideas • 2. Reduce the list to a workable size • 3. Compile a list of features • 4. Assign an indicator of relative likelihood of failure

  38. Likelihood of failure

  39. 4. Determine the impact

  40. 5. Assign numerical values • H, M, L • Is your system safety-critical? • Add together likelihood of failure and impact of failure

  41. CSE 7314 Software Testing and Reliability Robert Oshana End of Lecture oshana@airmail.net

  42. CSE 7314 Software Testing and Reliability Robert Oshana Lecture #4 oshana@airmail.net

  43. Chapter 2

  44. 4. Determine the impact

  45. 5. Assign numerical values • H (3), M (2), L (1) • Is your system safety-critical? • Add together likelihood of failure and impact of failure

  46. Risk priority

  47. Summed priorities

  48. 7. Review/modify the values • Modify based on additional information from analysis • Likelihood-of-failure-indicators • Team history • Complexity • Usability • New technology • Defect history

  49. 8. Prioritize the features

More Related