1 / 38

Product risk analysis in practice

Product risk analysis in practice. Kees Blokland. NO RISK NO TEST. Why risk based testing?. Exhaustive testing. Because we cannot test all possible ways through the software. Why risk based testing?. Testers must make choices. Test strategy. What to test and what n ot to test

ima-hurst
Download Presentation

Product risk analysis in practice

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Product risk analysis in practice Kees Blokland NO RISK NO TEST

  2. Why risk based testing?

  3. Exhaustive testing Because we cannot test all possible ways through the software Why risk based testing?

  4. Testers must make choices Test strategy What to test and what notto test What to test thoroughly, what to test superficially What to test first, what to test later What to test once, what to test each software build What to test by senior testers, what by junior testers What to test scripted, what to test exploratory, … just to name a few

  5. Testers must make choices We base these choices on the level of risk What to test and what notto test What to test thoroughly, what to test superficially What to test first, what to test later What to test once, what to test each software build What to test by senior testers, what by junior testers What to test scripted, what to test exploratory, … just to name a few

  6. What is risk?

  7. Risk defined WIKIPEDIA The probability or threat of quantifiable damage, injury, liability, loss, or any other negative occurrence that is caused by external or internal vulnerabilities, and that may be avoided through pre-emptive action.

  8. How to determine the level of risk?

  9. V = I x R Voltage Current Resistance Electrical engineering: Ohm’s law

  10. R = L x I Risk Likelihood Impact Testing: Test’s law on software failures NO RISK NO TEST

  11. BMW needs to call back 10.000 cars in China due to a software defect Doors open spontaneously Gear switches from parked state and car starts rolling by itself from the hill source: computable.nl

  12. Huge traffic jam due to system failure Huge traffic jams due to failing information boards source: computable.nl

  13. Information still usable…

  14. Basic process of Product Risk Analysis

  15. Basic process of Product Risk Analysis • Preparation • Risk identification • Risk classification

  16. 1. Preparation • Divide system in components payment • New? • Changed? • Unchanged? preview login download upload templates shopping card

  17. 1. Preparation • Divide system in components • New / changed / unchanged • Identify attributes • Functionality / performance / security / usability / …

  18. 1. Preparation example

  19. 1. Preparation • Divide system in components • New / changed / unchanged components • Identify attributes • Functionality / performance / security / usability / … • Compose product risk assessment team • Testers / developers / users / designers / business analysts / product owners / …

  20. 2. Risk identification • Identify product risks in lines of text like: “This screen does not show the right values, so that all customers get the wrong invoice.” “The process is not ready in time, which means …” “A visitor has access to classified information, so …” • No risk found: empty cell • Combine comparable risks • Component or attribute missing: add it! • No consensus?  split risks in separate items

  21. 2. Risk identification example

  22. 3. Risk classification • Determine likelihood of failure Frequency of use Inexperienced developers New tools High time pressure Complex functions Many changes Many interfaces Error prone functions Input needed from developers/testers L M H

  23. 3. Risk classification • Determine likelihood of failure • Determine impact Losing income Losing customers Societal Damage Claims by users Repair costs Damage to company image Compensation effort and cost Development and test cost Clients / Users / Business perspective L M H

  24. 3. Risk classification example • Determine likelihood of failure • Determine impact • Look up risk class

  25. 3. Risk classification example

  26. Managing a risk analysis meeting • Limit meeting duration (2-3 hours) • Start with most interesting topics • Keep to the purpose of the meeting: product risks • project risks  action item • Incomplete specifications  action item “Performance is a risk, but we miss acceptance criteria” • Small releases: meeting not always required Always communicate!

  27. How to map risks on tests?

  28. Do you remember this slide?

  29. How to map risks on tests? Low risk Medium risk Component Test + System Test High risk + System Integr. T. Test levels + Acceptance Test Performance Special test Security Implicit All Usability Superficial Specific Techniques Junior Thorough Medior Testers Formal Senior Model based Multiple Free format Session based Exploratory testing Session based by experts

  30. How to map risks on tests? - continued Tested once Manual regression test Test rounds Multiple Automated regr. test Each - Experts Informal Review Specific Formal Focus Junior - Medior Testers Code review Pairing Senior Pairing with testers

  31. Do you remember this slide?

  32. What to test first and what later? First execute tests on high risk components / features / aspects Most serious defects Then execute tests on medium risk components / features / aspects Time Less serious defects Low severity defects In the end execute tests on low risk components / features / aspects Project deadline Low severity defects undiscovered

  33. Remarks on • PRA in agile / scrum • PRA and outsourcing • PRA and large organizations

  34. PRA in agile / scrum • Order of development (including test) is based on • Business value • Risk • Risks are also translated into acceptance criteria Business epic/feature PRA at epic/feature level Acceptance criteria User story PRA in sprint Test cases Acceptance criteria

  35. PRA and outsourcing • Does supplier know impact of failing software???? • Does supplier know where to focus testing??? • Risk profile = basis for project estimation 50% higher risk Fixed price? 35% average risk 20% lower risk % testing effort Communication!

  36. PRA and large organizations • Break PRA down in multiple levels: • Prioritize features in release level PRA • More high level • More focus on business value / impact • Product Risk Analysis on each feature • More detailed level • More focus on design and build A Online payment B Book preview B Mobile view C Website stylerefresh payment preview login download upload templates shopping card

  37. Conclusion • Product Risk Analysis is a communication process • Translate risk into a clear test strategy • Make the process fit to the context No exact science Product Risk provides the key to maximum test efficiency and minimum residual risk NO RISK NO TEST!

  38. 谢谢! Thank you! kees.blokland@polteq.com

More Related