Making ISTQB Certification Work for You

1. Making ISTQB Certification Work for You As a test manager or an aspiring test manager

2. Where are you now? • You’ve invested in your team • You have a well-educated group • Common knowledge • Common goals • Common language • They are eager to apply their knowledge • You are eager to get a return on your investment and leverage the training into high quality products

3. But…. • But your company doesn’t appreciate testing • Your schedules are too tight • Your budgets are too strict • The code you’re testing is somewhat less than functional • You must be in the real world! • So, how do you get that return on your investment?

4. Bridging Ideal and Reality • There are 12 main steps that we need to take to leverage good practices and knowledge • These steps will provide a return where we need it – improved quality and value • It’s important to set a realistic schedule • “Realistic” depends on your environment and needs • Remember those pesky projects in progress

5. How do you make it work? • Identify the perceived value • Meet the schedules and budgets • Improve quality • Spread the knowledge • Push testing upstream • Do the static testing • Expand into white box testing • Pick, use and improve tools • Manage • Build satisfaction • Count the numbers • Show the actual value

6. 1. Identify the Perceived Value • What is the perceived value of your team? • Do you get told to just work faster, add more people from the street, work weekends…. • Does development get told that? • The less valued team will get the schedule/budget/manpower cuts • Look around, ask around, how is your team perceived? • Understanding the perception tells you what you need to change/fix

7. Fixing Perceived Value • Use the test plan to get the stakeholders to understand testing scope and goals • Engage in the risk analysis and help people understand that there is risk • Track and publish your metrics • Entry/exit criteria (don’t bend the rules) • Cost of Quality • Defect Detection Percentage • Real numbers from real projects get attention

8. 2. Meet the Schedules and Budgets • Who’s holding the software when the end dates are missed? • Who’s charging to the budget when it overruns? • Who really caused the slip and the overrun?

9. Fixing Schedules and Budgets • Track the testing schedule in days allotted, not start/end • Track the testing budget in money allotted • Let the PM manage the overall project • Track the test coverage (risk / requirements / code) and map coverage to schedule • Be ready to answer “what could you do if you had two more weeks to test”

10. 3. Improve Quality • Goal: Highest quality possible • Obstacles - define “possible” • Time, training, manpower… • Incoming quality often defines limits of outgoing quality • “Just do the best you can….” so we can blame you later… • Time, Time, Time, Time, Time

11. Fixing Quality • Incoming quality issues should be dealt with via the entry criteria (test plan) • Unit testing and static analysis are not optional • Objective metrics are used to determine “testability” • Employ those test strategies your team knows • If time is an issue, use risk-based • Track the residual risk as the time runs out

12. Fixing Quality • Break into the arsenal of testing techniques • Reduce tests via equivalence partitioning • Test the combinatorials with your advanced team • Verify code coverage; better code = shorter testing time • Apply the right techniques to the right situation • Use experience-based to leverage the team’s experience and knowledge and find gaps

13. 4. Spread the Knowledge • Time-constrained teams tend to silo • Cross-training isn’t done because there isn’t time • Faster to use the experts in the area • If someone leaves…. Uh-oh • No time to develop and advance skills

14. Fixing Knowledge Spreading • ISTQB training builds a base level of knowledge • Growing from that common base is easier than starting new • Find and create opportunities to leverage known techniques • Build an environment of learning

15. 5. Push Testing Upstream • Bad code makes for long testing schedules • “Throw it over the fence” is too common • Regardless of lifecycle model, testing has to start with the requirements • Continuous testing = continuously improved products • Testing is not “owned” by the test team • The test team may feel like salmon

16. Fixing Upstream Testing • Talk to development • Implement code coverage tools to verify unit testing • Create a plan for real integration testing • Check your entrance/exit criteria • Are they measurable? • Can you make them stick?

17. 6. Do the Static Testing • Not enough time for reviews • Reviews are not valued • The test team’s input is not valued • Not enough time for reviews • The documents to be reviewed don’t exist… • The test team is busy on other projects • Not enough time for reviews

18. Fixing Static Testing • Your team knows how to do high quality reviews • Pick the review type that makes sense • Review the requirements and designs first • Start with a small group if needed • Document the results • Move into code reviews and static analysis • Be sure to make reviews a standard part of test documentation as well

19. 7. Expand into White Box Testing • Does your DDP indicate that you are missing things? • Do you think your black box coverage is great? • Can’t do white box testing • Not enough time • Not enough skill on the team • Not enough access to the code

20. Fixing White Box Testing • Check the defects you are missing – could you have caught them? • Don’t estimate, use code coverage tools to determine your black box coverage • Start with targeted white box to address risky areas not covered by black box • Use your Technical Test Analysts for this

21. 8. Pick, Use and Improve Tools • Using sub-optimal tools is often a time waster • People get frustrated with poor or awkward tools • Data can be lost when tools don’t interface • Time spent creating reports could be better spent • Data dissemination is difficult with some tools

22. Fixing Tools • Make sure you have the best tools available using the info you learned on picking tools • Make sure you are using the tool optimally • Get rid of fields you don’t need and quit annoying people • Add the critical fields needed for COQ and DDP • Get your TTA’s to code “glueware” to make the tools work smoothly together • Create and publish reports that prove your value

23. 9. Manage • Seems obvious, right? • A lot of test managers forget to manage • Managing a team should not be fire fighting • If you are swamped, you need to get your processes under control • People need to have some of your time • Finding and creating metrics should not be a significant part of your job

24. Managing • Use the test plan to plan projects • Implement the proper control metrics to monitor and report • Follow up on the escapes and fix the issues • Automate metrics publishing (save time!) • Train your people and create opportunities • Build relationships though risk analysis and reporting

25. 10. Build Satisfaction • Is your test team the recipient of comments: • “How did you miss this?” • “Why does testing take so long?” • “The developers should just do the testing” • Test teams that cannot produce quality products are frustrated and updating resumes • Hiring and training is a significant investment

26. Fixing the Satisfaction Problem • Be realistic and let them use their knowledge • Track the metrics so you can explain what was done vs. what should have been done • Track the risk mitigation • Track the coverage • Test team’s are happiest when honest metrics are communicated • Prioritize the testing and test to the priorities • If time is insufficient, the metrics will show it • The team can’t be blamed for what they can’t do • Deflecting blame is demoralizing; stop the blame

27. 11. Count the Numbers • Do you have the metrics you need to explain the status of a project? • Check the defect tracking system – is it getting accurate data? • Reports and graphs that are too complicated will not be read • No one will dig for the real numbers – it’s easier to assume they don’t exist

28. Fixing the Metrics • Plan to track and use the tracking to plan • Pre-emptive metrics stop the blame game • Your people know why metrics are important • Make sure your tools are tracking the important data • For DDP, need to know what was found after releaseas well as what was found in all testing • For COQ, need to know phase introduced and phase found • Make reports/dashboards available and readable

29. 12. Show the ACTUAL Value • The value of testing has to be shown in time and cost savings • Return on investment is required • It’s easy for others to see testing as an expense with little return • Quality doesn’t just happen, it has to be planned, implemented and maintained

30. Fixing the Visible Value • Use those metrics – Real numbers for real projects show actual value • Risk mitigation • Coverage • Defects found (DDP) • Cost of quality (COQ) • People should feel good about testing, but should also understand residual risk • Quality must be embraced by all stakeholders

31. Long Term • Continuing education keeps testers current, interested and is a job benefit • Good testers always want to improve – look to Advanced and Expert levels • Leverage your team knowledge • Keep your good practices in place – always! • Make sure your processes and tools work for you • Make sure your reporting is consistent and automated

32. Goals for the Certified Team • Improving • Learning • Building • Producing Quality • Performing Consistently • Finding Satisfaction • Maintaining Pride