Welcome

1. Welcome IPMA and SolutionsIQ Professional Event Testing, Testing, 1…2…3… Improving software quality -- one bug at a time

2. Agenda • Building the Test Framework • Jan McCollum, SolutionsIQ • Break • Practical Panel Discussion • Cheryl Hainje – AFRS Product Manager, OFM • Dotti Lane – QA Project Manager, OFM • Tim Vessey – POS Project Manager, LCB • Stein Wang – Quality Assurance Lead, SolutionsIQ • Break • Testing Templates & Checklists

3. SolutionsIQ Overview • SolutionsIQ is a full-spectrum IT services company • 25 years of technology services and solutions • 400+ consultants • Corporate headquarters (Bellevue, WA) • Professional Services (Bellevue, WA) • Oregon Branch Office (Lake Oswego, OR) • 8+ years of serving the State of WA • DOC, AOC, LCB, DNR, DOL, LNI, & DSHS

4. SolutionsIQ Expertise • Professional Services Division • Consulting and Analytical Solutions • Project management • Assessments and feasibility studies • Design and architecture roadmaps • Development and Test Solutions • Full life cycle development projects • Custom application development • EAI, portals, and business intelligence • Quality assurance and testing solutions

5. Building the Testing Framework Jan McCollum Manager, Quality Assurance and Testing Solutions

6. Setting Goals • Knowing WHAT you want is as important as knowing how to get it • Defining the vision • Defining the timeline • Gaining acceptance and buy in

7. Defining the Vision • To define the vision look at what came before • What went well • What went badly • What now • Where do you want to go

8. Testing vs. Quality Assurance • Testing is about finding bugs • Quality Assurance is about preventing them!

9. Quality Assurance • Takes time • Is about the overall effort – including development • Methodologies can be very formal

10. QA Applied to Testing • Quality assurance principals applied to the testing effort will produce higher quality work

11. Establishing a Timeline • The 6 month / 1 year / 3 year plan • Implement processes and strategies that give the best return on investment

12. Quality Testing Roadmap • After the goals and objectives are complete, make them real by publishing the quality testing roadmap

13. Quality Testing Roadmap • Roadmap should include… • Test team structure • Communications plans • Test processes • Test procedures

14. Quality Testing Roadmap • Test scope • Test dependencies and impacts • Automation transition plan • Test deliverables

15. Gaining Acceptance and Buy In • Development • Business management • Project management • IT management • Customer/product support

16. Making it Happen! • Organizational structure • Qualified candidates • Roles and responsibilities

17. Test Planning • The master test plan: a one-stop shopping guide for your project • Contents • Contributing documents • Sign-off procedures

18. Test Planning • Test matrix and test suites • Detailed test steps • Pass/Fail results • Tester who performed tests

19. Test Planning • Test case design – what is a good test case? • Accurate – tests what it’s designed to test • Repeatable, reusable – has a life after this release • Economical – no unnecessary steps

20. Test Planning • Test case design • Traceable to a requirement • Appropriate for test environment, testers • Self-standing has enough information for anyone to run

21. Test Planning • Test case design: How to make good test cases better • Setup, environment, data • Steps, actions and expected results • Use active voice in expected results • System displays this, does that • Simple, conversational language

22. Test Planning • Test case design: Why work to improve test cases? • Productivity – less time to write and maintain cases • Testability – less time to execute them • Scheduling – better reliability in estimates

23. Defect (Bug) Management • Deciding upon a tool • Easy of configuration • Ability to add/change fields • Reporting capabilities • Integrated solution

24. Defect (Bug) Management • The bug lifecycle • Who can create bugs • Who can assign bugs • Who can close bugs

25. Defect (Bug) Management • The bug triage meeting • Purpose and who should go • Reporting • Determining a trend • Bug metrics • Number of bugs found • Bugs found in production vs. test cycle

26. Moving On • Improving the process: Requirements traceability • Test cases for each requirement • Requirements matrix • Tracing requirements to defects

27. Moving On • Improving the process: Risk-based testing • You can’t test everything so test what is important • The risk list and how to use it to drive test strategy

28. Broadening Your Scope • Build verification testing • Also called smoke or acceptance tests • Is a subset of the major functional areas • Integration testing • Testing the entire system

29. Broadening Your Scope • Compatibility testing • How application works with other apps • Configuration testing • Testing on different configurations • Setup testing • Testing the installation • Regression testing • Verify if bug fixes are successful

30. Broadening Your Scope • Black box testing • White box testing • Grey box testing

31. Improving Quality • Testing metrics – measure your success • Bug tracking metrics • Number found • Number found per component • Daily bug find rate

32. Improving Quality • Test case effectiveness • Metric: Test case effectiveness; test case effectiveness = bugs found in test/total found * 100 • Test coverage • Metric: Test coverage (absolute) = tests conducted/total tests * 100

33. Improving Quality • Test team performance • Metric: Test process effectiveness: test process effectiveness = bugs fixed/bugs found * 100 • Metric: Planned days vs. actual days in test

34. Improving Quality • QA and test involvement early! • Design reviews • Why testers should attend • Develop and use checklists • Project closeout meetings • You should have them

35. Questions? • For additonal information, email • JMcCollum@SolutionsIQ.com