Michael Burnside Twitter: @ SQAEvangelist Blog: http://sqaevangelist.blogspot.com / Website: http:// sqaevangelist.com

1. Effective Quality Assurance using agile software development process – The Quality-Centric approach to developing the best software possible Michael Burnside Twitter: @SQAEvangelist Blog: http://sqaevangelist.blogspot.com/ Website: http://sqaevangelist.com/ Software Quality Assurance, Quality Engineering, and Web and Mobile Test Automation Expert 02 May 2014

2. Effective Quality Assurance using Agile Software Development Process Table of Contents: • Prerequisites • Quality Statements • Forward • Example of devices used every day that use and run software programs for successful operation • Harsh Realities of QA using Agile • Necessary Stakeholders • Process Goals • Typical QA Tasks • Sprint Milestones • Milestone Timelines • References

3. Effective Quality Assurance using Agile Software Development Process Prerequisites: • The Reader has basic knowledge and familiarity of the Agile process and how it is used in regards to software development. • The Reader has basic knowledge of Quality Assurance tasks and activities.

4. Effective Quality Assurance using Agile Software Development Process Quality Statements • Every single person at your company is responsible for product quality. • You cannot “test” quality into software. In other words, merely testing software has no appreciable impact on software quality. • The testing assets are used to provide a more meaningful and comprehensive insight into likely user experience and product capabilities and limitations. • Adhere to Sprint milestones and allow the process to make the decisions for your team so that each member can focus maximum energy on their contribution to product. • Always remember that everyone that uses your products wants it to be awesome.

5. Effective Quality Assurance using Agile Software Development Process Forward: The use of software programs in society today is so ubiquitous, transparent, and executed on such a massive scale that most people in almost every non 3rd-world society use devices that run software that requires successful execution and is now entirely crucial to the advancement of the desires and wishes of people using the software and to all societies, in general. In 2014, most of our normal life activities and productivity are entirely dependent on working software for the next level of human activities (Assuming that food, water, and sleep are the basic human needs and activities).

6. Effective Quality Assurance using Agile Software Development Process Example of devices used every day that use and run software programs that human beings depend on for successful operation Examples of devices that can be generally regarded as a used device on an hourly/daily basis that run software programs: • Watches • Personal and Work Computer (laptop or desktop) • Mobile device such as a tablet computer, such as iPad, Android, Windows CE (tablet) or “smart” phone (iPhone, Android, Windows CE). • Microwave oven for cooking/reheating food. • Regular oven for cooking food. • Databases / data storage. This data is used for a variety of reasons. • Nuclear energy producing power plants and military-controlled devices of mass destruction. • Cable TV set top boxes • Digital HD, Non-CRT Televisions

7. Effective Quality Assurance using Agile Software Development Process Harsh Realities of QA using Agile: • Performing QA tasks within an agile Sprint is much more difficult than any other type of software development methodology previously used. In other words: QA is really, really hard when the team is using Agile. • Missing milestones within the sprint puts exponential difficulty and stress on the QA team if risk is not mitigated or dealt with properly.

8. Effective Quality Assurance using Agile Software Development Process Necessary Stakeholders: At the very minimum, your team needs to include the following stakeholders WITH NO EXCEPTIONS. • Product Manager • Software Development team • Quality Assurance team Optional (but desired): • ScrumMaster to handle the Agile process aspects of the project, or, a Project Manager with an Agile focus. That’s it! If one of these people does not exist in your team right now, THEN GO AND HIRE THEM!

9. Effective Quality Assurance using Agile Software Development Process Process Goals • To deliver to your customers the highest quality of software possible. • To make your customers love using your software and become dependent on it for good enhancement of daily life or job experience. • Become dependent on it for their own job/personal life success and happiness. • To maximize efficiency and purpose of each member in the Agile workgroup. • To eliminate any possible ambiguity in product requirements, acceptance criteria per test case, and product deliverable dates.

10. Effective Quality Assurance using Agile Software Development Process Process Goals (Continued) 6. If you fail, you fail as a team, not because of an individual or group of individuals on the team. For example, if a critical/show stopper bug appears in a production environment, the first question that you should NOT be asking is: how did the QA Team miss that one? Who on the QA team was responsible for this? The QA team is entirely dependent on the software being developed by the software development team and pushed into the QA environment. The QA team cannot possibly test every single test case manually and whatever contribution of testing coming from the test automation.

11. Effective Quality Assurance using Agile Software Development Process Process Goals (Continued) The correct question should be: How did the bug get into production and let’s find the fail mode and correct it immediately and for the future; but how did we AS A TEAM miss this one and what can be improved IN OUR SOFTWARE DEVELOPMENT PROCESS so that this does not happen again? Very rarely, a bug enters the production environment because of the incompetence or failure of a QA person. More often than not, it is because not enough thought went into developing, testing, and understanding the impact of the change in an environment where it may not be testable (developer: local environment; QA person: QA testing environment; Release Engineer: Staging environment). The most common cause is inadequate and unprepared test environment or intense product schedule release pressure.

12. Effective Quality Assurance using Agile Software Development Process Typical QA Tasks What are some of the tasks that need to be done, regardless of software development process used, before the next release of software? • Decide which stories (new features) need testing done to be considered complete. • Creation of test cases for new stories that describe: description, steps, and acceptance criteria. A collaborative effort involving development team story implementer, QA resource(s) to test, and Product Manager, and any other stakeholder involved. • Review and Approval of test cases from all stakeholders. • Documentation of test cases using a test case management system. • Prioritize which test cases are “sanity”, “smoke”, and which ones to automate. • Automate selected test cases to reduce test automation debt (regression testing automation) • Manual testing of new features (and if time permits, automate them). • Log bugs, improvement requests. • Close Resolved bugs • Identify any and all activities that are attributing to product risk and alert the team about the risk.

13. Effective Quality Assurance using Agile Software Development Process Sprint Milestones: The Quality-Centric software development process using Agile. Milestones (Part 1): • Test cases collaboratively developed per story by the following stakeholders: Product Manager, QA, and Development team members. This includes: purpose of test case (what are we testing?), intricately and well-defined steps to execute the steps of the test case, and lastly (and most importantly) Acceptance Criteria (what result data or user experience will validate that the test passed?). • Once a complete list of test cases are created for each story in the iteration, an email is sent from the QA person to the developer and Product Manager asking for approval of all identified test cases, steps, and acceptance criteria. At this point, the Product Manager and any software developers have a chance to review the test cases purpose, steps, and acceptance criteria. Feedback may be given to change any part of the 3 identified data set for the test case, but eventually, THE SET OF TEST CASES (WHICH IS ULTIMATELY A TEST PLAN) MUST BE agreed to via an email from the Product Manager and Developers to the QA person. This action gives a SHARED RESPONSIBILITY if there is a problem during testing of the software or a critical bug reaching into the production environment.

14. Effective Quality Assurance using Agile Software Development Process The Quality-Centric software development process (Continued). Milestones (Part 2): • As the software developer starts to write the software to implement the features in the story, a QA person may or may not (but usually can) start to implement the execution of test cases. When the software is ready to be tested, the developer sends an email to the QA person that indicates the software is checked in and the feature is ready to be tested. If not, in some cases mock testing implementation can be created (use of JMock or other interfaces, etc) by the QA person to mock test the software. • Development on each story feature needs to be completed by at least 30% of time before the end of the iteration. If not, most likely not enough time will be available to test the story feature (and also test regression) and it is recommended to move the story completion to next sprint, and back out or comment out the code to avoid product risk. The team must determine how any unfinished story (development or QA task) can potentially cause risk to a released product.

15. Effective Quality Assurance using Agile Software Development Process The Quality-Centric software development process (Continued). Milestones (Part 3): • The QA manual and automated tester implement the test cases. Sometimes there are companies that employ a separate/disconnected manual QA (subject matter expert – SME) and automated test implementation team per product and then a QA test automation team to write automated tests. This disconnect is a risk to the company’s test validations. It is incredibly important that if the manual and automated test case implementer are not the same person, that they need to be in complete sync to verify the test of the story is working. Most likely, the most competent subject matter expert on the QA side is the manual tester, not the automated tester (if on a separate team that provides basic automation work for the company). If so, the manual QA tester MUST VALIDATE the acceptance criteria of the feature whether it be of a manual or automated test implementation and be a constant future validator or passing tests. The manual QA tester is responsible for running the test cases and it may be easy for them to do that through a Continuous integration (CI) mechanism. But they are responsible for getting updated on the results of each test run and perform the appropriate actions if there is a test error. • Agile stories are NOT COMPLETE unless all tasks have been finished, and that includes any QA tasks (manual or automated). QA team should validate with development and Product Management that any test cases are indeed working and any regression errors are dealt with.

16. Effective Quality Assurance using Agile Software Development Process The Quality-Centric software development process (Continued). Milestones (Part 4): • The QA team and Product Management team are the stakeholders who determine when a product is of adequate quality and ready to be released to the end user. • Automation test debt can occur in times where QA is waiting for new features to be ready for testing. • The software development team NEVER decides when a product is ready to be released.

17. Effective Quality Assurance using Agile Software Development Process Milestone Timelines (10 working day sprint) Example • Days 0 – 3 (or 0 – 30% of sprint time) • Test cases identified for each story • Test case acceptance criteria and approval by all stakeholders • Catch up on automating regression testing (reduce test automation debt) • Days 3 -7 (or 30 – 70% of sprint time) • Catch up on automating regression testing • Test any story ready for QA • Day 7 • Code freeze deadline; all unfinished stories from development team get automatically pushed to next sprint.

18. Effective Quality Assurance using Agile Software Development Process Milestone Timelines (10 working day sprint) Example (Part 2) • Days 7 – 10 (or last 70 – 100% of the sprint) • New feature testing manually or by any previously ready test automation • Regression testing of all required test cases to be executed • Day 10 - General Availability • Sanity and smoke testing for production release *** Don’t miss these milestones. Doing so will increase product risk and chances for negative user experience ****

19. Effective Quality Assurance using Agile Software Development Process

20. Effective Quality Assurance using Agile Software Development Process Thanks! I sincerely hope this presentation was useful to you and that your company or organization implement the information and processes in this presentation to the fullest extent. If you have any feedback, please feel free to get in touch with me (via Twitter) at @SQAEvangelist. I would love to hear very constructive comments on how to improve this presentation and make it more effective.