Effective Strategies for Managing Large Global Test Programs Through Automation

1. Managing Large Global Test Programs Through Automation of Automation K A CHANDRAKANTH SENIOR MANAGER - SOFTWARE TESTING PRACTICE PricewaterhouseCoopers Service Delivery Center (Bangalore) India Private Limited

2. Abstract The paper demonstrates the methodologies adopted for resolving complexities in the areas of test project management through impactful implementation of techniques like Automation of Automation in projects that deal with large independent multiple globally distributed agile scrum teams which operate out of offshore development centers (ODC) which are expected to co-exist with the onshore counterparts however at the same time are scattered across different locations within and are expected to collaborate together due to interdependencies which truly brings out the creative ways of ensuring that we deliver the best customer satisfaction by means of accomplishing the completeness of definition of done. The approach highlights the various options that one can explore when there are challenges around floating scope, physical locations with restriction and a delivery model with heavy inconsistencies which often try to destabilize the impact to the schedule and the ways through which we can successfully ensure that cost, quality and time is not compromised while we focus on time to market with high quality deliverables. Some of the other suggested improvisation methods would include adaptation to Scale Agile Framework, Lean Testing, Risk Based Approach and Few-in-a-box

3. Client Challenges • Operating from a ODC (offshore development center) through a clean room • Offshore Team does majority of technical work and Onshore Team only co-ordinates • with client • Shorter Sprint Cycles of 2 weeks including Automation in same Sprint • Using Agile Scrum But….. • Lack of consistent test strategy - inadequate unit test, lack of automation coverage, lack of usability testing and undefined quality metrics. This resulted in low code quality impacting user experience and loss of productivity • Automation of large scale features and maintaining of around 3000+ test scripts • Frequent change in scope • External dependencies on third party for test data • Unstable Test Environment • Different Agile Teams using different methodologies without reuse

4. Model Adopted

5. Grooming • Plan a Sprint Grooming Session ahead of Sprint Planning • Identify the User Stories which have dependencies • Plan on resolving the dependencies by working with multiple stakeholders • Set timelines to resolve dependencies • Park the User Stories back into the Product Backlog if the dependencies are time consuming to resolve and if they have greater complexities • Identify the User Stories that can be taken forward without major dependencies • Ensure all stakeholders are in sync with the planned User Stories to be taken forward for implementation in the Sprint Grooming Sprint Planning Development Testing Daily Stand Up Demo Sprint Retrospective

6. One For One Approach • Each Tester to be provided with one user story to manage for end to end testing • The Tester has cross functional knowledge and someone who can do both functional and automation testing • The Tester first starts preparing the functional test cases, followed by development of automated scripts for the same test cases. They would then perform functional test execution and finally the automation test execution • This ensure that a particular user story in a sprint is assigned to each tester and they complete the testing for the same in the same sprint to provide end to end delivery Tester 1 Tester 2 User Story - 1 User Story - 2 Functional Testing Automation Testing Functional Testing Automation Testing

7. Risk Based Testing • Focus on the critical testing required and consider the features with highest risk factor as a priority to test first. • Reduce the volume of testing but ensure quality is maintained and that the schedule is adhered with. • Categorize the modules based on their risk factor and criticality and the testing is then applied to the modules which are of highest risk and that are most likely to be accessed by the end users. • Assign business value for the tests that are critical and the ones with highest business value to be considered on priority for testing • Lower risk modules are assigned lesser priority and they are considered for testing based on the schedule availability

8. Two In a Box • Consider a Developer and a Tester and pair them • Put them in a box as a team • Both the individuals need to possess cross functional knowledge however a tester is not always expected to possess development knowledge however a developer can still contribute towards testing • Between the two, assign a particular user story for which they are responsible for development, testing and meeting the DoD within the given sprint • By end of the sprint, they are required to successfully deliver the user story with completeness through a demo • Between them they can interchange their roles as required to deliver the goal • The testing would include both functional and test automation

9. Scaled Agile Framework (SAFe) • Consider Lean-Agile implementation to build a Continuous Delivery Pipeline for improving the user experience and accelerating time-to-market. • Evaluate the value streams and map them to process, teams and systems • Identify the required SAFe configuration that can be adopted for transformation • Launch one agile release train (ART) to begin with. Come up with a program increment plan. Plan Do Check Act • Evaluate the release through the system demo. Retrospect the accomplishment and adjust the strategy based on the continuous improvement required.

10. Virtualization • Often the live environment would not be available on time to perform both development and testing • The environment availability is dependent on a third party vendor which manages the infrastructure for the client • Environment goes down more often than expected thus causing hindrance to deliverables and schedule • Creating a service virtualization setup through tools like CA-LISA helps in building a virtual setup that can simulate the environment required to perform actual work • This method eliminates integration dependencies during development by allowing teams to substitute working simulations for designs. This gives teams more continuity in their development process — a key enabler for scaling Agile

11. Benefits and Advantages

12. References & Appendix • Scaled Agile Framework (SAFe) • http://www.scaledagileframework.com/# • Risk Based Testing • https://www.slideshare.net/QASymphony/using-jira-for-risk-based-testing-qasymphony-webinar • Service Virtualization • https://en.wikipedia.org/wiki/Service_virtualization • Two in a Box • https://www.slideshare.net/Ademita1/hcm-overview-deck-1

13. Author Biography • K A Chandrakanth is a hands on Techno-Senior Manager with PwC SDC Bangalore. He has wide exposure to product testing, services testing and IT consulting experience. He has around 17 years of core testing skillset and has extensively worked on functional testing, test automation, performance testing, security testing and web services testing areas. He has excelled in test management activities and has played multi dimensional roles in project management, program management, software testing, quality consulting and quality assurance. He has contributed in setting up large Testing Center of Excellence groups to focus on Kaizen initiatives and has worked for key global clients in defining the strategy and approach for managing complex applications. He has wide knowledge on testing methodologies including Agile and has played a variety of roles like product owner. scrum master etc.. for leading global clients. He has authored several papers and regularly participates in global testing conferences. He is a certified scrum master and also has completed his certification in scaled agile framework (SAFe 4.0). He has played client facing role and managed the transformation of large legacy programs into the newer technological advances through effective implementation of cutting edge technologies

14. Thank You!!!