Enterprise transformation from legacy QA to DevOps

1. Enterprise transformation from legacy QA to DevOps The DevOps market is predicted to experience a growth rate of over 20% through this year. By 2025, it is projected to grow by $12.4 billion at a CAGR of 24.9%. This impending market explosion can be attributed to the benefits that organizations are reaping after implementing DevOps. With 27% of their resources and time dedicated to testing, companies that adopted DevOps methodologies achieved 440x shorter time from committing to release, 46x more frequent code deployment, and 96x faster time to resolve (TTR). While these benefits of DevOps adoption seem lucrative, organizations report several challenges related to people, processes, technology, and information. A decade ago, the failure/challenge rates were as high as 60-70%. The overall project & portfolio management consulting & vendor spend is increasing against the increasing failure rates. McKinsey reported that 70% of all change initiatives fail. The top 10 reasons for failure are: 1.Incomplete requirements 2.Lack of user involvement 3.Lack of resources 4.Unrealistic expectations 5.Lack of executive support 6.Changing requirements 7.Lack of planning 8.Didn’t need it any longer 9.Lack of IT management 10.Technology illiteracy 50% of the time, DevOps implementation is challenged by ‘people issues’. The secret to overcoming these hurdles lies in the resolution of these issues and changing the behavior. Behemoths like Amazon, Etsy, Facebook, Google, Netflix, and WalmartLabs are leading the race because they know the secret. Etsy deploys to its production servers 50 times a day while Facebook releases 100 million lines of code daily. Google runs over 500 million tests per day inside its systems and Amazon deploys code every 11.7 seconds, on average. The unification of software development, testing, and deployment activities enables enterprises to improve and evolve software products rapidly and efficiently. Scot Raymond Hanley, Head of QA at Synovus, spoke at our recent Meetup in Atlanta about transforming from legacy QA to DevOps in the enterprises. This blog summarizes the key takeaways from his in-depth presentation. Let’s dig in:

2. Evolution of behavior leading the evolution of quality The annual release frequency in Waterfall is reduced to quarterly/monthly releases with Agile. With DevOps, the release frequency has shortened to weeks, days, and hours. The segregated, siloed Dev, Ops, and Test teams work collaboratively, together, simultaneously under the DevOps framework. From long, phase-wise testing at the end of a long development cycle, we have moved on to in-sprint testing strategies and unattended automation aligned with build frequency in the CI/CD pipeline. DevOps methodology allows organizations to focus on in-sprint end-to-end testing, non-functional engineering, continuous integration & testing, test data engineering, build readiness for operation and release, UX engineering & assurance, & automated test enabling functions such as test data & environment engineering. The transition, rather, evolution from Waterfall to Agile & DevOps is not only technical –it is behavioral. It is about instilling end-to-end visibility across the SDLC. It is about exposing a failure and making it transparent to learn from it and avoid it in the future. It is about emphasizing Quality Assurance to lower risks, allow incremental changes, and conquer change resistance. Transitioning from Quality Assurance to Quality Engineering Cloud computing & serverless architectures are in vogue. Organizations are formulating container-first strategies. They are adopting DevSecOps and dual-shift quality automation approach for making quality inherent in the SDLC. This shift from traditional to built-in quality approach is mainly because of the rising cost of defects in production. Extreme automation with integrated quality engineering techniques is becoming a cornerstone of daily deployments. A paradigm shift is being observed towards BDD with TDD delivery practices while the role of quality is getting extended to Ops and post-production. Navigating your DevOps implementation journey For a successful implementation, organizations need to take an all-rounded DevOps approach that includes automation, data engineering, non-functional engineering, continuous monitoring, governance, and infrastructure automation. 1. Automation: Unit, services, and UI automation •

3. CI/CD pipeline integration • 2.Data engineering: Centralized test data management Data creation from production Synthetic data creation • • • 3. Continuous monitoring: Infra & App monitoring Threshold enablement Root cause analysis Auto alert configuration • • • • 4. Non-functional engineering: Performance engineering Proactive performance management DevSecOps – security assurance program • • • 5. Governance: Metrics and KPI monitoring – Dev, Test, and Ops • 6. Infrastructure automation: Automated test environments management – provisioning, troubleshoot, and self- healing • Steps involved in the DevOps implementation process For DevOps to be successful, the Dev, Ops, and Test teams must work in complete tandem with one another. The key steps involved in the DevOps implementation are: 1.Assessment: Before starting the implementation process, organizations need to take a step back, analyze, assess, and understand their culture, governance policies, and team dynamics. This assessment will help them chart an effective change management plan.

4. 2.Align with Agile methodology: DevOps being an extension of Agile mandates alignment of the software development practices with the Agile methodology. Quality Engineering techniques such as container- first strategies, dual-shift quality automation, DevSecOps, pipeline automation, and service-less architecture should be implemented. It is essential that in-sprint testing is facilitated, and the acceptance criteria are defined. 3.Tools orchestration: This step involves the identification and configuration of tools that will fit in with the organizational objectives and help fulfill the goals intended to achieve through DevOps implementation. These tools should then be plugged in with the CI engine. 4.Continuous testing: To facilitate continuous testing and dual-shift automation, extreme automation must be performed at sprint and release levels. By automating the regression tests and analyzing the impact, it becomes possible to move ahead without breaking anything. Non-functional engineering should also be taken care of at this level. 5.Enable continuous delivery: To enable continuous delivery, build and release should be automated. Facilitation of continuous integration and environment provisioning is done at this stage. 6.Ensure continuous feedback: An automated feedback loop should be initiated, which means that the process should be continuously monitored for any defects. With successful DevOps implementation, organizations can improve their deployment frequency and application quality, stabilize their operating environments, and achieve faster recovery times. In the DevOps implementation journey, metrics are the guide that build traction and show the way. It is critical that organizations establish the following metrics : 2. Governance: 1. Measure: Total cost of quality (actual vs deferred) Culture Effectiveness Delivery efficiency SLA/OLA adherence End-user satisfaction • Velocity Quality Security Performance Business value User experience • • • • • • • • • • •

5. 4. Build: 3. Plan: Unit test pass percentage Static analysis results Code coverage Build success/failure rate Defect status by severity Build lead time Technical debt Number of pull requests • Requirements ambiguity index Requirements stability index Schedule variance Number of features planned Estimated story points Team velocity Sprint progress • • • • • • • • • • • • • • 5. Test: 6. Deploy & operate: Percentage of requirements tested Percentage of automation coverage Test case effectiveness Total critical functional defects Defect arrival vs. closure Defect removal efficiency Vulnerability index Mean time to detect/fix • Deployment success rate Number of features delivered Number of roll-backs Change failure rate and volume Mean time to recovery Application availability Percentage of change in user volume Customer ticket volume • • • • • • • • • • • • • • • To sum up The latest State of DevOpssurvey report talked about ‘Stage 0’ in the five stages of DevOps. This stage involves building the foundation, which is of the essence in ascertaining the success of DevOps implementation in any organization. Implementing DevOps testing practices helps executives transform the organizational culture, bring about greater collaboration across teams, inculcate dual-shift practices, create an environment of growth for Software Development Engineer in Test (SDETs) – all of which are necessary to provide DevOps solutions. At Cigniti, we standardize efforts and ensure accelerated time to market with DevOps testing solutions. We also focus on delivering improved deployment quality with greater operational efficiency. Our DevOps testing specialists enable your application testing efforts with their rich

6. experience in Continuous Integration (CI) testing and can configure and execute popular CI tools helping your DevOps teams. With our DevOps testing services, you can achieve benefits of automated and continuous Quality monitoring, automated provisioning of virtualized test environments, standardized processes, templates, guidelines & checklists for DevOps, integration build, deployment, end-to-end test automation, & reporting, and integration of our DevOps solutions and engineers with your DevOps teams.