The Paradigm Shift – Testing 3.0

1. The Paradigm Shift – Testing 3.0 Arun Kumar Singh

2. Agenda • Testing 1.0 • Testing 2.0 • The changing market space • The new needs • Testing 3.0 • The 3 type of supplier and the opportunity for them

3. Software fails Source – IEEE Spectrum

4. Testing 1.0 Definition : Software Testing is the process of executing a program or system with the intent of finding errors. [Myers79] Lifecycle Part of debugging process , dynamic change Techniques Processes (typically) undocumented Tending to be driven in an ad hoc uncontrolled and reactive manner by users or events. Infrastructure chaotic or unstable environment for the processes Organization one team. no differentiation. Development team did testing Success due to competence and heroics of the people in the organization and not on the use of proven processes.

5. Testing 1.0 T Market conditions • Small & nascent IT • Small pockets of manual activity automation • Startup / innovation/ research environment • Small developments • Technology use/excellence drive C Q Outcome products and services that work; however, they frequently exceed the budget and schedule of their projects

6. Testing 2.0 Definition : Any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results. [Hetzel88] Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test.[1] Software testing can be stated as the process of validating and verifying that a software program/application/product: • meets the requirements that guided its design and development; • works as expected; and • can be implemented with the same characteristics. 1. ^Exploratory Testing, Cem Kaner, Florida Institute of Technology, Quality Assurance Institute Worldwide Annual Software Testing Conference, Orlando, FL, November 2006 2. Wikipedia

7. Testing 2.0 Techniques testing processes are well characterized and understood, part of SDLC or PLC described in standards, procedures, tools, and methods. improved over time Using precise measurements, management can effectively control the software development effort. ……. identify ways to adjust and adapt the process to particular projects without measurable losses of quality or deviations from specifications.Specialized tools and techniques ( statistical models ) are used Infrastructure planned & dedicated Organization well defined and separate testing organization. Exec mgmt support. testing G&Os aligned to corporate vision and mission

8. The TCoE Company XYZ Business unit A in geo A Business unit B in geo B Business unit C in geo C Benefits Layers Demand consolidation & forecasting • Standard engagement process • Reduced cycle time Demand Management • Enterprise-wide common processes • Improved maintainability SME services QA Consulting/Best Practice Exchange/domain & product expertise Core Services • Standardization of process / templates • Compliance across Geos and Business Units • Improved productivity Manual Testing Automation SBB..xxx performance Infrastructure consolidation Shared test environments • Reduced Capex and Opex

9. Cost to Quality ^ McConnell, Steve (2004). Code Complete (2nd ed.). Microsoft Press. pp. 29. ISBN0-7356-1967-0. 

10. Software quality factors Typical Software Quality Factors [Hetzel88] 

11. Testing 2.0 T Market Conditions • Complete enterprise computerization • Large projects • E2E business process software • Package software / enterprise in a box • Global enterprises • Multi channel /always-on client interface • Increased rate of refresh • Technology platform • Hardware • Business requirements • Localization with globalization • Offshore delivery models C Q

12. Testing 2.0 - Challenges Tracking/measuring and defining correct outcomes & benefits. Lack of management support (little or no visibility into the effectiveness ) Resistance from suppliers. Organization change... Limited tools to encourage the managed approach with HR metrics Defining & doing the right amount of testing Moving from technology to business process testing Domain expertise Cost to quality measures Disconnected from the end customer experience Offshoring : communications, right skill sets, management-vendor relations, global infrastructure, ever increasing expectations/innovation

13. The emerging market space – PEST analysis Economic factors • Inflation • Employment • Disposable income • Business cycles • Energy availability and cost • Others? Socio-cultural factors • Demographics • Distribution of income • Social mobility • Lifestyle changes • Consumerism • Levels of education • Others? Political / Legal factors • Monopoly / Oligopoly legislation • Environmental protection laws • Taxation policy - Tax levies and tax breaks • Employment laws - minimum wage or anti discrimination laws. • Government policy, Legislation • Voluntary codes and practices • Market regulations • Trade agreements, tariffs or restriction Technological factors • New discoveries and innovations • Speed of technology transfer • Rates of obsolescence • Internet • Information technology Rapid change in business environment

14. Analyst market outlook Overall: Discrete or standalone global testing services spending reached $9.4 billion in 2010, Growth: Worldwide growth estimated at a five-year CAGR of 15.4%. Some analyst have Geo-wise: United States has led other regions in adoption of discrete testing services, in terms of both traditional and newer cloud-based testing services models……. an increase in regional interest and adoption of these services in Canada, Latin America, Western Europe,and parts of Asia/Pacific is seen. Message for testing huge growth – plan for it. Be ready with right resources outcome driven delivery and financial models would be needed move from technology to business process Source : IDC

15. Market space – organizational metamorphosis Key Findings • CFO influence in IT is growing; 42% of IT organizations report directly to the CFO. • Organizations are willing to spend to gain competitive differentiation and improve efficiency. • CFOs cite "gaining business insight" and "improving business efficiency" as top technology opportunities • CFOs would like to reduce ERP instances, and expressed a preference for "vanilla" implementations — i.e., implementations that do not require excessive customization. • CFOs are still concerned about achieving business benefits from IT. • IT success relies more on understanding and working with the business than on technological excellence. Impact on Testing: • Communicate business benefits in financial terms to senior financial executives. Move from technology to business process testing and focus on cost-benefit analysis. • Understand CFO’s view of technology investment decisions. • Testing teams must be able to explain costs, benefits and share information/influence non-IT employees Source : Gartner

16. Analyst market outlook Trends that are key to growth in this services segment: • Test strategy consulting • Evolution and maturation of testing services portfolio • Testing complexity and optimization • Technology excellence is hygiene factor. Business process assurance should be the focus • Ready to go solutions Emerging/Future Application Testing Services Trends: • Model-driven testing • Mobile application testing services • PaaS testing • Cloud testing — many variants • Cross-cloud testing • Crowd sourcing for testing • Test data management • TaaS Source : IDC

17. WIIFM - Testing Service Providers • Testing as custodian of customer experience. • Move the testing 2.0 excellence to virtual environment ( cloud, hosted test beds ) • Mobile application service on vendor hosted environment with remote access • Cloud testing offerings • Service delivery and pricing models – offer Range and flexibility • test service ROI : monitor and provide • Product IP protection • Market and build brand presence in testing services: • Downward price pressure • Partner alliances • Public crowd sourced testing models: it will continue to gain traction in 2011 and beyond Source : IDC

18. WIIFM - Large testing service provider • Provide Testing 2.0 , your core competence to more buyers. • Experiment with testing 3.0 with your large and strongly engaged clients • For large testing deals be ready with right team, at right place at right time • Show cost flexibility • Be ready to move from 100% complete testing to business-need based testing • Domain competence will be key to differentiate, as client decision makers become non-technical • Lab(s) on hire ( especially with cloud coming) will become a need • Specialize in sub-practices in testing. • Develop Joint testing solution with product and tools vendors • Expand offerings – be E2E testing service provider • All shore delivery – round the clock execution. • To handle cost pressure – looks at class B and C cities in India eastern European countries.

19. WIIFM - Small testing service providers • There is huge room for Testing 2.0. Use on of the three strategies to penetrate market • Cost leadership • Focus ( e.g. finance, TSP etc.) • Differentiation • Implement the TCoE model within your organization. Structure your testing group accordingly • Do joint solutions ( and GTM strategy ) development with package and tools Alliance partners. This will increase market reach and penetration. • Develop client advocates • APAC / ASEAN market has huge potential. Lower margins but easy penetration wrt larger vendors • Experiment with local workforce ( visa, Indian staff cost, onsite HR policies) • Establish labs and demos. Develop process assets.

20. WIIFM - Entrepreneurs – new testing service providers • Implement TCoE aligned org structure internally • You may start with testing 2.0 offering to large firms with old IT groups & 3.0 to niche product/service vendors. • Decide on one of the three strategic choices to penetrate market • Cost leadership • Focus ( e.g. finance, TSP etc.) • Differentiation • Cloud (and disruptive technologies) will be a boon / entry point • Bunch of excellent testing practitioners will make a huge difference. Seniors are there in the country now. • Build brand • Develop solution and do joint GTM with product/tools vendors • Experiment with alliance with small firms in other countries. Cater to their off shoring dis-advantage • Participate in local testing boards by becoming their members • propose alternate financial models to compete with large vendors • Process assets and package/domain accelerators will be hygiene factors

21. Arun Kumar Singh Thank You Vice President Arun.singh@wipro.com

22. IDEAL: initiating, diagnosing, establishing, acting and learning model ( carnegie mellon university ) • A Users Guide for Software Process Improvement • This document describes a software process improvement (SPI) program model, IDEAL, which can be used to guide development of a long-range, integrated plan for initiating and managing an SPI program. The purpose of this document is to provide process improvement managers with a generic description of a sequence of recommended steps for SPI.