Cloud Testing Future of Software Testing Dated: 13 th November 2010 By: Somil Katiyar

1. Cloud Testing Future of Software Testing Dated: 13th November 2010 By: Somil Katiyar

2. Agenda • Understanding Industry Needs (5 mins) • Common pain points in Software Testing • Trends as per Industry Analysts • Conclusion • Cloud Testing Solution (10 mins) • What is Cloud Testing • How is Cloud Testing different from Traditional Model? • Rationale of Cloud Testing • Drivers for Cloud Testing Adoption • Case Studies • Asian Airline major adopts Testing Platform as SaaS • Cloud Based Load Testing for a Travel Company • Challenges with Cloud Testing (5 mins) • Solutions & Strategies (5 mins) • Potential Areas for Cloud Testing • Business Model • Transformation Approach • Q & A (5 mins)

3. Common Pain points in Software Testing Understanding Industry Needs Testing is Expensive Not able to test enough Multiple projects, maintenance work in parallel Cannot fast track testing due to environmental limitations Load testing is performed in Production Skill set not readily available for Open Source tools Commercial testing tools are expensive

4. Trends as per Industry Analysts Understanding Industry Needs The average enterprise IT department devotes up to 50 percent of its entire technology infrastructure to development and test, with up to 90 percent of the available test infrastructure remaining idle - IBM Research A delay in website page loading by even one second can impact conversions by 7% and customer satisfaction by 16% - Aberdeen Group Those using cloud facilities to run applications, 49.8% said they were doing so experimentally or for prototyping; 28.6% for non-critical business applications and 21.6% for business critical applications - Evans Data Survey The average online business had 8 days of downtime a year due to performance issues leading to lost revenues - Aberdeen Group

5. Trends as per Industry Analysts Understanding Industry Needs July 2010 “The Coming Upheaval In Tech Services” Budget Cuts, Rising Business Demand, And Too-High Operating Costs Mean IT Needs Help

6. Conclusion Understanding Industry Needs • In general, there is a increasing need for Testing prior to production “roll-out” • But customers are looking for cheaper alternatives • This need is further being magnified due to smaller IT budgets • Customers recognize redundancies in Testing operations • And are looking for opportunities to cut “waste” • They recognize Cloud Based Testing as a viable option • However, they are hesitant as there is a “perception of concerns” • There is a compelling economic value associated with Cloud Based Testing • Hence, organizations are adopting Cloud Based Testing by prototyping • It, probably, will continue to grow and is the future of Software Testing”

7. What is Cloud Testing? Cloud Testing Solution While there are many definitions being provided for Cloud Computing (most of them similar), we prefer to agree most with Forrester’s definition: A standardized IT capability (services, software, or infrastructure) delivered via Internet technologies in a pay-per-use, self-service way. Customer Network Test – Anytime - Anywhere VIEW REPORTS ONLINE OR BY ENCRYPTED EMAIL Service Provider Remote Operation Center People Process On-Demand Testing Portal Internet Performance Indicators: Availability Security Usability Support Tools Infrastructure 1 2 3 SCHEDULE & MANAGE TEST FROM ONLINE PORTAL TEST CONDUCTED & MANAGED AUTOMATICALLY OVER THE INTERNET

8. How is Cloud Testing different than Traditional Model? Cloud Testing Solution Source: Software On Demand’s Impact on the Software Industry: Transformation, Extinction or Rebirth?; July 2005 Source: U.S. Software On-Demand Delivery Model 2005–2009 Forecast; IDC #33493 June 2005

9. Rationale of Cloud Testing Cloud Testing Solution • Eliminate idle time due to unstable environment or unavailability of Test data • Eliminate the need to obtain specialized skill-set in testing or technology (ex. Mainframe, QTP etc.) People • Eliminate Process Management overheads • Eliminate Quality management overheads • Simplified Software Testing Process for organizations moving to Cloud based testing Process • Eliminate unutilized capacity • Remove cost for managing testing tools • Organizations can choose from variety of tools without having to invest a high upfront cost Tools • Eliminate unutilized capacity • Remove cost for managing testing infrastructure • Organizations can have multiple test environments without having to invest a high upfront cost Infrastructure

10. Drivers for Cloud Testing Adoption Cloud Testing Solution Service Providers End Users Technology & connectivity makes it possible Recurring revenue stream Single version for all customers Strong customer feedback loop Easier to trial Reach broader target market Pricing matches economic benefits Reduced initial outlay; complexity and risk Reducing IT Operations cost Easier to trial Focus on core business

11. Case Study: Asian Airline major adopts Testing Platform as SaaS Case Studies Business benefits • Strict adherence to security, reliability and availability requirements • Seamless Integration with leading testing and test management tools • Customization options to allow for innovation & differentiation • Functions & features that offer the flexibility of commercially licensed testing and test management tools Business need • Turnkey testing process lifecycle outsourcing with SLA • 24X7 availability & reliability • Reduced dependencies on internal IT • Save time on support, maintenance, and upgrade, etc • No multiple version upgrades • Simplified value-based pricing model • Estimation technique to accurately forecast demand • Training of users on the Testing Platform as SaaS • Defining SLA’s for service delivery • Integration issues with existing testing processes Key Challenges

12. Case Study: Cloud Based Load Testing for a Travel Company Case Studies Business benefits • Web portal has undergone a technology change from PSP to Java requiring Load and Stress Testing • Additional requirement for capacity assessment and recommendations • Client didn’t have a dedicated Test Environment or Load Testing Licenses • Cost for conducting Load Testing in a Traditional way was not acceptable • Replicating multiple locations in Load Test Business need • Conducted Cloud Based Load Testing using Open Source Tools • Saved over 80% of cost as compared to Traditional Model • Transitioned load test execution knowledge & Load Testing scripts to the customer for subsequent usage • Training on the open source tool • Monitoring and performance analysis issues • Estimation challenges • SLA’s Key Challenges

13. Challenges with Cloud Testing Data privacy & security Business Domain Knowledge Integration – with existing IT processes Integration – with existing Testing processes Availability Performance Reliability of Test Results Technical Support Training Organizational Cultural Impact

14. Potential areas for Cloud Testing Solution & Strategies Independent Testing Services aligned to the SDLC Software Development Lifecycle DEPLOYMENT DESIGN POST PROD. SUPP. BUILD ANALYSIS Testing Process Test Case Design Sanity Testing Test Environment Setup Test Execution User Acceptance Testing Test Planning / Strategy Static Testing Test Data Setup Automation Analysis Automation Framework Script Generation Script Execution Non-Functional Test Requirements Non-Functional Test Strategy Workload Modeling Load / Stress / Security and Usability Testing TEST MANAGEMENT / DEFECT TRACKING • Sections highlighted in green are “preferred” areas to be moved to cloud due to: • Business domain knowledge constraint • Knowledge of customer’s environment, processes, and IT Governance

15. Business Model Solution & Strategies Usage Based T&M Fixed Price Fixed + T&M Globally Distributed Provider Managed Testing As A Service Bundle Location Specific Customer Managed Globally Distributed Customer Managed Pricing Model • Bundled solution offering • Costing & Pricing • Providing 1 price for all services inclusive • Defining Service Level Agreements • Supported by Rewards and Penalties Traditional  Testing Delivery Model Mainstream consumers Small businesses Mainstream enterprise customers Early-adopter enterprise cust. Customers

16. Transformation Approach Solution & Strategies Mitigate Risk & Transform