Testing as a Service TaaS on the Cloud

1. Testing as a Service (TaaS) on the Cloud Prepared by Jerry Gao Computer Engineering Department San Jose University December, 2011

2. Several Definitions by existing work: Definition #1: “TaaS – automated software testing as a cloud-based service. (by George Candea) From SoCC '10 Proceedings of the 1st ACM symposium on Cloud computing. In their view, TaaS ? (1) a public certification service, akin to Underwriters Labs, that independently assesses the reliability, safety, and security of software; (2) a "home edition" on-demand testing service for consumers to verify the programs they are about to install on their PC or mobile device; and (3) a "programmer's sidekick" enabling developers to thoroughly and promptly test their code with minimal upfront resource investment.” What is TaaS?

3. Definition #2: “Testing-as-a-service (TaaS) is a new model to provide testing capabilities to end users.”[Lian Yu] Definition #3: “Testing Services for Cloud are on the rise. Primarily because companies seek to reduce costs, speed time to benefit and improve quality of the applications they move into production. …. Companies with high defect rates and difficulty in responding quickly to provision test environments…..“ [AppLabs] Why do we need TaaS?

4. What are the major benefits? Provide an effective and efficient way to obtain a virtual and scalable test environment over a cloud infrastructure Share and leverage computing resources using Clouds Obtain required computing resources at anytime Share and reuse software test tools (with licenses) Use a utility model (pay-as-you-go) as a way to charge the provided testing services over Clouds Support testing operations in 365/7/24 Enable large-scale test data and traffic simulation for system testing Importance, Advantages, Benefits

5. Testing as a Service

6. Levels of Testing as a Service (TaaS)

7. Testing Environments for TaaS

8. TaaS - User Perspectives

9. A Typical Workflow for TaaS

10. TaaS Service Classification

11. A Road Map for Testing as a Service (TaaS)

12. TaaS Environment Service

13. TaaS Management Service

14. On-Demand Test Service

15. Test Simulation Service

16. Test Simulation in Infrastructure on Clouds

17. Test Solution Service

18. TaaS Cost Models, Pricing and Billing

19. TaaS Service Product Family

20. Primary Issues and Challenges: Issue #1: How to provide effective on-demand testing service? Issue #2: How to provide an automatic cost-effective virtual test environment for SaaS leveraging cloud computing resources to address cloud scalability? Issue #3: How to address the emerging test solution services in composition, integration, connectivity and portability? Issue #4: How to provide dynamic validation solutions and techniques in clouds to support SaaS and cloud-based applications? Major Issues, Needs, and Challenges

21. Primary challenges: Challenge #1: Lack of industry QoS standards for TaaS Lack of standards in test tools and their connectivity and interoperability to support TaaS service. Lack of standard connectivity interfaces and protocols between existing test tools and SaaS and TaaS services Challenge #2: Security testing of cloud-based software and SaaS Major Issues, Needs, and Challenges

22. Primary Needs: Need #1: QoS standards and testing requirement analysis and modeling to support TaaS planning, test complexity and cost-estimation for TaaS contracts and Service-Level-Agreement(s). Need #2: TaaS enabled SaaS testing platforms w/o technology lock-in on specific cloud technology Need #3: TaaS solution service compositions and integration methods and tools Need #4: TaaS-oriented cost and pricing models for charging end-users and clients Need #5: Cloud enabled solutions to support end-to-end tracking and monitoring in test automation for SaaS and cloud-based applications Major Issues, Needs, and Challenges