Enabling Automated Integration Testing of Cloud Application Services in Virtualized Environments

1. Enabling Automated Integration Testing of Cloud Application Services in Virtualized Environments Tariq M. King, Annaji S. Ganti, and David Froslie

2. Outline • Introduction • Background • Research Problem • Approach • Prototype • Discussion • Related Work • Conclusion & Future Work

3. Introduction • Cloud provides ubiquitous, on-demand access to computing resources “as services” via the Internet • Operational Perspective • Availability, Scalability, Efficiency, … • Development Perspective • ReusableComponents, Info. Hiding • Highly Complex Local Infrastructure • Testing Perspective?

4. Introduction Motivation • Cloud poses several new testing challenges but also brings a number of benefits to software testing Challenges of Testing Cloud Applications • High Complexity, Remoteness, Information Hiding, Autonomy, High Performance and Dependability… Benefits of Software Testing In The Cloud (STITC) • Computational Power and Storage • Virtualization Theme: Use Benefits to Overcome Challenges

5. Introduction RESEARCH Position “Enabling Automated Integration Testing of Cloud Application Services in Virtualized Environments” • STITC is an emerging discipline with potential to significantly change how we do testing • STITC 2010Workshop focused on three key areas: • Testing IN the Cloud – leveraging the cloud for test execution and testing in a virtualized environment • Testing OF the Cloud – validating applications that are hosted and deployed in the cloud • Testing TO the Cloud – moving the testing process and other assets to the cloud

6. Background Cloud Computing • “Services” in cloud computing typically fall intothree categories: Software-as-a-Service| Google Docs, Office Web Apps Platform-as-a-Service| Google App Engine, Windows Azure Infrastructure-as-a-Service | Amazon EC2, Rackspace

7. Background Virtualization • Many cloud platforms offer Virtualization support • Creating abstract computing resources from more powerful physical resources Hosted Architecture • Hypervisor Architecture

8. Research Problem SCOPE • To narrow the research scope, we analyze a specific cloud application development scenario • One host (Provider B) is developing an application service that will extend the functionality of a service offered remotely by another host (Provider A)

9. Research Problem ISSUES • Even with its limited scope, the scenario reveals many integration testing issues and challenges: • Difficult for Provider B to Setup Some Tests for his application since he does not have full control and observation of Service A. Possible Solutions? • Developing Accurate Stubs is Hard since Provider B has no knowledge of Service A’s implementation • Provider B Limited to Black Box Testing of those parts of his application implemented by Service A • Service B can Change w/oProvider B’s Knowledge i.e., Update to Service A Need to Retest Service B

10. Approach • Our approach to the research problem introduces the notion of Test Support as-a-Service (TSaaS) • Prior to deploying Service A, Provider A would have tested it, which typically involves creating: • Test Artifacts – Automated Test Drivers and Stubs • A Test Bed – Hardware/Software Environment • TSaaS seeks to reuse such existing test automation for the development and delivery of a set of test support services for collaborating partners

11. Approach Overview • Provider Bhas full access to a test copy of Service A • Test hooks, instrumentation, and scaffolding reuse • Test data (non-confidential) can be made available • Testing does not interrupt service in production • Concerns: Maintenance, Scalability, Security Provider A exposes TSaaS to Provider B

12. Approach ARCHITECTURE Hypervisor-Based Virtualization and Secure Access

13. Approach CONTROL FLOW Control Flow When Realizing Test Support Services

14. Prototype Proof of concept • Designed with goal of making its components easily deployable to Windows Azure® • Web & Worker Roles (ASP.NET, IIS7, WCF Services) • VM Role (Windows 2008 R2 Server Image) • Application Domain: Credit Reporting/Banking

15. Discussion The primary reason for developing the prototype was to investigate the feasibility of implementing TSaaS Successes • Able to quickly develop and expose a cross section of test support operations for the credit service by reusing its existing test artifacts and tooling • Test authoring and execution, code coverage, error reporting, and server-side diagnostics • Facilitates independently setting up preconditions, applying inputs, making assertions, doing cleanup

16. Discussion Accidental difficulties Technical Challenges • Locating and Integrating Required Technologies • Combines programming knowledge, test automation skills, and runtime virtualization expertise • Configuring Virtual Hard Disk Images of Servers • Enable Auto-Login, Install Security Patches, Disable Reboot after Auto-Update, Set User Privileges • Emphasized the importance and need for VM tools for manipulating and updating these images

17. Discussion Key Research Challenge • Provision of standardized interfaces and guidelines for performing testing in the cloud and developing online test support services for the cloud. Limitations • Lack of formalized evaluation of the approach • Narrow scope of research problem and no evidence of generality of the approach • Feasible to Implement Practical

18. Related Work • Formal modeling and model-based testing criteria for cloud applications (Chan 2009) • Model-driven approach to security testing of cloud applications (Zech2011) • Industry challenges for runtime integration and testing for software services (Greiler 2009) • Many works on testing and testability of SOA(Mei 2008, O’Brien 2007,Tsai 2006)

19. Conclusion and Future Work • Presented an approach to support integration testing of cloud applications using virtualized environments – Test Support as-a-Service (TSaaS) • Described implementation of a TSaaS Prototype • Provides some evidence that approach is feasible • Future work calls for Evaluation and Extension • Azure Experiments, Alternative Implementation, Standardization, Expand Problem Scope, Benefits Virtualization Benefits/Implications, Performance • Building Self-Testing into the Cloud (STITC 2011)

20. Acknowledgements • Neha Kale, MS Student (NDSU) • Dr. Scott Tilley, Professor (Florida Tech), and the participants of STITC 2010 • Alisson Sol, Knowledge Engineer (Microsoft) and the participants of 2011 Microsoft Fargo Engineering Day Excellence Expo • CASCON 2011 Reviewers

21. Thank You! Questions?¿Preguntas?問題Sawwalвопросы質問domandeερωτήσεις