Collaborative Testing of Web Services -- The Service oriented framework and implementation in Semantic WS

1. Collaborative Testing of Web Services-- The Service oriented framework and implementation in Semantic WS Hong Zhu Department of Computing and Electronics Oxford Brookes University Oxford OX33 1HX, UK

2. Acknowledgement • Mr. Yufeng Zhang, MSc student at the National University of Defence Technology, China • Mr. Qingning Huo, PhD student at Oxford Brookes University, UK • Dr. Sue Greenwood, Oxford Brookes University, UK Seminar at Tsinghua University

3. Overview • Analyse the impact of the novel features of service-orientation on software testing • Identify the grant challenges in testing WS applications • Propose an framework to support testing WS • Report on prototype implementation of the framework and case studies • Compare with related works and discussion of future work Seminar at Tsinghua University

4. Characteristics of Web Services • Web services is a distributed computing technique that offers more flexibility and looser coupling so that it is more suitable for internet computing. • The dominant of program-to-program interactions • The components of WS applications (such as service providers): • Autonomous: control their own resources and their own behaviours • Active: execution not triggered by message, and • Persistent: computational entities that last long time • Interactions between components: • Social ability: discover and establish interaction at runtime • Collaboration: as opposite to control, may refuse service, follow a complicated protocol, etc. Seminar at Tsinghua University

5. Registry register service Search for services registered services request service Requester Provider deliver service WS technique stack • Basic standards: • WSDL: service description and publication • UDDI: for service registration and retrieval • SOAP for service invocation and delivery • More advanced standards for collaborations between service providers and requesters. • BPEL4WS: business process and workflow models. • OWL-S: ontology for the description of semantics of services Seminar at Tsinghua University

6. A typical scenario Car Insurance Broker Suppose that a fictitious car insurance broker CIB is developing a web-based system that provides a complete service of car insurance. • End users: • Submit car insurance requirements to CIB • Get quotes from various insurers • Select one insurer to insure the car • Submit payment information • Get insurance document//confirmation • Broker: • Take information about the user and the car • Check the validity of user’s information • Get quotes from insurers and pass them to the user • Get user’s selection of the insurer • Get insurance from the insurer and pass the payment to the selected insurer • Take commissions from the insurer or the user Seminar at Tsinghua University

7. CIB’s service requester GUI Interface WS Registry CIB’s Services Bank B’s Services Insurance A1’s Services Insurance A2’s Services Insurance An’s Services Structure of the CIB application Other service users End users Could be statically integrated Should be dynamically integrated for business flexibility and competence, and lower operation and maintenance cost Seminar at Tsinghua University

8. Testing own side services (1) • Similar to test software components. • Many existing work on software component testing can be applied or adapted with special considerations: • The stateless feature of HTTP protocol; • XML encoding of the data passing between services as in SOAP standard; • Confirmation to the published descriptions: • WSDL for the syntax of the services • workflow specification in BPEL4WS • semantic specification in e.g. OWL-S. Progresses on these issues have been made in the past few years Seminar at Tsinghua University

9. Testing own side services (2) • Dealing with requesters’ abnormal behaviours • The requesters are autonomous, thus they may stop cooperation in the middle of a transaction for many reasons, such as intentional quit, network failure, or failures of requester’s software system due to fault. • Burdens are on the testers to ensure that the system handles such abnormal behaviours properly. • Dealing with unexpected usages/loads • As all web-based applications, load balance is essential. • But, the knowledge of the usage of a WS may not be available during the design and implementation of the system. • Dealing with incomplete systems • A service may have to rely on other services to perform its functionality properly, thus hard to separate the testing of the own services from the integration testing, especially when it involves complicated workflows. • In the worst case, when WS is dynamically bound to the other services, the knowledge of their format and semantics can only be based on assumptions and standards. Seminar at Tsinghua University

10. Testing of other side services • Some similarity to component testing, however, the differences are dominant • Lack of software artifacts • Lack of control over test executions • Lack of means of observation on system behaviour Seminar at Tsinghua University

11. Lack of software artifacts • The problem: No design documents, No source code, No executable code • The impacts: • For statically bound services, • Techniques that automatically derive stubs from source code are not applicable • Automatic instrumentation of original source code or executable code is not applicable • For dynamic bound services, • Human involvement in the integration becomes completely impossible. • Possible solutions: (a) Derive test harness from WS descriptions; (b) The service provider to make the test stubs and drivers available for integration. Seminar at Tsinghua University

12. Lack of control over test executions • Problem: • Services are typically located on a computer on the Internet that testers have no control over its execution. • Impact: • Control over the execution has been essential to apply existing testing techniques and will continue to be essential for testing services: • An invocation of the service as a test must be distinguished from a real request of the service. • System may be need to be restarted or put into a certain state to test it. • The situation could become much more complicated when a WS is simultaneously tested by many service requesters. • Possible solution: • The service provider must provide a mechanism and a service that enable service requesters control the testing executions of the service. Currently, there is no support to such mechanisms in W3C standards of WS. Seminar at Tsinghua University

13. Lack of means of observation • The problem: • A tester cannot observe the internal behaviours of the services • The Impacts: • No way to measure test coverage • Possible solutions: • The service provider provides a mechanism and the services to the outside tester to observe its software’s internal behaviour in order to achieve the test adequacy that a service requester requires. • The service provider opens its document, source code as well as other software artifacts that are necessary for testing to some trusted test service providers. Seminar at Tsinghua University

14. Testing service composition • The need to deal with diversity • the parts may operate on a variety of the hardware and software platforms • different deployment configurations • delivering different quality of services. • The need of testing on-the-fly • testing just before the invocation Consequences of testing on-the-fly: • The need of non-intrusive testing: the test invocations of a service must be distinguished from the real ones • service provider: the normal operation of the service must not disturbed by test activities. • client: do not actually receive the real services and do not incur the cost • The need of automation: all test activities to be performed automatically. A typical scenario of service oriented computing is that a service requester searches for a required function in a registry, and then dynamically links to the service and invokes it. Seminar at Tsinghua University

15. Dealing with diversity • The problem • The need to deal with diversity • The Implications • testing must be performed in a heterogeneous environment. • different service requesters may well have different test requirements to meet their own business purposes • Possible solutions • Universal powerful testing tools: expensive if not impossible • Collaboration and integration of a variety of tools Seminar at Tsinghua University

16. Testing on-the-fly • The problem • Test just before the invocation while the parts are in operation • Implications • human involvement is impossible • Not to interference with the normal operation • Possible solutions • Fully automated testing process • Mock services, etc. Seminar at Tsinghua University

17. The proposed approach • A WS should be accompanied by a testing service. • functional services: the services of the original functionality • testing services: the services to enable test the functional services. • Testing services can be either provided by the same vendor of the functional services, or by a third party. • Independent testing services: • Provider: testing tool vendors companies of specialized in software testing • The services: • to generate test cases, • to measure test adequacy, • to extract various types of diagrams from source code or design and specification documents, etc. Seminar at Tsinghua University

18. Architecture of service oriented testing Seminar at Tsinghua University

19. Illustration of service oriented testing Seminar at Tsinghua University

20. How does the system work The Scenario • Suppose the car insurance broker want to search for web services of insurers and test the web service before making quote for its customers. Testing the integration of two services Information about the car and the user Insurer Web Service IS Car Insurance Broker CIB Insurance quotes customer Seminar at Tsinghua University

21. Seminar at Tsinghua University

22. Collaboration Process in the Typical Scenario Seminar at Tsinghua University

23. Automating Test Services • The key technique issues to enable automated online test of WS: • How a testing service should be described, published and registered at WS registry; • How a testing service can be retrieved automatically even for testing dynamically bound services; • How a testing service can be invoked by both a human tester and a program to dynamically discover a service and then test it before bind to it. • How testing results can be summarized and reported in the forms that are suitable for both human beings to read and machine to understand. These issues can be resolved by the utilization of a software testing ontology (Zhu & Huo 2003, 2005). Seminar at Tsinghua University

24. STOWS: Software Testing Ontology for WS • Ontology defines the basic terms and relations comprising the vocabulary of a topic area as well as the rules for combining them to define extensions to the vocabulary • STOWS is base on an ontology of software testing originally developed for agent oriented software testing (Zhu & Huo 2003, 2005). • The concepts of software testing are divided into two groups. • Knowledge about software testing are also represented as relations between concepts Seminar at Tsinghua University

25. STOWS (1): Basic concepts • Tester: a particular party who carries out a testing activity. • Activity: consists of actions performed in testing process, including test planning, testcasegeneration, testexecution, resultvalidation, adequacymeasurement and testreportgeneration, etc. • Artefact: the files, data, program code and documents etc. inovlved in testing activities. An Artefact possesses an attribute Location expressed by a URL or a URI. • Method: the method used to perform a test activity. Test methods can be classified in a number of different ways. • Context: the context in which testing activities may occur in software development stages to achieve various testing purposes. Testing contexts typically include unit testing, integration testing, system testing, regression testing, etc. • Environment. The testing environment is the hardware and software configurations in which a testing is to be performed. Seminar at Tsinghua University

26. Tester Atomic Service Composite Service Test planning Test Case Generation Test Execution Test Activity Result validation Adequacy measurement Report generation Structure of basic concepts: Examples Seminar at Tsinghua University

27. Capability Artefact 1 1 0-* Activity Method Capability Data 0-1 0-1 Context Environment 1-* <<enumeration>> Capability Data Type Input Output STOWS (2): Compound concepts • Capability: describes what a tester can do • the activities that a tester can perform • the context to perform the activity • the testing method used • the environment to perform the testing • the required resources (i.e. the input) • the output that the tester can generate Seminar at Tsinghua University

28. Task Artefact 1 1 1-* Activity Method Task Data 0-1 0-1 Context Environment 1-* <<enumeration>> Task Data Type Input Output • Task: describes what testing service is requested • A testing activity to be performed • How the activity is to be performed: • the context • the testing method to be used • the environment in which the activity must be carried out • the available resources • the expected outcomes Seminar at Tsinghua University

29. STOWS (3): Relations between concepts • Relationships between concepts are a very important part of the knowledge of software testing: • Subsumption relation between testing methods • Compatibility between artefacts’ formats • Enhancement relation between environments • Inclusion relation between test activities • Temporal ordering between test activities • How such knowledge is used: • Instances of basic relations are stored in a knowledge-base as basic facts • Used by the testing broker to search for test services through compound relations Seminar at Tsinghua University

30. Tester T2 * C Matches T Capability Task C2 * * * C1 * T1 * Contains Match IsMorePowerful MorePowerful Contain Compound relations • MorePowerful relation: between two capabilities. • MorePowerful(c1, c2) means that a tester has capability c1 implies that the tester can do all the tasks that can be done by a tester who has capability c2. • Contains relation: between two tasks. • Contains(t1, t2) means that accomplishing task t1 implies accomplishing t2. • Matches relation: between a capability and a task. • Match(c, t) means that a tester with capability c can fulfil the task t. Seminar at Tsinghua University

31. Definition of the MorePowerful relation A capability C1 is more powerful than C2, written MorePowerful(C1, C2), if and only if • C2’s capability is included in C1’s activities • C1 and C2 have the same context. • Environment of C1 is the enhancement of the environment of C2. • The method of C2 is subsumed by C1. • For each input artefact of C1 , there is a corresponding compatible input in the input artefact of C2 • For each output artefact of C2 there is a corresponding compatible output artefact of C1. Seminar at Tsinghua University

32. Definition of the Contains relation A task T1contains task T2, written Contains(T1, T2), if and only if • T1 and T2 have the same context, • T1’s activities include and T2’s activities, • The method of T1 subsumes the method of T2, • The environment of T2 is an enhancement of the environment of T1, • For each input artefact of T1, there is a corresponding compatible the input artefact of T2, • For each output artefact of T2 , there is a corresponding compatible the output artefact of T1. Seminar at Tsinghua University

33. Definition of the Matches relation A capability C matches a taskT, written Matches(C, T), if and only if • C and T have the same context, • C’s activities include T’s activity, • The method of C subsumes the method of T, • The environment of T is an enhancement of environment of C, • For each input artefact of T , there is a corresponding compatible input artefact of C, • For each output artefact of C, there is a corresponding compatible the output artefact of T. Seminar at Tsinghua University

34. Properties of the compound relations (1) The relations MorePowerful and Contains are reflexive and transitive. (2) c1, c2Capability, tTask, MorePowerful(c1, c2) Matches(c2, t) • Matches(c1, t). (3) cCapability, t1, t2Task, Contains(t1, t2) Matches(c, t1) Matches(c, t2). Seminar at Tsinghua University

35. Prototype implementation • Representation of STOWS in OWL • Both basic and compound concepts are classes in OWL and represented as XML data definition • Use STOWS in Semantic Web Services • Compound concepts represented in OWL are transformed into OWL-S Service Profile for registration, discovery and invocation • UDDI /OWL-S registry server: using OWL-S/UDDI Matchmaker • The environment: Windows XP, Intel Core Duo CPU 2.16GHz, Jdk 1.5, Tomcat 5.5 and Mysql 5.0. Seminar at Tsinghua University

36. Activity Context Environment Method Capability data Input Artefacts Output Artefacts ServiceCategory INPUT PARAMETERS ContextMark EnvironmentMark MethodMark Artefacts… OUTPUT PARAMETERS Artefacts… Service profile Capability Transformation of STOWS in OWL-S Seminar at Tsinghua University

37. Ontology Management • Motivation • All the terms used in the capability description for test service registration and discovery and invocations must be first defined in the ontology. • However, it is impossible to build a complete ontology of software testing • the huge volume of software testing knowledge • the rapid development of new testing technique, methods and tools. • It is only a framework and has not been attempted to be complete. • Therefore, the ontology must be extendable and open to the public for updating. • An ontology management mechanism is provided to enable the population of the ontology Seminar at Tsinghua University

38. The ontology management mechanism • It provides three services to users: • AddClass: to add new concept • DeleteClass: to delete concept • UpdateClass: to revise concept of the ontology • Restrictions on the manipulation of the data model • Authority Checker: • elementary classes • form the framework of the ontology STOWS. • None of them could be pruned down • extended classes • attached to the elementary classes to define new concepts • instances of the concepts. • added by the users and can be deleted from the hierarchy • Conflict Checker • the new class to be added does not exist in the ontology • the class to be deleted has no subclasses in the hierarchy Seminar at Tsinghua University

39. Structure of OMS Seminar at Tsinghua University

40. Test brokers • A test service that compose existing test services • Decompose test tasks into subtasks • Search for test services to carry out the subtasks • Select test services from candidates • Coordinate the selected test services • Invoke them in the right order • Pass data between them • Collects test results, etc. • Itself is a test service as well. • There may be multiple test brokers owned by different vendors. Seminar at Tsinghua University

41. Architecture of the prototype test broker We have developed a prototype test broker to demonstrate the feasibility of the approach. Seminar at Tsinghua University

42. Test broker process model Seminar at Tsinghua University

43. Case Studies: Overview Case Study 1: An automated software testing tool CASCAT is wrapped into a test service called TCG Case Study 2: A service specific test service called T-NCS is developed for testing a Web Service that provide Numerical Calculation Services (NCS) Case Study 3: Automated composition of TCG and T-NCS through test broker to perform on-the-fly testing of NCS. Seminar at Tsinghua University

44. Case study 1: (a) The subject • The subject CASCAT: • Automated component testing tool for EJB • Generate test cases from algebraic specification written in CASOCC • Execution of EJB on test cases and reports errors if any axiom in the specification is violated Bo Yu, Liang Kong, Yufeng Zhang, and Hong Zhu, Testing Java Components Based on Algebraic Specifications, Proc. of ICST 2008, April 9-11, 2008, Lillehammer, Norway.  Liang Kong, Hong Zhu and Bin Zhou, Automated Testing EJB Components Based on Algebraic Specifications, Proc. of TEST’07/ COMPSAC’07, Vol. 2, 2007. pp717-722.   Seminar at Tsinghua University

45. Case study 1: (b) Registry • Service Profile of CASCAT • ServiceCategory: “TestCaseGenerationServices”. • Input artefact: specified by the class CasoccSpecification, which is a subclass of Specification and stands for algebraic specification in CASOCC. • Context: “ComponentTest”. • Environment: ‘not limited’. • Method is CASOCC-method, which is a subclass of SpecificationBased method. • Output artefact: test case. Seminar at Tsinghua University

46. Case study 1: (c) Submitting search requests • Test requester: • a service was built that plays the role of test requester. • It constructs test tasks and submits them to the test broker to search for a test service • Test task that it produced is to generate test case from CASOCC specification in the context of the test as component test. Seminar at Tsinghua University

47. Example of test task <Task rdf:ID="testcaseTask"> <inputArtefact rdf:resource="#CasoccSpec"/> <outputArtefact> <TestCase rdf:ID="testcase"> <Location rdf:datatype="http://..."> http://resourse.nudt.edu.cn/testcase/ fictitioustestcase.txt </Location> </TestCase> </outputArtefact> <hasActivity> <TestCaseGeneration rdf:ID="GenerateTestcase"/> </hasActivity> <hasMethod> <CASOCC-method rdf:ID="CASOCCBasedMethod"/> </hasMethod> <hasContext rdf:resource="#ComponentTest"/> </Task> Seminar at Tsinghua University

48. Case study 1. (d) Search and discovery • Test Broker: • Once receives a test task, it generates a capability description from the test task • Constructs a Service Profile. • Then calls the API of the Matchmaker Client to search for test service providers. Seminar at Tsinghua University

49. Case study 1: (e) Invocation • A Java Enterprise Bean was deployed on Jboss platform • A formal specification of the bean was written in CASOCC. • The web service of CASCAT is invoked as a web service to generate test case of the component. • The result: • Test cases as an instance of the OWL class TestCase. • Stored as a file on a web server • The Location attribute of the file is returned by the service. Seminar at Tsinghua University

50. Case study 2: Overview • Goal: to develop a test service (T-NCS) that supports testing a Web Service that provide Numerical Calculation Services NCS • Tasks: • Registered: • Capability is described in the ontology represented in OWL-S • Searchable: • It can be searched when the testing task matches its capability • Invoked through the internet • As a web services to execute test cases on the NCS when invoked Seminar at Tsinghua University