Cooperative Agent Approach to Quality Assurance and Testing of Web Software

1. Cooperative Agent Approach to Quality Assurance and Testing of Web Software Hong Zhu Dept. of Computing, Oxford Brookes University Oxford OX33 1HX, UK Email: hzhu@brookes.ac.uk

2. Outline • Identify the key issues in quality assurance and testing of web applications • analyse the cause of difficulties • identify the essences and incidents • analyse uncertainties underlying the development of web applications • discusses their implications • identify the requirements on tools • Propose an approach to support both development and maintenance of web-based applications • Software growth process model • Software growth environment • cooperative multi-agent system • ontology of software development • A prototype system with emphasis on testing and quality assurance Apply Lehman’s theory of software evolution

3. Search for the causes of difficulties • The causes of difficulties in software development can be essences or incidents (Brooks, F. P. Jr, 1987) • Essences: • the difficulties inherent in the nature of software • they are irreducible • will not disappear • cannot be solved through technical solutions • Four essences for all software: • Complexity, Conformity, Changeability, Invisibility • Incidents • difficulties that today attend its production • but are not inherent • What are the particular forms of the essences in web applications? • What are the incidents in developing web applications? • How to overcome the difficulties?

4. Lehman’s theory of software evolution How to classify software systems according to their evolutionary behaviour? Classification of software systems Why software systems have evolutionary behaviour? The driving force behind evolution Uncertainties in software development Laws of software evolution What are the common characteristic properties and rules of software evolution?

5. Classification of software systems • Classification of software systems according to what ‘correctness’ means to the particular system • S-type: required to satisfy a pre-stated specification • correctness is the absolute relationship between the specification and the program • Examples: many safety critical applications • P-type: required to form an acceptable solution to a stated problem in the real world • correctness is determined by the acceptability of the solution to the stated problem • E-type: required to solve a problem or implement an application in a real-world domain which often has no clearly stated specification • correctness of such a system is judged by the users Many kinds of web applications such as e-commerce, e-science, e-government, online learning etc. belong to the E-type!

6. Uncertainties in software development • The driving force of software evolution is the uncertainty in development • There are three types of uncertainties associated with software development • Gödel-like uncertainty: • arises because software systems are models • The representation of a model and its relationship to the real world is Gödel incomplete • The properties of a program cannot be completely known • Heisenberg-type uncertainty: • Using a system inevitably change the user’s perception and understanding of the application • A common phenomenon: • users’ uncertainty about requirements (‘I’ll know it when I see it’) • changing requirements • Pragmatic uncertainty: • Human errors and risks due to: • the adaptation of a new development method • the use of a new software tool or programming language, etc.

7. Impacts of uncertainty on web applications • Heisenberg-type uncertainty in developing web applications • Many web applications are novel to the majority of the users • Users are bound to change their perception and understanding of the application. ===> Changeability • For example: web-based e-banking systems • Assumptions are made based on the knowledge of customer’s habits using local branches and via telephone banking facilities • At beginning, some online banking websites close at night • Access only to the account in the bank • Users’ understanding of e-banking systems and their way of banking started to change once they used e-banking • The busiest time for online banking is at evening around 8:00pm ~10:00pm • Transactions often involve multiple accounts from different banks • it was unpredictable what requirements would emerge before actually implementing and using the system

8. Gödel-like uncertainty in developing web applications • The development of a web application heavily depends on the existence of an accurate model of the real world • There are few mature theories and methods that help developers to build a good model of the system and environment of web applications • The models are complicated due to: ===> Comformity • Execution on distributed, hypermedia, heterogeneous computer platform • Open to the environment, hence vulnerable to malicious attacks • Dependence on unreliable and uncontrollable hardware and software resources; • have to be fault tolerant and • cooperative to other systems • The model has high complexity due to: ===> Complexity • Complexity in computation due to complicated business logics, and enterprise organisational structures and operational processes • Large amount of data in various formats fixed up and linked together • Concurrency in data processing • Code and data are fixed up

9. Pragmatic uncertainty in developing web applications • Technology and tools • web technology rapidly developed in the past a few years and changed frequently • a large number of new techniques emerged • lack of tool supports • Human factors: • A large number of persons have entered the IT profession • Many of them have limited education, experience and training • Constantly under the pressure of time-to-market • Large human resource turnover => lost knowledge of the system • These are incidents rather than essences! • Principles proved to be effective for software testing and quality assurance should be equally applicable to the web applications • Existing methods, techniques and tools must be adapted to the new web technology • New methods and techniques must also be developed for deal with the novel features of web applications

10. Laws of evolution

11. Implications of Lehman’s laws • Lehman’s laws can be considered as a ‘survival guide’ for the evolutionary development of E-type systems. • Violating Lehman’s laws in the development of an E-type system may well mean a death penalty. • Lehman’s laws provide the clues for: • how to organize software development processes • how to construct software tools The state of death becomes visible when demands for modifications of the program cannot be intelligently answered although the program may continue to be used for execution and provide useful results. (Peter Naur, 1992)

12. Requirements of tools derived from the theory • To support evolutionarily development of web applications • to prevent the system’s complexity increasing and quality declining out of control • quality assurance and testing, verification and validation techniques must fit into this evolution process • to enable a long term sustainable evolutionary processes • to constantly maintain a good knowledge about the system • to support developers’ continuous effort to adapt the system to meet users’ new requirements • to build feedback loops between the users and the developers • To deal with the diversity of the components and execution platforms • Various testing, verification and validation tools must be integrated into one environment • The environments must also be easily extended so that new tools can be integrated in the future

13. Proposed approach: growth environment • The software system and its evolution environment co-exists throughout the application system’s whole lifecycle. • The environment: • monitor the evolution process and record the modifications of the system and the rationales behind the modifications • extract, collect, store and process the information about the application system and its performance • present the knowledge of the system and its evolution process to human beings or other software tools when requested • interact with the users and developers cooperatively to obtain feedbacks from the users • grows with the application system • new tools are integrated into the environment to support the development and maintenance of new components • the knowledge about the system is accumulated over the time. • No distinction between development environment (such as CASE) and run-time support environment (such as middleware) • The relationship between the tools and the system is similar to: • a tree and its natural environment where it is growing, • a human and his/her social environment

14. Key issues to be addressed • Key issues in the construction of growth environments : • The tools in the environment must be able to cooperate effectively with each other • interact with each other through a flexible communication and collaboration protocol • The environment and its tools must be able to cooperate effectively with human users • codify the contents of messages in an ontology which represents knowledge about the application domain and software engineering at a high level of abstraction • The environment must be highly extendable • new tools can be easily added into the system, removed from the system, or upgraded by new versions

15. Proposed approach: cooperative agents • The software growth environment consists of the two types of agents. • Service agents: • provide various supports to the development of software systems in an evolutionary strategy • fulfil the functional requirements of development and quality assurance and testing, verification and validation • each service agent is specialized to perform a specific functional task and deal with one representation format • cooperate with each other to fulfil more complicated tasks. • Management agents: • manage service agents • the registration of agents’ capabilities, • task scheduling, • monitoring and recording agents’ states • monitoring and recording the system’s behaviours

16. Development Service Agents Application Sever Development Sever Knowledge Base (Global) Application System Management Agents System Resource Knowledge Base (Local) Application Service Agents Development Service Agents Management Agents Network ... Client Client Server Server ... Interface Agents Server Client Application Service Agents Architecture of software growth environment

17. Prototype system: MAISGE • MAISGE: Multi-Agent Information System Growth Environment • Generic architecture of software growth environment • MAISGE-AquIS: • Developed in the AquIS project at Oxford Brookes University AquIS stands for Agent-based quality assurance of web-based Information Systems

18. Agents for testing web applications

19. Relations Compound relations Basic relations Capable_of More_ powerful Subsumes Ontology of software testing SW testing Concepts Basic Concepts Compound Concepts Capability Method Tester Context Task Activity Environment Artefact

20. Conclusion • Characteristics of Web-based applications: • by nature evolutionary • satisfying Lehman’s laws of evolution • Key problems to be addressed • supporting their sustainable long term evolution • diversity in platform and representation • Basic requirements of such a software environment • to facilitate flexible integrations of tools • to enable effective communications with human beings • Our solution: Cooperative multi-agent software growth environment • various tools are implemented as cooperative agents • interaction with each other and human users using ontology • Current state: • designed and implemented a prototype system for testing web applications • Further work: testing as services

21. References Brooks, F. P. Jr, No silver bullet: essence and accidents of software engineering, IEEE Computer, 1987, pp10~19. Lehman M. M. and Ramil, J. F. Rules and Tools for Software Evolution Planning and Management. Annals of Software Engineering, Special Issue on Software Management, 11(1) 2001, 15-44. Lehman, M. M. Uncertainty in Computer Application. C.ACM, 33(5), 1990, 584-586. Naur, P. Programming as theory building, Computing: A Human Activity. ACM Press, 1992, 37-48. Huo, Q., Zhu, H., and Greenwood, S., A Multi-Agent Software Environment for Testing Web-based Applications, Proc. of COMPSAC'03, Dallas, 2003, 210-215 Huo, Q., Zhu, H. and Greenwood, S. Using Ontology in Agent-based Web Testing. Proc. of ICIIT’2002, Sept., 2002, Beijing, China. Zhu, H. and Huo, Q., Developing A Software Testing Ontology in UML for A Software Growth Environment of Web-Based Applications, To appear in Software Evolution with UML and XML, Hongji Yang (eds.), Idea Group Inc.