1 / 62

An Automated Test Strategy Based on UML Diagrams

PISATEL. An Automated Test Strategy Based on UML Diagrams. F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ** *PISATEL LAB, IEI-CNR, Pisa, Italy ** ERICSSON LAB ITALY. PISATEL. PISATEL

zenia-chaney
Download Presentation

An Automated Test Strategy Based on UML Diagrams

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PISATEL An Automated Test Strategy Based on UML Diagrams F. Basanieri *, A. Bertolino *, E. Marchetti *, A. Ribolini *, G. Lombardi **, G. Nucera ***PISATEL LAB, IEI-CNR, Pisa, Italy** ERICSSON LAB ITALY

  2. PISATEL PISATEL Pisa Initiative on Software Architectures for Telecommunications PISATEL_LAB The joint conduction by the academia in Pisa and ERI of strategic research projects on software for telecommunications, selected on an annual basis

  3. PISATEL Pisatel Objectives • Main areas of interest are: • software engineering methods and tools, • software architectures, distibuted systems, networks, real time systems, and protocols. • Currently the active projects are: • Integrating Wireless, Wireline and Internet Networks • Analysis and Development of Real Time Software • Quality And Validation of Software Architecture More details: http://www.iei.pi.cnr.it/ERI/

  4. PISATEL An Automated Test Strategy Based on UML Diagrams

  5. PISATEL Objectives • Use for test planning the same UML diagrams developed (e.g., by Rational Rose) for design and analysis • Tool-supported generation and maintenance of an updated and strategic test plan, as the design evolves.

  6. PISATEL (Realistic) Preconditions It is plausible that …. • in early design phases many of the UCs are defined at a high level and only few of them are properly refined. • there is not yet the specification of every possible scenario, but only few SDs already exist that realize the defined UCs. • different people or teams take part to the development, everyone familiar with a different component or functionality.

  7. PISATEL Fundamental Guidelines • FG1: use the UML diagrams developed during specification and design, without imposing to the UML designers any additional formalism, or ad-hoc effort specifically for testing purposes. • FG2: test planning for integration testing must cope with high level diagrams, even though not yet completely specified, or refined

  8. PISATEL Development Process Trade-off • UML is a notation and does not impose a specific process. • A test strategy needs to refer to a systematic and documented design process We want to impose minimal requirements: • Use Cases (UCs) organized into a hierarchy • Express relevant scenarios as Sequence Diagrams (SDs)

  9. PISATEL Designer (from Logical View) ArgoUML vers.0.7 Libraries ArgoUML GEF UML Meta-Model Argo CheckList Kernel Sequence Diagram: CheckList... Sequence Diagram: Kernel / Design Critics Kernel_Sequ... User Model To Do List Sequence Sequence Sequence Diagra... Diagram: Desi... Diagram: To ... Requirements • The reference documentation is derived by collecting together and automatically organizing the diagrams developed during the design phase without additional manual effort of formalization. • Inconsistencies and incompleteness are highlighted (Design weaknesses)

  10. PISATEL Cow_Suiteconsists in the junction of two main components: • UIT (Use Interaction Test) and • CoWTeSt (Cost Weighted Test Strategy) • Cow_Suite =Cowtest pluSUITEnvironment Cow_Suite approach The processing of the Cow_Suite approach starts by analysing the project description*, and proceeds in parallel with the design and analysis refinement steps. [*: the internal description of the project model, given by Rational Rose in a .mdl file]

  11. PISATEL • the Main Tree: a systematic support to decide • how to distribute the available test effort between the many potential test cases • how to prioritise among the test cases when a functional coverage is fixed. Cow_Test • the UITTree:automated generation of abstract test cases from the UML diagrams. • the Test specification: a systematic support to generate the test procedures and subsequently the test scripts. UIT Cow_Suite tool The Cow_Suite tool consists of three elements working in combination:

  12. PISATEL Selection Strategy Test case Test case Cow_Suite Test case Test case Test case Test case Test case Test case SDs selection according to the strategy chosen Test case Test case Interaction with the user for deriving executable test procedures Test case Test case Test case Test case Test case Automatic UIT application for Test cases derivation Test case Obj 1 Obj 2 Obj 3 Method1() U I T Method2() Test specification Method3() Cow_SuiteScheme Test Procedures list

  13. PISATEL Main steps • Derive a Reference Tree • Deduce the Critical Profile • Select the Test Strategy • Derive the Test Cases • Implement the Test Procedures

  14. PISATEL Hierarchical Tree 1.1 Tree Derivation Use Cases (UCs) and Sequence Diagrams (SDs) of the .mdl project design realized with Rational Rose are automatically organized by Cow_Suite tool in a sort of hierarchical (pseudo-)tree, starting from the Main Use Case Diagram, UCD. The case study is ArgoUML, a tool supporting OO design with UML

  15. PISATEL Hierarchical Tree 1.2 Tree Derivation • Looking at the tree the Cow_Suite users can get a complete view of the status of the specification of the various functionalities; • The tree represents an ordered and organized documentation, that is continuously updated with the latest changes and progressively completed

  16. PISATEL 0.75? 0.60? 0.65? 0.65!!! To Do List Sequence Diagram: To ... 2.1 Deduce the Critical Profile • Systems are naturally composed of different parts realizing one or more functionalities, each with different importance for the overall system performance. • The effort devoted to the test of different functionalities must be planned and scheduled considering their relative “importance” (risk-based testing, reliability guided testing, etc…). • Following level by level the tree structure of the UCs and SDs, the strategy allows the people in charge of the development of one or more functionalities to simply annotate the corresponding node in the tree with a number (weight).

  17. PISATEL Hierarchical Tree 2.2 Weighted tree derivation The user annotates every node with a value, leaf weight, representing the relative “importance” of this node (be it a UC or a SD) with respect to the other nodes at the same level in the tree

  18. PISATEL 2.3 Weights management The weight value belongs to the [0,1] interval and must be assigned so that the sum of the weights associated to all the children of one node is equal to 1. The weights are user modifiable as the development proceeds.

  19. PISATEL Integration Testing vs Integration Stage • To realize a complete system functionality, it is generally necessary to execute several actions realizing lower level functionalities • A system functionality is realized by the different components interaction. • Integration Testing: testing interactions between system components (processes, packages, subsystems… ) at an appropriate level of abstraction. • Integration Stage: is useful because generally the functionalities are not specified at the same level of detail and with the same number of UCs and SDs

  20. PISATEL 2.4 Integration Stage Selection • The first integration stage is represented by the main UC and the SDs (if any), which are children of this node (hence they are at level 2 of the tree). • The i-th integration stage is represented by the UCs positioned in at i-th level of the tree and every SDs, children of these nodes, situated at i+1-th level. NOTE: the SDs at level i+1 represent the interaction between the different components that realize the functionalities described in the UCs at i-th level.

  21. PISATEL SDs at 5th tree level Example: 4th Integration Stage The Cow_Suite user selects the integration stage (4) at which the test is conducted. The nodes that are considered by Cow_Suite tool for the subsequent evaluations are all and only those belonging to the 4thintegration stage plus all the tree leaves existing at higher levels (3rd, 2nd, 1st).

  22. PISATEL Designer (from Logical View) ArgoUML vers.0.7 Libraries ArgoUML GEF UML Meta-Model 0.4 0.6 0.3 FW=0.6 FW=0.12 2.5 Final weights assignment • The weights assigned to each node are used to define a relative importance factor, in terms of how risky is that node and how much effort should be put to test it. • Considering every node, from the root down to the elements belonging to the integration stage considered, its final weight is computed as the product of all the nodes weights on the complete path from the root to this node. • The final resulting weight is an element of discrimination for choosing amongst the tests to execute.

  23. PISATEL NT Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case Test Case 3. Cow_Test Strategy Two different situations are considered in test cases planning: 1: Cowtest_ing with fixed number of tests • A certain budget is available, which is translated in terms of number of tests, NT, to execute. • In this case the tool select the most adequate selection of NT tests from among the (many) various test cases that could be potentially conceived.

  24. PISATEL 10 2 2 Minimum Number of tests to execute = 16 1 1 3. Cow_Test Strategy 2: Cowtest_ing with fixed functional coverage • A certain percentage of functionalities must be covered. • In this case by the tool is possible to define in advance which are the functionalities to be covered and the minimum number of tests to execute. Functional Coverage = 90%

  25. PISATEL Cowtest_ing with fixed number of tests. The selection of the most suitable distribution of Test Procedures is developed on the basis of leaves weights. 3.1 Select the Test Strategy

  26. PISATEL Cowtest_ing with fixed number of tests Using the resulting final weight at the chosen integration stage, nw, for each SD the number of Test Procedures, NTtest, can be easily derived

  27. PISATEL Cowtest_ing with fixed functional coverage. The most critical system functionalities are selected for reaching the fixed coverage. Test Procedures are distributed accordingly. 3.2 Select the Test Strategy

  28. PISATEL Cowtest_ing with fixed functional coverage Considering the final weights, nw, of every node the tool can: • select the functional test cases to be run, by ordering in decreasing manner the nw*100 values and summing them, starting from the heaviest one, until the fixed coverage, C, is reached. • Calculate the minimum number of tests with respect to the fixed coverage percentage.

  29. PISATEL Notes • To make a more practical prediction in terms of man/hours, or required budget for testing, it would be necessary to be able to estimate the cost of the different test cases. • An alternative application of Cowtest is considering a certain budget, B, planned for the testing instead of the fixed number of test NT. In this case is possible to evaluate the “relative cost”,b, for the testing of every leaf of the fixed integration stage.

  30. PISATEL Selection Strategy Test case Test case Cow_Suite Test case Test case Test case Test case Test case Test case SDs selection according to the strategy chosen Test case Test case Test case Test case Test case Test case Test case Automatic UIT application for Test cases derivation Test case Obj 1 Obj 2 Obj 3 Method1() U I T Method2() Method3() Cow_SuiteScheme

  31. PISATEL SDs selection N. Test Procedures Final weight 3.3 Sequence Diagrams Selection For each fixed integration stage, a weighted subtree is derived according to the choosen test criterion. Only the relative SDs are considered for Test Case generation.

  32. PISATEL SDs Selection 4. Test Case Derivation The tool uses the UIT method, Use Interaction Test, to derive the Test Cases. The method is mainly based on the SDs (particularly objects, messages and parameters)

  33. PISATEL 4.1 Use Interaction Test methodology • A method to systematically derive Integration Test suites from UML diagrams. • Inspired at large by the Category Partition method: it defines classes of functional equivalence by selecting significant values (choices) of relevant categories (parameters or data) • Incremental test strategy (over the Use Case organizational structure)

  34. PISATEL 4.1.1 Sequence Diagram analysis Horizontal axis shows a set of objects, Test Units, that interact through messages. • Those are systemunits separately testable (class testing) for exercising a possible use of system. • Example: • DecisionModel, Decision, GoalModel, Goal.

  35. PISATEL 4.1.2 Sequence Diagram analysis Vertical axis shows: • the exchanged messages, Interactions Categories, involved in object interactions; • their parameters and relevant inputs, Settings Categories. DecisionModel Settings: _decisions Interaction: DecisionModel() getDecisions() DecisionInteractions: Decision(name:String,priority:int) GetName(), GetPriority() ............

  36. PISATEL Additional Information: Class Diagram Complementary analysis of the Class Diagram description (mainly used for searching the Settings Categories). These are attributes (or a subset of them) of a class (and the corresponding SD’s object) that affect object interactions and are represented by input parameters used in messages or data structures.

  37. PISATEL 4.1.3 Search of Message Sequences MessageSequences: set of Messages (in temporal order), involved in a SD, used by objects to define and elaborate specific functionalities. Example: getDecisions() getPriority()

  38. PISATEL Detailed Test Case Description 4.2 Test Case Derivation For each traceable Message Sequence a Test Case is generated. It contains the list of all its Settings and Interactions Categories and their values.

  39. PISATEL Conditions A Test Case (TC) can be associated to some feasibility conditionsformallyexpressed (notes or specifications tab). In this case TC is divided in different subcases.

  40. PISATEL Number of associated Test Procedures Number of Test Procedures The final weight of every SDs is used to automatically derive the number of Test Procedures associated to each Test Case.

  41. PISATEL Selection Strategy Test case Test case Cow_Suite Test case Test case Test case Test case Test case Test case Test case Test case Interaction with the user for deriving executable test procedures Test case Test case Test case Test case Test case Test case Obj 1 Obj 2 Obj 3 Method1() U I T Method2() Test specification Method3() Cow_SuiteScheme Test Procedures list

  42. PISATEL The user interacts with the tool for inserting Choices values. 5.1 Test Procedures Specification Test Specification: For each identified category, we consider all its possible values and constraints (“choices”).

  43. PISATEL 5.1.1 InteractionsCategories Specification Interactions Categories: objects interactions, exchanged messages. The tool implements two possible choices for the user: the default values, the user values

  44. PISATEL State Diagram If a State Diagram is linked to a specific Test Unit (class), further information can be found about the relevant configurations of InteractionCategories State diagram of the “Critic” class

  45. PISATEL 5.1.2 Specification of SettingsCategories Settings Categories: parameters and inputs relevant for the Test Unit. The tool implements two possible choices for the user: the default values, the user values.

  46. PISATEL 5.2 Test Procedures Implementation A Test Procedure is automatically generated from a Test Specification, for each possible combination of choices of every category involved in a Messages Sequence.

  47. Test Procedures: an example Test Procedure getDecisions() getName() Opening a saved file _decisions Naming Note: Such a Test Case must be repeated for all possible values of _decisions Test Specification: DecisionModel: Settings: _decisions Naming Storage Modularity Inheritance Stereotypes Relationships…….. Interactions: getDecisions Opening a new file Opening a saved file After a modification and before saving After a modification and after saving DecisionModel() Class constructor

  48. PISATEL Constraints on Choices Some choices combinations could result contradictory or meaningless. Following the Category Partition methodology, constraints must be added to the SettingCategories choices. Constraint are specified using: • Properties, that can be assigned to certain choices and testing for by other choices. • IF Selector, is a conjunction of properties that were previously assigned to other choices.

  49. PISATEL Example SettingsCategories: pattern size: empty [Property Empty] single character [Property Not Empty] many character [Property Not Empty] quoting pattern is quoted [Property quoted] pattern is not quoted [if Not Empty] pattern is improperly quoted [if Not Empty]

  50. PISATEL SettingsCategories: Specification Constraints Test Specification: Decision(_name, _priority): _name Naming [if priority<5] Class Selection [if priority=0] Storage Inheritance Modularity….. _priority 0 [Property priority=0] 1 2 3 7 [Property not priority<5]

More Related