Graphical Testing With UML The UML Testing Profile

1. UML Testing Profile Graphical Testing With UMLThe UML Testing Profile Eric Samuelsson eric.samuelsson@telelogic.com

2. UML Testing Profile Agenda • Background • Key concepts • Advanced concepts • Summary

3. UML Testing Profile Background • Test technologies are not on par with modern design technologies such as model-driven architectures (MDA) • There’s a need for a widespread graphical test language • OMG realized this and issued and RFP for a graphical testing profile to UML • Using the UML ensures widespread use across a wide range of application areas • Numerous tool vendors support UML • UML is a proven de facto standard • But UML is focused on definition ofsystem structure and behavior and has no built-in testing constructs...

4. UML Testing Profile UML Testing Profile RFP • Issued by OMG in July 2001 • LOI’s submitted by • Telelogic, Rational, SofTEAM, FOKUS, Ericsson, Motorola, IBM • One joint submission • Initial submission made in April 2002 • Revised initial submission made in June 2002 • Terminology changes • Elaboration of Test Data • Final submission in January 2003

5. UML Testing Profile Requirements and restrictions • Define structural and behavioral test concepts • Conformance testing only • Black-box functional testing • Based upon the UML meta-model • UML 2.0 as proposed by the U2 Partners • Inter-operation with existing testing technologies • TTCN 3 • Junit • ...

6. UML Testing Profile Current proposal • A graphical testing language based the current UML 2 metamodel • Reusing as many UML concepts as possible • Recognition important for UML users • No need to re-invent the wheel • Designed with TTCN 3 in mind • Semantics basically follows TTCN 3 semantics • But caters for other technologies like JUnit as well

7. UML Testing Profile Current proposal • Defined as a UML profile • Will be recognized by UML users • Basically a number of stereotypes • Easy to implement for tool vendors • Models will be interchangeable • Any tool implementing XMI will be able to exchange models UML «profile» Testing Profile «import» «stereotype» TestCase «metaclass» BehavioralFeature

8. UML Testing Profile Key Concepts • Test Architecture • Test Context, Test Configuration • Test Component, SUT • Arbiter • Test Behavior • Test Case • Verdict, Validation Action • Defaults • Test Data • Logical partitions, Pattern matching • Test Deployment

9. UML Testing Profile Test Objective • An informaldescription of what to test in one test case • Commonly expressed using natural language • Can also be expressed using other diagrams, such asuse case diagrams and sequence diagrams ”Verify that if a user inserts and authorizes a valid card correctly, he is able towithdraw money if he has sufficient funds.” Withdrawal Client Client <<include>> Tester InvalidPin

10. UML Testing Profile Test Case • A specification of one test, including • Test objective • Test behavior • Defined as a BehavioralFeature in UML • It’s method is the behavior of the test case «testCase»+validWithdrawal() : Verdict «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict

11. UML Testing Profile Test Behavior • Describes the expected behavior of a test case • Defined using any type of UML behavior • Interactions, statemachines and activities • Facilitates specification of arbitrarily complex behavior Idle WaitCode

12. UML Testing Profile Test Architecture • A complete specification of all types used in the tests • An ordinary UML package • Contains Test Contexts, Test Components, Ports, Interfaces, etc. «testArchitecture» ATMTest

13. UML Testing Profile Test Architecture Example «testArchitecture» ATMTest IAccount «testContext» ATMContext «testComponent» HWEmulator bePort IScreen, ICardMachine, IMoneyBox -pinOk : Boolean -enteredPIN : String -message : String «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict «testComponent» BankEmulator hwCom IATM accounts * Account ATM::CardData balance : Integer number : String * pinCode : Integer number : String cards credit(a : Integer) debit(a : Integer) isPinCorrect(c: Integer) : Boolean

14. UML Testing Profile Test Component • Implements (parts of ) the behavior of one or more test cases • Each test case is normally executed by several components • Ordinary classes or components «testComponent» HWEmulator IScreen, ICardMachine, IMoneyBox -pinOk : Boolean -enteredPIN : String -message : String hwCom IATM -currentCard 0..1 ATM::CardData pinCode : Integer number : String isPinCorrect(c: Integer) : Boolean

15. UML Testing Profile Test Context • A classifier that contains • a number of test cases and their behavior and objectives • default behavior used by the test cases • configuration of test components • test control behavior • Can be a class, a component or a collaboration «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict

16. UML Testing Profile Test Configuration • A specification of the structural aspects of test cases • Defines the parts participating in the test case and their interconnections • Test Components, SUT, Ports, Interfaces and Connectors • Defined as the internal structure of the Test Context «sut» atm : BankATM be : BankEmulator netCom bePort atmPort hwCom accnts : Account [0..*] hwe : HWEmulator

17. UML Testing Profile Test Control • Test execution can be controlled is two ways • By defining the classifier behavior of a test context • Or by invoking the test cases from outside of the context

18. ref ref UML Testing Profile Test Control by ClassifierBehavior sd ATMContext invalidPIN() «testContext» ATMContext «testCase» +validWithdrawal() : Verdict +invalidPIN() : Verdict -authorizeCard() : Verdict [verdict == fail] [verdict == pass] validWithdrawal()

19. UML Testing Profile Test Control by Invocation ad myOp «testContext» MyTestContext verdict := context.TC1() MyClass -context -verdict : Verdict «testCase» +TC1() : Verdict +TC2() : Verdict -TC3() : Verdict * +myOp() [else] [verdict == pass] verdict := context.TC2()

20. UML Testing Profile Test Data • Parameters, default values, return values are reused from UML without changes • Logical partitions • Pattern matching • TDB

21. UML Testing Profile Test Deployment • Test Deployment is important, but there is probably no difference between deploying test systems compared to ordinary UML systems. • We will try to reuse the existing UML deployment concepts without changes • Under investigation

22. UML Testing Profile Advanced Concepts • Test Evaluation • Verdict • Validation action • Arbiter • Default behavior • Structural and behavioral reuse

23. UML Testing Profile Test Evaluation • A number of concepts are used to evaluate test results • Verdicts • Validation actions • Arbiter

24. alt UML Testing Profile Verdicts and Validation Actions • Each test component is associated with a verdict • Validation Actions are used to set the verdict • A Validation Action sets the verdict of a test component and reports the verdict to the Arbiter «sut» Client ATM storeCardData(cardData) display(”Enter PIN”) isPinCorrect(pin) true false setverdict(fail)

25. UML Testing Profile Arbiter • A test component that knows how to evaluate the test behavior • Separates test evaluation from test case specification • One behavior can yield different results • Each test context has a default arbiter • Implicit connections to all test components • TTCN 3 semantics • Advanced arbiters can be defined • Caters for more advanced semantics «arbiter» myArbiter setVerdict(v : Verdict)

26. enterPinDefault UML Testing Profile Default behavior • Defines behavior for unexpected observations • Can be expressed using any UML behavior • Can be applied to several different levels: • Test component • Message reception • Alternative statement • ... sd authorizeCard «sut» Client ATM storeCardData(cardData) display(”Enter PIN”) isPinCorrect(pin) true

27. alt UML Testing Profile sd enterPinDefault self getStatus statusOk setverdict(pass) Default expressed as interaction... display(”Connection lost”) setverdict(inconc) ejectCard() setverdict(fail)

28. UML Testing Profile enterPinDefault getStatus / ^statusOk, setverdict(pass); ...and as statemachine ejectCard / setverdict(fail) * display(”Connection lost”) / setverdict(inconc) The representations are semantically equivalent

29. UML Testing Profile Meta Default • Each test component has a meta default to cater for differences in semantics • Can be redefined MetaDefault * * / defer

30. UML Testing Profile Reuse of Test Structure and Behavior

31. UML Testing Profile «testContext» ATMContext «sut» atm : BankATM be : BankEmulator accnts : Account [0..*] netCom bePort atmPort ATM hwCom hwe : HWEmulator ac: AuthorizationContext Client

32. ref UML Testing Profile sd ValidWithdrawal «sut» hwe be atm atmPort netCom ac.authorizeCard() selectOperation(withdrawal) true withdraw(amount) findAccount(cardData) account debitAccount(account, amount) true deliverMoney(amount) display(”Take cash”) true

33. UML Testing Profile Summary • The testing profile enables complete graphical specification of tests for conformance testing of UML models • Defined as a UML profile • Reuses existing UML concepts whenever possible • Designed for inter-operation with TTCN-3 and other technologies • Due date • January 6th, 2003 • More information • Testing Profile Consortium • OMG’s Testing Profile page