V&V&S

1. Verification, Validation and Synthesis: doing away with defects V&V&S

2. Verification and Validation Barry Boehm (Boehm, 1981): Verification: To establish the truth of correspondence between a software product and its specification (from the Latin veritas, “truth”). Validation: To establish the fitness or worth of a software product for its operational mission (from the Latin valere, “to be worth”). Verification: are we building the product right? Validation: are we building the right product?”

3. Recall the important errors we make: • We plan a functionality that is not in harmony with the purpose of the system. • We define the functionality in a way that is internally inconsistent. • We implement a different functionality than the one we planned.

4. Developers Verification and validation • Validate that: • the functional requirements are right • the functional design is right • the execution is right • Verify that: • that the design satisfy requirements • that the implementation satisfies the design • Check for consistency and properties check check verify verify check Validate Validate

5. Quality and quality assurance methods Quality: process quality = meeting the specification. system quality = playing the role required by the environment. Methods: 1. Constructive methods that aim to generate the right results in the first place => Model driven development, design synthesis, readable models. 2. Corrective methods that aim to detect errors and make corrections. => Verification and validation.

6. Results Requirements Design Implementation Techniques Corrective methods (V&V) Constructive methods Defects and errors Model, Check,Synthesize -design -implement. Reviews Analysis Testing

7. Techniques for checking and analyzing models • Syntax analysis: by model editor • Static semantic analysis: by model analyzer/inspector • Static interface compatibility • Dynamic semantic analysis: by tools for reachability analysis (also called modelchecking) • Dynamic interface compatibility • Execution: by simulation and testing • Verification: by manual inspection and checking formal relationships between models • Validation: by manual inspection, by simulation, testing and interface compatibility checks

8. Static interfaces • Java Interfaces – One-way method invocations with parameters • CORBA Interfaces –defined using IDL • Web services – One way remote invocations with parameters - defined using WSDL • APIs – One-way invocations with parameters. • UML Interfaces – One-way invocations or messages with parameters • SDL Interfaces – Two way asynchronous messages with parameters • UML Collaborations – Two way invocations or messages with parameters

9. Static interface checks illustrated using SDL (1) • Derive valid input and output signal sets from process graph • Check signal sets against signal lists on each connection: P2in = L2 + L3 + L5 P2out = L1 + L4 + L6 • Check signal destinations: P2 out-to-p1 = L1 P2 out-to-p3 = L4 P2 out-to-env = L6

10. Static interface checks (2) • Valid input and output signal sets against gate signal lists once for each type • Signal lists against each other for each connection • Signal destinations

11. Static interface checks (3) • Find logical signal routes by signal list partitioning conserving flows • Signal sets against signal lists • Signal destinations Who talks together here?

12. Dynamic analysis Liveness = something good will eventually happen, e.g. sometimes a service is successful (system specific) safety = nothing bad will ever happen, e.g. no deadlocks (system independent) 1. Reachability analysis Full (up to 10**5 states) Controlled partial (up to 10**8 states) Random simulation (larger) 2. Structural reasoning Invariants Logical inference on equations, rules, axioms Model checking 3. Execution Simulation Testing

13. Reachability analysis • Iteratively generate all reachable system states • Analyse each state for • deadlocks; • incompleteness; • other errors. • Analyse the generated graph • partial deadlock; • liveness; • etc.

14. [Rb,Eb] [Gb,Sa,Qa] [Aa] A B [Ga, Sb,Qb] [Ra,Ea] [Ab] A B 1 1 Ra Sa Rb Sb Sb Ga Sa Gb 3 8 3 8 Gb Qa Ga Qb 1 1 Aa Ab 5 5 Ea Eb Qb Qa 1 1 Are there any problems here?

15. Step 1: Make transition charts

16. Step 2: Generate reachability graph Find all possible behaviours considering that a signal transfer takes two steps: • send • consume

17. The errors: Unspecified receptions and deadlocks may occur The reason: mixed initiatives not properly handled Graph “explodes” because independent behaviours are interleaved and signals are queued Many paths lead to the same error Findings Is there a simpler way??

18. [Rb,Eb] [Gb,Sa,Qa] [Aa] A B [Ga, Sb,Qb] [Ra,Ea] [Ab] A B 1 1 Ra Sa Rb Sb Sb Ga Sa Gb 3 8 3 8 Gb Qa Ga Qb 1 1 Aa Ab 5 5 Ea Eb Qb Qa 1 1 Analyzing role behavior directly in SDL Association role behaviour = the visible behaviour of one object when observed from another (associated) object. Mixed initiative state = a state with inputs from two independent sources (i.e. association roles) mixed initiative state invisible from B

19. Role behaviour and input consistent roles • Deriving a role behaviour: • Replace invisible inputs by “none” (or just make a mental note) • Remove invisible outputs (or just ignore them) • The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) • Input consistency: • An invisible transition is a transition with a none input. • An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. • The role is input consistent iff every invisible transition in the role is input consistent. Are there input inconsistent transitions here? A 1 invisible transitions none Sa Sb Ga 3 8 Gb Qa 1 5 none Qb 1

20. Role behaviour and input consistent roles • Deriving a role behaviour: • Replace invisible inputs by “none” (or just make a mental note) • Remove invisible outputs (or just ignore them) • The result is a process graph with only visible inputs and outputs. (or just a mental view on the original graph) • Input consistency: • An invisible transition is a transition with a none input. • An invisible transition is input consistent iff the next-state explicitly accepts all the visible inputs of the (present) state. • The role is input consistent iff every invisible transition in the role is input consistent. A 1 invisible transitions none Sa Sb Ga 1 and 3 are not input consistent because 3 does not accept Sa 3 8 Gb Qa 1 5 5 and 1 are input consistent because 1 accepts more than 5 none Qb 1

21. How to ensure input consistency ? • When checking - for each role: • Find invisible transitions • Check that every invisible transition is input consistent • Consider transitions from mixed initiative states • When designing: • for each mixed initiative state, check for each role that all transitions are input consistent. Mixed initiative state A invisible transition 1 none Sa Sb Ga 1 and 3 are not input consistent 3 8 Gb Qa 1 5 none The intuition behind this is that the associated process will not immediately observe an invisible transition and therefore may send the same output in both states of the invisible transition. 5 and 1 are input consistent Qb 1

22. S-Rule: Input consistent process Design SDL processes such that all their role behaviours, i.e. the behaviour visible to processes in their environment, are input consistent. Check the mixed initiative states

23. Dynamic Interfaces • Not only static lists of operations/messages • But also the the dynamic behavior that must be obeyed • Defined by (dual) interface role behaviors • That are projections of local behavior onto interfaces A B 1 1 1a 1b 1 none Sa Sb Ga 3 8 Gb Qa 1 5 none Qb 1

24. session Compatibility and interface roles Interface roles are ”projected and distilled” behaviours visible on interfaces, or associations. Given two state machines A and B: • Derive the interface role behaviours • Project: hide and transform • Distill: gather, minimize and merge • Detect inconsistencies • Correct inconsistencies and repeat from 1 • Validate role compatibility 3 1 B 1 A 1a 1b 2 2

25. Two interface roles (a-roles) in state transition form • Interfaceroles (alsocalleda-roles) definedistilledinterfacebehavior • dual rolescan be obtained by: • miroring • and conflictresoulution • Heremixedinitiatives show up as nodes withboth output and input transitions • anyconflictshere? a b b1 a1 !m2 ?m2 ?m1 !m1 a2 b2 b3 a3 ?m3 !m3 ?m4 !m4 b4 a5 b4 a4

26. conflict resolution after mixed initiatives a b b1 a1 Two kinds of behaviour: service behaviour and conflict reolution behaviour The service/feature behaviour of a dual roles can normally be obtained by mirroring Conflict resolution in general, can not be obtained by mirroring. Hence, a dual role may contain two parts: a mirrored part providing the core service logic and a conflict resolution part. In this example a and b are dual! !m2 ?m2 ?m1 !m1 a2 ?m1 b2 b3 a3 ?m2 ?m3 !m3 ?m4 b6 !m4 a5 !m4 b4 a5 ?m4 b4 a4 b1 a1 ?m1 !m2 ?m2 !m1 b2 b3 ?m2 ?m1 a2 a3 !m3 ?m4 ?m3 !m4 b4 b4 a4 a4

27. Correcting for input inconsistency = resolving mixed initiatives • Decide on resolution strategy: select the winning initiative if they have different goals – typically using priority • Add transitions for all inputs that are not handled • Check again to ensure that result is input consistent

28. TaxiCentral operatorInterface lineInterface op acdOp acdLi li op[n] ACD Ii[m] acdTd taxiDispatch td TaxiDispatch td taxiInterface taxi taxi[l] The taxi central with interface collaborations Interface behavior is here explicitly defined by the collaborations

29. loop ref msc operatorInterface Sequence diagrams for Interface behavior operatorInterface • Global behavior • Good overview • Normally incomplete • Roles, not complete objects acdOp op msc operatorInterface op acdOp available call(caller) answer tourOrder(TourInfo) confirm end

30. loop ref msc operatorInterface Compatibility with sequence diagrams What does it mean? • Normally that roles exhibit at least the specified sequences! Is this sufficient? What about the unspecified sequences? operatorInterface msc operatorInterface op acdOp acdOp op available call(caller) answer tourOrder(TourInfo) confirm end

31. Components and interactions, e.g. SDL and MSC • The state machine behaviour shall “contain” all the interaction behaviours in addition the state machine behaviour may contain: • cases not treated by the MSCs • behaviour on other interfaces • behaviour in other services MSC (instance) behaviour tree SDL SDL (object) behaviour tree MSCs “contains” structure

32. Verifying SDL • generate all possible traces defined by the MSCs • check that all traces may execute in the global behaviour tree for the corresponding SDL (using the Validator) MSC behaviour tree MSCs SDL SDL behaviour tree

33. taxiInterface process taxi DCL ti TourInfo; td taxi process td DCL ti TourInfo; confirm tourOrder(ti) finished available unavailable available pickup pickup pickup 5 1 1 3 2 4 5 1 3 4 3 2 3 2 2 4 1 unavailable none none tourOrder(ti) tourOrder(ti) available pickup none finished confirm pickup available none none none Role state machines for complete interface behvior With resolution of mixed initiatives A class/type playing the taxi role shall have projected interface behavior equivalent to the taxi role defined here

34. session Recall compatibility and interface roles Interface roles are ”projected and distilled” behaviours visible on interfaces, or associations. Given two state machines A and B: • Derive the interface role behaviours • Project: hide and transform • Distill: gather, minimize and merge • Detect inconsistencies • Correct inconsistencies and repeat from 1 • Validate role compatibility 3 1 B 1 A 1a 1b 2 2

35. ... a basis for incremental and scalable compatibility checks ... as well as for service discovery and service selection If Class_A is typed with Call.a (alternatively by A alone) and Class_B is typed with Call.b (alternatively by B alone) Consistency of links can be checked statically and Service opportunities discovered and selected dynamically Class_A Class_B a S-roleA A B 1 b S-roleB A B 1 Call Call Call A-roles and collaborations for semantic interfaces b a y:Class_B X:Class_A 1 1

36. B 1c 2 A 1 1a 1b 2 C 1d 1 3 2b D 3 1 1b 3b 1 3 session Dual roles are fully compatible a-roles: • iff the original state machine is input consistent, then the dual a-role may be constructed, • and used to discover, compare and select compatible services, • and to partially synthesise state machines.

37. Stock Info Server requestor provider Application Sq:StockQuote Using collaborations to type interfaces Sq:StockQuote • Compatibility of collaboration roles modelchecked at design time • Conformance with contract checked for each interface at design time • Simple compatibility assurance at runtime requestor provider

38. Stock quote Sq:StockQuote requestor provider

39. Project Project Stock Info Server Compare Compare Application Conformance with interface roles Sq:StockQuote • Project component behavior to interface role behavior • Compare interface role behavior with collaboration role behavior: are theyequivalent or substitutable?

40. Equivalent state machines Informally: • Two states are equivalent if they have the same set of transitions and the next states are equivalent. • Two state machines are equivalent if their initial states are equivalent • Equivalent state machines cannot be distinguished by an external observer, and can replace each other • A role state machine can be replaced by a subtype (with the extension guarded by inputs)

41. Interfaces in Access control

42. Collaborations as semantic interfaces userAgt panel EnterCard StoreCID dp vu ap pep cep Deny Accept Card Entry PIN Entry Validate Accept Open Deny Card Entry PIN Entry ceu peu du ou au EnterPIN Deny-eject Pass-eject Validate each collaboration separately Check that both sides play compatible roles Validate the ordering (flows)

43. Arctis ESMs: dynamic interfaces of building blocks • One ESM per building block • Not one per interface • The inner behavior of a building block must be compatible with its ESM • When analyzing a composition of blocks, the inner behavior of each block is replaced by its ESM

44. Types of interfaces and roles Collaboration activity Collaboration activity role role Internal IF, ESM Internal IF,ESM cep Card Entry ceu A-role A-role Semantic Interface Semantic Interface

45. Synthesizing state machines from sequence diagrams • Direct realization: • A projection of the global behavior onto the component state machines. • Find realization problems: • Event orderings that are possible in the direct realization, but not specified. • Caused by mixed initiatives, weak sequencing, non-causal ordering • Resolve the realization problems • For incomplete sequence diagrams: • Add the omitted behavior