Using Software Architecture For Code Testing

1. Using Software Architecture For Code Testing Written By: Henry Muccini Antonio Bertolino Paola Inverardi Presented By: yong wang

2. Introduction • SA is emerging as a promising approach as it is used to design and analyze complex distributed system. • Using suitable abstraction it makes large application manageable. • SA support formal modeling of a system with topologic description and behavioral one in term of component and connector

3. Introduction • Crucial part of development process is testing. It can be used to test that the dynamic behavior of program by observing the execution of the program on selected set of test case. • Specification-based testing checks the implementation Under Test(IUT) fulfils specifications, to test the important properties against which the ITU has to be tested. • This paper explains how complex systems dynamic behavior and specifications can be tested using SA.

4. Goal & approach • For complex system, it is impossible to test all class. • The goal of approach is to provide a test manager with a systematic method to extract suitable test classes for higher level of testing and to refine them into concrete test at the code level. • It is based on specification of SA dynamics which is used to identify useful schemes of interaction between system components and to select test classes corresponding to relevant architectural behaviors.

5. approach • SA dynamics are modeled by Labeled Transition System (LTS). • The approach consist of deriving suitable LTS abstractions, called Abstract LTS or ALTS. ALTS offer specific views of SA dynamics by concentrating on relevant features and abstracting away from uninteresting ones. • ALTS is a tool to understand what must be tested. It represents a model of the system

6. Overview Of Approach to SA Testing • Four Steps 1. Software Architect looks at SA dynamics from different viewpoints, defines various obs-functions over SA model each one highlighting a specific perspective of interest for a test session. 2. Applying each obs-function to LTS, an Abstract LTS (ALTS) can be generated which is reduced LTS showing only interested behaviors with respect to selected views.

7. Overview Of Approach to SA Testing 3. The software architect chooses a set of important patterns and behavior (path over ALTS) to be tested. 4. Finally these high level tests are passed to the software tester, who actually has to run the test and observe whether the current implementation “conforms” to its architectural model. deals with the execution of these tests at the code level

8. Overview Of Approach to SA Testing • Steps 1 to 3 presents a rigorous method to extract architectural tests from an SA specification • Step 4 deals with the execution of these tests at the code level

9. Definition:Labeled Transition System (LTS) • LTS consist of (S, L, S0, Sf, T) S = set of states L = set of distinguished labels (actions) denoting the LTS alphabet. S0=initial state. Sf= Final State. T = Transition Relation labeled with elements of L.

10. Case Study: The TRMCS • The Teleservice and Remote Medical Case System (TRMCS) is a system that provides monitoring and assistance to disabled or elderly people. • A typical service is to send relevant information to a local phone center so that the family, along with those concerned with medical or technical assistance, can be timely notified of critical circumstances.

11. Case Study: The TRMCS(arch) Box=components; arrow = connectors Arrow label = data elements exchange through channel

12. Case Study: The TRMCS • Four components: 1. USER send either “alarm” or “check” message to the router process. Then he/she waits for acknowledgement from the router. In this case two users can concurrently send alarms and checks. 2. ROUTER waits for signals (check or alarm) from the user. It forwards the alarm message to Server and monitors the state of the User via check message. 3. SERVER dispatches the help requests. 4. TIMER sends a clock signal at each time unit.

13. THE APPROACH • 4 steps of our approach to SA-based testing. • STEP 1 Obs-Functions Definition • STEP 2 Abstract LTS Derivation • STEP 3 Test Selection • STEP 4 Code-Level Testing

14. Step 1: Obs-functions defintion Problem: SA---> Labeled Transition System models SA dynamics. LTS provides a global, monolithic description. Vast information flattened into graphic. Extracting relevant for validation is difficult task. Aim: provide software architect with a key to decipher the LTS. Common practice is to derive a set of simplified models from global SA model which will provide different system views.

15. Step 1: Obs-functions defintion • Ideal Allow for the formal derivation from the LTS of reference models for testing, each representing a relevant pattern of behavior. Notation: Test ing criterion: each of the selected patterns of behavior. It divide LTS action L into two groups: Relevant Interaction R and Non Relevant Interaction NR. R U NR=L, R ∩ NR=null.

16. Step 1: Obs-functions defintion • Using obs-function map the relevant interactions to domain D and Non relevant interactions to τ • Obs function can be considered as hiding operator as it hides non relevant information Obs: LL’, obs(r ∈R)=d ∈D, obs(n ∈NR)=τ, L’=DU τ LTS paths If p = I1I2…..In : obs(p)=obs(I1I2…..In)=obs(I1)obs(I2)…..obs(In) We therefore have SA testing criterion an obs-function that maps the relevant LTS label.

17. STEP 2 Abstract LTS Derivation • We need a small automaton from the LTS which still expresses all high level behavior. ALTS. • The ALTS is obtained by two transformations: • By relabeling each transition τ∈ LTS in the LTS transition according to obs-function. • By minimizing the resulting automaton with respect to a selected equivalence relation. Trace-equivalence: reduce much number of τ transitions and nodes Bisimulation equivalence: system evolves step by step

18. STEP 2 Abstract LTS Derivation The abstraction is applied over this LTS with respect to selected obs-function. (LTS) The trace equivalence minimization function is applied. (ObsLTS) c) The resulting ALTS is shown in Fig (trace-based ALTS) This figure represent branch minimization. (bisimulation-based ALTS)

19. STEP 3: Test Selection • Once the ALTS is derived we move to deriving a set of complete paths appropriately covering the ALTS. • Now ALTS have less paths the tester might think to apply extensive coverage criteria on it. • But ALTS corresponds to many possible LTS paths. • Therefore less thorough coverage criteria seems more practical • Use McCabe’s test technique as it is a good compromise between arc and path coverage in the case of ALTS coverage

20. STEP 3: Test Selection • When considering what to take as the specification of an “architectural test” we have two options. • consider an ALTS complete path as the test specification. In this case, the test is specified at a more abstract level and tester instinctively focuses on testing restricted set of interactions(use) 2. identify those LTS paths of which the selected ALTS path is an abstraction. As LTS is more detailed than ALTS in this case tester will have more information about how to perform tests, but stricter requirements i.e. tester does not have much freedom in choosing the code level tests.

21. STEP 4 Code-Level Testing • In this step tester use architectural paths from step 3 to test the implementation. Two problems are encountered while doing this 1. TRACEABILITY- concerns “relating the abstract values of the specification to concrete values of the implementation”. 2. TEST execution– entails forcing the Implementation Under Test (IUT) to execute the specific sequence of events that has been selected. This is a difficult as non-determinism is introduced.

22. STEP 4 Code-Level Testing • Step 4.1 Map architectural components and actions into their low-level implementation. analyze the system implementation to understand how architectural actions have been implemented in the code by sequences of partially ordered method calls. • Step 4.2 Take into account how an ordered sequence of actions is implemented by a sequence of low-level functions. Map actions into sequences of method calls. Run sequentially and run concurrently.

23. STEP 4 Code-Level Testing • Step 4.3 Run code and evaluate execution traces with respect to the expected ones to analyze the source code conformance with respect to the architectural behavior. Nondeterministic testing: repeat the launching of a program run under some specified input conditions several times until the sequence is observed.

24. Applying the Approach • TRMCS components behavior were specified by using Finite State Process(FSP)language • The behavior of each component is modeled by one or more FSP processes, and each process is described by an LTS which is automatically generated by Labeled Transition System Analyzer (LTSA) tools

25. Applying the Approach • By running the LTSA tool on the TRMCS FSP specification • 256 states of LTS • The LTS labels of interest for analysis are: • u[i].sendAlarm_To_Router: an alarm message is sent by the user to the Router • u[i].receiveAck_From_Router: the Router Acknowledgment messages received by the user • r.[i].sendAlarm_To_Server: the Router forwards the Alarm message to the Server • sa[0].sendAck_To_Router: the Server sends the Ack to the Router • r.sendNoFunc_To_Server: the Router sends the NoFunction message to the Server

26. Applying step 1 • Informal specification: Test the flow of an Alarm message from the moment a User sends it to the moment the User receives an acknowledgment. • Corresponding obs-function: AlarmObs:

27. Applying step 1 All the interaction that involve this component FR{a1,a2,no} messages the servers receivers from the Router TR{ack1,ack2} message sent to the router ServerRegression Obs-Function

28. Applying Step 3 • After the reduction and minimization algorithms • Shaded circle represents the initial state • After the Alarm is issued, the system reacted with one of the possible actions: • Elaborate the Alarm and send back an Ack, (Alarm1Dispatch) • Receive another message from another user, (Alarm2Dispatch)

29. Applying Step 3 • List of test paths derived according to McCabe’s coverage criterion follow corresponding test sequences

30. Reference to the ServerRegression Observation Associated with ALTS

31. With reference to the ServerRegression criterion, Fig 4b is obtained • 4b shows the interactions involving the Server component • Receives Alarm1 (FRa1) or Alarm2 (FRa2) from the Router • Then, receive other Alarms or send back the Ack (Track) • It can also receive the NoFunc message from the Router (FRno) • Parts 1b, 3b, 7b, and 8b are verified that the NoFunc message (FRno in fig. 4b) can b received by the Server

32. Applying Step 4.1: • Test sequence generated from AlarmObs ALTS has 3 test components (User1, User2, and Router) • Two Architectural events are • AlarmDispatch and AckDispatch • Understand how the source code implements these architectural actions • AlarmDispatch – represents a system input • E.g. user pushes the Alarm button of the device • The device will start by analyzing the actions associating with the pushing the alarm button • Then, the Alarm functionality implements

33. AckDispatch • The Alarm message is written into a socket • The Router periodically reads the socket for new messages • The Ack function is implemented • The Router writes into a socket and the user waits on a defined socket port

34. Objects and methods implements the Alarm • Fig. 5a: the User object creates the User Object • If_1 condition is verified • Afterward, (if_2), User presses the Alarm button • User then creates a ClientConnection • SendNMessages (SNM) objects • User calls the send() method of the SNM object • It then tries to create the socket • If the socket is created (if_3), SNM finally writes the Alarm message into the socket

35. From the Router side • Router graphical interface object (xRouter) creates a new instance of the MasterRouter object • In turn creates the ServerConnection object • Tries to create a ServerSocket • A process is then activated • Continuously checks the ServerSocket port • A new message is written in the socket (if_b) the socket is read and the Alarm is received

36. Applying step 4.2 • Analyze how an architectural test(sequence of actions) is implemented by the code. • TS3a:Alarm1Dispatch.Ack1Dispatch.Alarm2Dispatch.Ack2Dispatch. Alarm1Dispatch happens before Ack1Dispatch. • Code level two option: • Code-level sequences implements AlarmiDispatch and AckiDispatch run sequentially. • Low-level scenarios can be run concurrently. Method call may interleave.

37. Applying step4.3 • Execute the test sequence and verify the system implementation behaves as described by our SA test. • According to TRMCS implementation P, build an instrumented version P’ to store information about the execution of SM(synchronization method). Introduce the “synchronization Method” file. when an SM is called, a print operation is performed in this file which capture the following information:<SM ID, User ID, SM instance, time> • Running P’ several times with difference input order, Get file report. Each alarm operation is always immediately followed by the relative acknowledgment.

38. Some Considerations • Difficulty of Application Step 1.Obs-functions: is an empirical task based on a SA’s experience Tasks could be made easier through classification of observations By assigning semantics to the messages exchanges Step 2.Deriving ALTS: ALTS generation can be automated Step 3. Test Classes Selection: coverage criteria like McCabe’s test technique could be automated easily – extracting only paths that expose different behaviors

39. Step 4:Code-level Testing: Testers used to generate test cases Used to trace high-level info into code-level tests 4 Issues need to understanding which classes and methods that implement an architectural functionality More than one sequence can implement the desired behavior • This can be handled by requiring the intervention of the tester who manually evaluates each point of decision • And, performs a reasoned choice regarding which selection is most relevant for testing • Interactions are considered in SA description, not computations • Expected behaviors are described only • But, the implementation has to handle the exceptional behaviors

40. B. Approach Generality • Approaches are put into practice as well as independently • ADL is selected to describe the SA dynamics through LTS • Coverage criteria is selected to extract a limited number of ALTS paths • Implementation is used following the OO paradigm

41. Constraint: • SA dynamics is described via an LTS conforming the given definition • Different ADLs can be used to describe the SA • Different ALTS coverage criteria can be applied • Different implementation languages and programming paradigms can be used

42. Related Work Approach which is presented makes use of the following concepts. Testing concurrent and real time systems Conformance based testing Tracing information Architectural testing

43. Related Work Cont • Most of the work has been done on Testing Concurrent and Real Time, both are specification and implementation based with focus on unit testing. • Objective was to derive test plans for integration testing using the LTS , abstraction and event sequences used in different way. • Approach to defining ALTS path for high-level test classes is same as Carver and Tai ( Syn-sequence) but constraint to specification based testing of concurrent programs.

44. Related Work Cont. • ALTS path for SA restricts the event of a concurrent program when test are selected, the main difference is in the distance of abstraction level of the reference model from the implementation. • Selection of a test suit based on specification of implementation under test, and which is assumed to be given in form of Finite State Machine (FSM) • Transition tour, W-method, DS-method, UIO method and UIO methods were proposed

45. Related Work Cont. • TAG (tan et al) test generation tool based on FSM and generated by given specifications but the draw backs of this tool were due to use of FSM with limited number of states and impossibility of ensuring complete coverage. • Software system using statecharts: natural language scenarios were designed through statechart explaining information on test case generation, thus enabling test cases generation through path traversal.

46. Related Work Cont. • Bogdanov presented a method to extract test cases from state charts to prove the implementation is behaviorally equivalent to design. • Das-Boot tool were also presented to generate test suit for OO (object oriented) systems through statechart.  • Studies have carried out by Fujiwara, Bochmann and others to generate automatically test suits to test the conformance of an implemented system. • Petrenko uses FSM to drive a conformance testing, where system specification (S) its implementation (I) are formalized using FSMs to define “Conformance relations” (equivalence, quasi-equivalence, reduction) and Wp-method is used to generate test sequences.

47. Related Work Cont. • Tretmans presented LTS-based conformance testing, specification (S) using LTSs, implementation (I) using LTSs or IOTS (Input/output Transition System) and tests (T) were formalized using LTSs: Given S and possible implementation I. IMP (Implementation relations) to correlate S with I where I conforms to S iff I is “imp”with respect to S. • Fernandez et al. formalize the specification, implementations and test purpose by using IOLTS (input/output LTS). Automatic, on-the-fly generations of test cases were projected in form of TGV (test generation and verification) environment.

48. Related Work Cont. • They also used the SA-derived LTS as reference model but in different way. Implementation relations (conf, ioconf, ioco) were also defined and test whether the implementation is correct with respect to specification, based on the models described earlier. • Egyed proposed the “Tracing Information” where showed the way to detect traceability between software systems and their models, including some techniques. • Work has also been done in bridging the gap between software architectures and its requirements.

49. Related Work Cont. • The Architecture Testing has become more interesting and the authors have defined six criteria for architecture-based testing, adapting and specification based approaches. • They also analyzed the advantage of SA-level testing for reuse test and test further functional properties. • Harrold presented the approaches for using software architecture for effective regression testing. • She also proposed the architecture-based integration testing approach, which includes architecture traceability, simulation and slicing.

50. conclusion • The research has been carried out on SA philosophy, methods and tools can be useful within software development process • Mainly targeted the using of SA descriptions to improve conformance testing of a large system. • This literature was heavily relied upon the Software Architecture and Specification based testing. • Includes tools for design and analysis but very little had been said about SA- based testing.