180 likes | 392 Views
Traffic-aware Stress Testing of Distributed Real-Time Systems based on UML Models. Vahid Garousi , Lionel Briand, and Yvan Labiche (The results of the first author’s PhD work). Software Quality Engineering Laboratory (SQUALL) Department of Systems and Computer Engineering Carleton University
E N D
Traffic-aware Stress Testing of Distributed Real-Time Systems based on UML Models Vahid Garousi, Lionel Briand, and Yvan Labiche (The results of the first author’s PhD work) Software Quality Engineering Laboratory (SQUALL) Department of Systems and Computer Engineering Carleton University Ottawa, Ontario, Canada ICSE 2006, Shanghai, China Friday, May 26, 2006
Motivations • Distributed Real-time Systems (DRTS) are becoming more important to our everyday life. • Examples include command and control systems, aircraft aviation systems, robotics, and nuclear power plant systems. • A study on the sources of failures in the United States' Public Switched Telephone Network, as a large DRTS • Study period: 1992-1994 • Main Results: • 6% of outages were overloads • Overloads led to 44% of service downtime • Result: Although overload situations do not happen frequently, the failure consequences they result into are quite expensive. • Furthermore, there are reports indicating the high risk of failures due to network overload in the context of Distributed Control Systems (DCS) systems. • Therefore, we need to stress test DRTS w.r.t. overloads.
Real-time deadline=2 seconds 2 seconds m1 m1 An acceptable execution A deadline miss (real-time fault) Our Goal and Strategy in a Nutshell m1 Our stress test goal is to maximize the probability of such real-time faults by executing stress test requirements derived from UML models of a SUT.
Input Models • Sequence Diagram: Models the behavior of a SUT • Class Diagram: Used to estimate the data size of messages in SDs Standard UML diagrams Used in several OO methodologies, e.g., COMET (Concurrent Object Modeling mEThod) • System Context Diagram • An extension to class diagram which • Models the interaction between actors and a system • Used to model the number of multiple invocations of a SD, e.g., several sensors can trigger a SD. Our specific diagrams • Modified Interaction Overview Diagram (MIOD) • An extension to UML 2.0’s Interaction Overview Diagrams • Models the sequential and conditional constraints among SDs • Enables modelers to specify the valid sequences of SDs in a SUT, and our stress test methodology to derive valid stress tests • Network Deployment Diagram (NDD) • One of our specific package-diagram-like diagrams that models the network topology of a SUT
Modified Interaction Overview Diagrams (MIOD) • MIOD of a simplified ATM system:
Network Deployment Diagram (NDD) • A network topology: • Using UML packages to model this network topology:
Test Models * * *: Discussed in this presentation.
Test Models - Concurrent Control Flow Graph (CCFG) Concurrent Control Flow Graph (CCFG)
Test Models – Network Traffic Usage Pattern DataSize(msg)=2*(8* 1000)= 16,000 Bytes DataSize(reply)=8* 1000= 8,000 Bytes
We are here now Approach: Where we are now
Case Study • Some of the Target Systems • Distributed Control Systems (DCS) • Supervisory Control and Data Acquisition (SCADA) Systems • Application in: • Electrical power distribution grids and generation plants • Environmental control systems • Traffic signals • Our prototype system: SCADA-based Power System (SCAPS): • Overload monitoring in a power distribution grid • Detection of separated power grids
Case Study – Results • Baseline of comparisons for our stress tests: • Operation Profile-based Test Cases (OPTCs) which are derived from an operational profile. • Reminder: The operational profile is defined as the expected workload of a system. • 500 OPTCs and 500 ST cases were derived and executed.
Conclusions • A model-driven, stress test methodology • Aims at increasing chances of discovering faults related to distributed traffic. • Based on UML 2.0 model of a system • Relies on a careful identification of control flow paths and the distributed traffic they entail. • First results are promising • Stress test cases significantly increase the probability of exhibiting distributed traffic-related faults. • A prototype tool has been developed to derive stress test requirements
Some of the Future Research Directions • Investigating stress testing with respect to other resource types: e.g., CPU and memory. • Specifying stress test requirements and process using the UML 2.0 Testing Profile • Visualizing, and analysis of test requirements • Facilitate traceability between analysis, design, and testing artifacts • Improve test automation • Stress testing more complex distributed systems and investigating the effectiveness of the methodology
Thank you! • Any questions?
Test Models - Concurrent Control Flow Graph (CCFG) Concurrent Control Flow Graph (CCFG)