An Automata-based Approach to Testing Properties in Event Traces

1. An Automata-based Approach to Testing Properties in Event Traces H. Hallal, S. Boroday, A. Ulrich, A. Petrenko Sophia Antipolis, France, May 2003

2. Outline • Motivation • Event traces • Problem • Our approach • Implementation • Case study • Conclusions and extensions

3. Motivation • Analysis of distributed systems is complex and costly • Asynchrony • Lack of global timing • Absence of reference specification • A practical solution is to instrument the system to generate traces of events that can be visualized and analyzed further • This solution can be used to debug the system • During development • After deployment

4. distributed system of processes ... Event Event Monitoring Tool Trace Visualization Tools Visualization Vs Analysis Tools • Visualization tools facilitate the manual inspection of collected traces • elaborate ad-hoc algorithms • more efficiency • more efforts • reuse an existing model checker • more expressiveness • less efforts • Analysis tools automate the verification of properties in the traces Analysis Tools

5. Trace Analysis Problem • Given • A distributed system under test (SUT) • Some properties Verify whether the SUT satisfies the properties • Solution • Monitor the SUT and collect an execution trace • Model the collected trace • Use an existing model checker to verify the properties

6. Trace • Distributed processes generate local traces • Local events: state update, parameter change • Communication events: message exchange, RMI, RPC • Local traces are sequential • Communication • Asynchronous: send and receive events • Synchronous: rendezvous events • Point-to-point communication • Each message has a send and a receive in the trace • Each rendezvous involves at least two parties

7. Event Traces • Event ordering induced by local orders i and point-to-point communication • A trace is a partially ordered set E of all events • Causality relation on events  • If a i b then a  b • for every message m,send(m) receive(m) •  is transitive: If a b and b c then ac • Event trace a tuple of local traces with an irreflexive causality relation on all events

8. n1 = 3 n1 = 4 n1 = 5 pr1 m2 m1 time m3 pr2 n2 = 4 n2 = 6 n2 = 2 Lattice of Ideals • Encodes all the possible linearizations of E • Offers an efficient way to check properties

9. Problem • Given • An event trace of a distributed system • A set of properties • How to build the lattice of ideals to verify the properties? • Monolithic approach • build the lattice explicitly • use a model checker • Modular approach • model the event trace as a system of communicating automata • build the composition of automata • prove it is isomorphic to the lattice

10. send(m) receive(m)  {send, { } send receive} Our Approach • We use finite automata to model • Local traces of processes • states are ideals • transitions are events • Message delays • We build the composition of all automata • We prove composition of automata  lattice of ideals • Use the composition automaton to verify the properties • use an existing model checker • avoid full state space search

11. Implementation • We use SDL and ObjectGEODE (OG) • We model the SUT as an SDL system • Local traces: designated processes • Local events: SDL TASK • Communication: signal exchange • How to treat the message delay automata? • Individual processes • Individual queues • SDL “SAVE” • Properties are specified in GOAL of OG

12. distributed system of processes Event Event ... Monitoring Tool Pattern Library Trace Front End to ObjectGEODE Model results: 1. Property satisfied or not 2. Scenarios Property User System Interface Specification Specification GOAL Observer SDL Model User ObjectGEODE Simulator Workflow of the Approach • Front-End tool to ObjectGEODE • System specification • Pattern specification • Library of property patterns • Parameterized GOAL observers • State-based, event-based, mixed

13. Pattern Library • Property patterns already exist • Repository of common properties • Mappings to main formalisms used in finite state verification LTL, CTL, INCA, QRE,… • Library of GOAL observers Address finiteness of traces • Encode common patterns • Class: order vs. occurrence • Name: response, universality, ... • Scope: global, before, after, ... • Parameterized GOAL specification parameters are predicates on states, events, or both

14. observer response waitp success state success; error state error; true waits P true false true last_state S S false true false true true false success waitp last_state last_state last_state last_state true false true false true true false false waits error success waitp success waitp error waits Pattern Template • Name and Intent • Response • Cause-effect relationship • Class Order • Scope Global: the entire execution • Example resource granted after request S responds to P in the execution

15. TRAYSIS • Input: XML logfile • Output: SDL model • Features • Logfile conformance check • Synchronous/asynchronous • Statistics on the model processes, channels, variables, signals,... • Model customization scalability • Access to OG

16. Property Manager Supports property specification • Easy access to library • Customize observers

17. Case Study • An implementation of the Sliding Window Protocol • Extension to the PROFIBUS protocol stack • Supports communication in distributed power control system • Properties of interest • Maximum window size is respected • Total number of unacknowledged messages less than limit • Total number of messages in transit less than limit • Execution traces are collected using protocol analyzers • We used out tool set to automatically analyze the system • We have analyzed large traces (15k –20k events)

18. Conclusions and Future Work • Formal definition of event traces • A framework to model mixed communication modes (GALS) • Automata-based approach to analyze event traces • A component based implementation of the approach • A case study: the SWP • Target more general logfiles • Enhancement of the tool set

19. Merci beaucoup!