Supporting Model-Based Validation at Run-time

1. Supporting Model-Based Validation at Run-time Insup Lee and Oleg Sokolsky Department of Computer and Information Science University of Pennsylvania MDES 2003

2. Software Development Process • Requirements capture and analysis • Informal to formal • Consistency and completeness • Assumptions and interfaces between system components • Application-specific properties • Design specifications and analysis • Formal modeling notations • Analysis techniques (simulation, model checking, equivalence checking, testing, etc.) • Abstractions • Implementation • Manual/automatic code generation (refinement) • Validation • Testing • Model extraction and verification • Run-time monitoring and checking Requirements Design specification Implementation MDES 2003

3. Model-based validation at run-time • Run-time verification is to assure the compliance of an execution of a system with its formal requirements. • This is achieved by monitoring the current execution and checking properties. • The monitoring and checking (MaC) system can • detect incorrect execution of an application • predict error and steer computation • collect statistics of actual execution (e.g., QoS validation) MDES 2003

4. System Implementation Monitoring Refinement Implementation System Filter Event Recognizer + Checker Communication Run-time Check MaC Methodology System Spec Formal verification Requirements Spec Design MDES 2003

5. The MaC Languages • Meta Event Definition Language(MEDL) • Describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. • Target program implementation independent. • Primitive Event Definition Language (PEDL) • Maps the low-level state information of the system to high-level events. • Provides primitives to refer to values of variables and to certain points in the execution of the program. • PEDL is defined so that events can be recognized in time linear to the size of the PEDL specification • Depends on target program implementation • Abstract state: • events • conditions • auxiliary variables • Run-time state: • control locations • object state • local variables PEDL MEDL MDES 2003

6. Design of the MaC Languages • Must be able to reason about both time instants and information that holds for a duration of time in a program execution. • Events and conditions are a natural division, which is also found in other formalisms such as SCR. • Conditions, which are true or false for a finite duration of time (e.g., is variable x >5?) • Events, which are either present or absent at some instant of time (e.g., is control right now at the end of method f?). • Need temporal operators combining events and conditions in order to reason about traces. raiseGate start(position==100) end(position==100) position == 100 1:00:10 1:00:15 1:00:30 Time MDES 2003

7. Logical Foundation • Conditions interpreted over 3 values: true, false and undefined. • [., .) pairs a couple of events to define an interval. • start and end define the events corresponding to the instant when conditions change their value. MDES 2003

8. Meta Event Definition Language (MEDL) • Expresses requirements using the events and conditions, sent by event recognizer. • Expresses the subset of safety properties. • Describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. • propertysafeRRC = IC -> GD; • alarm violation = start (!safeRRC); • Auxilliary variables may be used to store history. • endIC-> { num_train_pass’ = num_train_pass + 1; } ReqSpec <spec_name> /* Import section */ import event <e>; import condition <c>; /*Auxiliary variable */ var int <aux_v>; /*Event and condition */ event <e> = ...; condition <c>= ...; /*Property and violation */ property <c> = ...; alarm <e> = ...; /*Auxiliary variable update*/ <e> -> { <aux_v'> := ... ; } End MDES 2003

9. Property checking • A MEDL specification can be seen as an automaton with auxiliary store running on a stream of events provided by the event recognizer aux. variables MDES 2003

10. PEDL for Java MonScr <spec_name> /* Export section */ export event <e>; export condition <c>; /* Overhead reduction */ [timestamp;] [valueabstract;] [deltaabstract;] [multithread;] /* Monitored entities */ monobj <var>; monmeth <meth>; /* Event and condition*/ event <e> = ...; condition <c>= ...; End • The language maps the low-level state information of the system to high-levelevents and conditions used in describing the requirements. • Provides primitives to refer to • primitive variables • beginnings/endings of methods • Primitive conditions are constructed from • boolean-valued expressions over the monitored variables • Ex: condition IC = (position == 100); • Primitive events are constructed from • update(x) • startM(f)/endM(f) • Ex: event raiseGate= startM(Gate.gu()); MDES 2003

11. Java-MaC Prototype Requirement Specification Program (Java source code) Requirements (MEDL) Monitoring Script (PEDL) Program (Java byte code) MEDL Compiler PEDL Compiler Instrumentation Information Filter Generator (JTREK) Static Phase Instrumented Target program Filter (Compiled PEDL) Event Recognizer (Compiled MEDL) Run-time Checker Dynamic Phase MDES 2003

12. Specifications for Stock Clients MonScr StockClient export event startPgm, periodStart, conFail, queryResend, oldDataUsed; monmethvoid Client.main(String[]); monmeth void Client.run(); monmeth void Client.failConnection(ConnectTry); monmeth Object Client.retryGetData(int); monmeth Object Client.processOldData(); event startPgm = startM(Client.main(String[])); event periodStart = startM(Client.run()); event conFail = startM(Client.failConnection(ConnectTry)); event queryResend = startM(Client.retryGetData(int)); event oldDataUsed = startM(Client.processOldData()); End ReqSpec StockClient import event startPgm, periodStart, conFail, queryResend, oldDataUsed; var long periodTime; var long lastPeriodStart; var int numRetried; var int numConFail; alarm violatedPeriod = end((perioidTime’ >= 900) && (periodTime’ <= 1100)); alarm wrongFT = oldDataUsed when ( (numRetries’ < 4)|| (numConFail’ < 3)); startPgm -> {periodTime’ = 1000; lastPeriodStart’ = time(startPgm) -1000; numRetries’ = 0; numConFail’ = 0;} periodStart ->{ numREtries’ = 0; numConFail’ = 0; periodTime’ =time(periodStart)-lastPeriodStart; lastPeriodStart’ = time(periodStart);} ... End MDES 2003

13. Extensions for Embedded Systems • MaC based on hybrid system models • Distributed MaC • Steering-based adaptation • Model-based profiling MDES 2003

14. Monitoring of hybrid systems • Embedded systems often contain continuously evolving as well as discrete components • Monitoring of continuous entities introduces additional challenges • Monitoring of discrete systems is event-driven, which is not enough for continuous changes • “Continuous monitoring”, in practice, involves sampling • Sampling rates for each continuous entity can be obtained from a hybrid model of the embedded system • Deriving properties of embedded systems: • Use a hybrid model as an oracle, driven by the observations MDES 2003

15. Distributed MaC • Run-time monitoring and checking of distributed systems • Local monitors and checkers • How to decompose a global property into locally checkable properties, i.e., • Given the set of nodes, N1,…, Nm and a global property Pg, find locally checkable properties, P1 ,…, Pm such that if Ni detects the violation of Pi, then Pg is violated. • Tradeoff between amount and frequency of communication and timely detection of property violation MDES 2003

16. Steering-based Adaptation MDES 2003

17. Steering-based adaptation • Steering provides feedback from the monitor to the system • When can a system be effectively steered? • the system is designed for run-time adjustments • e.g., Simplex architecture • the system naturally offers a degree of tolerance • e.g., routing protocols: flush buffers when performance decreases • Need timing guarantee over feedback loop • Real-time Java MDES 2003

18. Model-based Profiling • Collect statistics during monitored executions • Over multiple runs, such statistics will provide dependability and performance measures • Possible uses: • Collect coverage • Successful runs serve as tests • Enhance “synthetic” test suites • Infer dynamic interfaces (e.g. interface automata) • Record input and output patterns of successful executions • Helps in reusing the component in different contexts MDES 2003

19. Summary • The MaC architecture provides a lightweight formal methodology for assuring of the correct execution of a target program at run-time • Rigorous analysis, Flexible, Automatic • Identified several possible extensions to MaC • Reduction of monitoring overhead • Port the MaC architecture to platforms other than Java • http://www.cis.upenn.edu/~rtg/mac MDES 2003

20. Q & A MDES 2003