Model-based Testing and Monitoring for Hybrid Embedded Systems Li Tan Jesung Kim Oleg Sokolsky

1. Model-based Testing and Monitoring for Hybrid Embedded Systems Li Tan Jesung Kim Oleg Sokolsky Insup Lee University of Pennsylvania November 2004 System Design Research Laboratory

2. Outline • Motivations and overview • Model-based testing. • Generating model-based testers. • Model-based monitoring • Synthesizing model-based monitors from logic specifications • Instrumenting models • The case study on SONY AIBO dog • Design-level validation • “on-board” validation • Conclusion System Design Research Laboratory

3. The Goal • Goal: apply the state-of-the-art runtime verification technique to model-based embedded system design and validation domain. • Requirements: • Incorporating the rigid specification of the system properties using temporal logics. • Using the existing techniques in model-based design to reduce the cost of migration. • Supporting both design-level and implementation-level validations. • The procedure can be fully automatically by a set of software tools. System Design Research Laboratory

4. System model Coverage Criteria/ environment constrains mMEDLspecification I. Create Model-based Tester mPEDL event definition P2C instrumentator M2C Synthesizer Tester model II. Instrument the Model Instrumented model III. Synthesize Model-based Monitor Monitor model IV. Design Level Validation Self-testing and Self -monitoring Model Model Simulator CodeGenerator Self-testing and Self-monitoring Code Road Map Five steps towards model-based self-testing and self-monitoring embedded systems V.Implementation Level Validation System Design Research Laboratory

5. Embedded System Modeling with Hybrid Automata A hybrid automaton A={S, V, T, G, W, D, I, s0} extends a EFSM {S, V, T, G, W, s0} with continuous behaviors • S is the set of modes • V is the set of variables • T=V£ V is the set of transitions • G assigns each t 2 T a guard, a predicate over V. • W assigns each t 2 T an assignment for Vµ V. • D assigns each s 2 S a set of differential equations on V. • I assigns each s 2 S an invariant, a predicate over V. • s02 S is the initial mode. System Design Research Laboratory

6. An example: Dog Head Controller • Hybrid automata can be composed concurrently. • Hybrid automata can be composed hierarchically. • A mode can be a collection of submodes. || >10? x 45? x  . x -45 : The visibility of the ball x: The angle of the head : The angle of the ball ||  10? System Design Research Laboratory

7. Environ. Model System Model Tester Model + Simulator No Determination Coverage Criteria Coverage Checker Yes Test Trace Model-based Tester A model-based tester emulates an environment that supplies a test case Environment /hardware constrains System Design Research Laboratory

8. Runtime verification Runtime verification (monitoring) is to check the execution of a software program against its logic specification Java Programs MEDLspecification MEDL compiler Program Instrumentation Monitoring script events Java Programs +Filter Monitor Computer System Design Research Laboratory

9. System Automaton System Automaton Observer Monitoring Automaton Model-based Runtime Verification MEDLspecification Model Instrumentation Monitor Synthesizer Instrumented Model Simulator System Design Research Laboratory

10. mMEDL: expressing your properties • MEDL (Meta Events Definition Language) is a linear interval temporal logic for specifying safety properties. • MEDL is initially introduced for monitoring Java programs in MaC (Monitor and Checking) System [KKL01] • Syntax • Defined on conditions, events, and expressions. C := [E, E) | : C | C && C | C||C | Q  Q E := e | start(C) | end(C) | E || E | E&&E | E when C Q := time(E) | c |Q }Q Where e is primitive event, c is a constant, 2 {>, <, =}, and }2 {*,/,+,-} . System Design Research Laboratory

11. Semantics by example • Informal Specification: if the dog loses the ball 50 seconds after the ball becomes visible, an alarm should be raised. • mMEDL script Begin import event isVisible, isInvisible, track,lost; condition visible= [isVisible, isInvisible); event becameTruelost= lost when visible; alarm lostTrack=start (time(becameTruelost)-time(isVisible)>50); End • isVisible, isInvisible, track, and lost are primitiveevents • isVisible (isInvisible) occurs when  >10 becomes true (false). • Lost (track) occurs when |-x| > 10 become true (false). • An alarm is an event which indicates the violation of the safety requirement. . System Design Research Laboratory

12. Sequence of Primitive Events VE: last time when E occurs Synthesis is modularized T0: P=1ÆUp?VC:=1, dC:=t, P:=2 T0: P=0\and NewEvent? T:=t, P:=1 T0: P=2Æ VC=1 Æ Ve1 =t?VE:=t, P=0 || || P¹0Ç: NewEvent P¹1 P¹2 NewEvent´:(Ve1>T Ç Ve>>T Ç Ve3>T) Td: P=1Æ: Up Æ: Down?P:=2 Td: P=2Æ ~( VC=1 Æ Ve1 =t)? P=0 T1: P=1ÆDown?VC:=0, dC:=t, P:=2 Up´ (VC=0 Æ Ve1 =t) Down´ (VC=1 Æ Ve2 =t) “engine” automaton VC: current value of C dC: last time when C changes Violation is detected if VE>0 Synthesizing model-based monitor 0: import events e1,e2, e3 1: condition C=[e1, e2) 2: alarm E= e1 when C System Design Research Laboratory

13. mPEDL: define primitive events • Primitive events report the status change in the system model. • Primitive events are formally encoded in mPEDL (model-based Primitive Event Definition Language) • Events are defined as the changes on some predicate. MonScr Dogexport event isVisible, isInvisible, lost, track;monobj int dog.vision;monobj int dog.ball_pan_delta; event isVisible= start (dog.vision>10);event isInvisible= end (dog.vision>10);event lost = start (dog.ball_pan_delta>10);event track = end (dog.ball_pan_delta>10); End System Design Research Laboratory

14. Observer |-x| >10?Vlost=t || >10?VisVisible=t |-x|10?Vtrack=t ||10?VisInvisible=t Model Instrumentation • Model Augmentation • A observer automaton is concurrently composed with the model. • The structure of the model will not be changed. || >10? x 45? x -45 ||  10? System Design Research Laboratory

15. Design-level validation • The composition of instrumented model, tester, and monitor forms a self-validating Charon model • Tester supplies test trace during simulation • Model-based tester emulates the movement of a “virtual” ball by changing input variables b and q • The occurrence of an event is indicated by the changes on the event variables during simulation (a) Alarm Detection System Design Research Laboratory

16. Design-level validation (b) Primitive Events Emitted by Instrumented Model (c) The simulation trace of monitor System Design Research Laboratory

17. Tester Model-based Tester Model-based Monitor Generated Code Monitor Modular Code Generation Link as needed Implemtation-level validation System Model System Design Research Laboratory

18. Conclusion We proposed a framework for testing and monitoring model-based embedded systems. • The property being verified is encoded in a temporal logic. • It doesn’t require the changes on existing model-based design environment. • The framework is for both design-level and implementation-level validations. • It produces a self-testing and self-monitoring code for embedded system • A set of tools are developed to automate the process. System Design Research Laboratory

19. Thank you! System Design Research Laboratory

20. Computation overhead • The computational overhead of tester and monitor on simulator (1000 step @ 0.001 sec/step) • On SONY AiBo Dog, such slowdown doesn’t break real-time constrains. • The controller program is a real-time system • The slowdown is tolerated because the extra cost of tester and monitor can be observed within each integration step. System Design Research Laboratory

21. Implementation-level validation • The alarm is used for calling the external functions to perform visible actions. • “play” function is called when an alarm lostTrack is detected • The space overhead of tester and monitor. System Design Research Laboratory

22. MEDL semantics Interpreting MEDL on runs of hybrid automata, • A condition C maps a time period with true, false, or ?. If C is, • [E1, E2). C is true from event E1 to E2 (not included) • C1&&C2. C is true when C1 and C2 are both true. • C1||C2. C is true when either of C1 and C2 is true. • : C1. C is the dual of C1. • An event E maps a time instance with true or false. If E is, • start(C). E occurs at the time C becomes true. • end(C). E occurs at the time C becomes not true. • E1||E2. E occurs when either E1 or E2 occurs. • E1&&E2. E occurs when both E1 and E2 occur. • E1 when C. E occurs when E1 occurs and C is true • Q is an expression. If Q is, • time (E). Q’s value is the latest time E occurs. • Q1} Q2. Q’s value is Q1} Q2. . System Design Research Laboratory

23. Related works and future directions • Related works: “platform-specific” approach v.s. “general purpose” approaches. • NASA Ames: Java PathExplorer • UPenn: MaC tools • Works on synthesizing finite model/test-oracle from the formal specification [DilRam96,GiaHav01] • Ongoing and future researches • Optimizing monitor-synthesizing algorithm. • Approximating general linear temporal property as monitoring automaton. • Testing and monitoring the code from conventional sources. System Design Research Laboratory

24. Computation overhead • The computational overhead of tester and monitor on simulator (1000 step @ 0.001 sec/step) • On SONY AiBo Dog, such slowdown doesn’t break real-time constrains. • The controller program is a real-time system • The slowdown is tolerated because the extra cost of tester and monitor can be observed within each integration step. System Design Research Laboratory

25. Itemized space overhead The space overhead of tester and monitor. System Design Research Laboratory