300 likes | 487 Views
The State of Hybrid Model-Based Testing. Michiel van Osch 21-06-2005. Index. Introduction Testing a Water Tank Controller Hybrid Model-Based Testing Conformance Relation Test-case generation Work in Progress Questions. ?“stop traffic”. ?“stop traffic”. !“red”. !“red”. pass.
E N D
The State of Hybrid Model-Based Testing Michiel van Osch 21-06-2005
Index • Introduction • Testing a Water Tank Controller • Hybrid Model-Based Testing • Conformance Relation • Test-case generation • Work in Progress • Questions
?“stop traffic” ?“stop traffic” !“red” !“red” pass Model-Based Testing Test-tool Discrete Model Discrete System
Hydrolic pressure Coffee with Milk and Sugar “brake” “1”, “2”, “1” Desired Temperature: 25o Room Temperature Hybrid Systems ABS
Lens temperature Lens temperature !”open Cooling Valve” !”open Cooling Valve” pass Hybrid Model-based Testing Test-tool Hybrid-Model Hybrid System
Challenges • Theory: • Develop Conformance Concepts • Develop Test-case Generation Algorithms • Prove Correctness and Applicability of Theory • Practice: • Implement Tooling • Exactly Model Relevant Aspects • Connect Models, Tools and Systems
Index • Introduction • Testing a Water Tank Controller • Hybrid Model-Based Testing • Conformance Relation • Test-case generation • Work in Progress • Questions
The water Tank Controller set point Tank Controller messages Water Tank Heater Temperature Sensor Tank Controller • Discrete input: Set point • Discrete output: communication • Continuous input: Temperature (from sensor) • Continuous output: Heat (from Heater)
Hybrid I/O automaton Specification (1) Tank Controller • Discrete input: Set_setpoint(sp) • Discrete output: Print(“warning”) • Continuous input: ST • Continuous output: H
Specification (2) ST H ST H input output input output Heater OFF Heater ON
Set_Setpoint(sp) Set-point ST Temp. Sens. Print(..) Message H Heat Test Setup Tank Controller Temperature Sensor Heater Specification Test-tool Implementation
Tank Controller Temperature Sensor Heater Testing... Test-tool Specification Implementation Temp. Sens. ST Heat H Specification Implementation
Tank Controller Temperature Sensor Heater Temp. Sens. ST Heat H Testing… Set_setpoint(SP) Set-point Test-tool Specification Implementation Specification Implementation
Tank Controller Temperature Sensor Heater Temp. Sens. ST Heat H Testing… Test-tool Specification Implementation Specification Implementation
Tank Controller Temperature Sensor Heater Temp. Sens. ST Heat H Testing fail Print(“warning”) “error” Test-tool Specification Implementation Specification Implementation
Tank Controller Temperature Sensor Heater Temp. Sens. ST Heat H Testing… fail Test-tool Specification Implementation Specification Implementation
Index • Introduction • Testing a Water Tank Controller • Hybrid Model-Based Testing • Conformance Relation • Test-case generation • Work in Progress • Questions
Conformance A hybrid implementation is input-output conform a hybrid specification if for all observable traces of the specification: The set of possible output actions and trajectories, after the execution of a trace on the implementation, is a subset of the set of possible output actions or trajectories, after the execution of a trace on the specification.
Some Decisions • Defined on Hybrid I/O Automata • Input Action Enabled • Input Trajectory Enabled • Non-zeno • State Based: possible output actions and trajectories in a certain state • Input and output trajectories are treated together • There is no quiescence observation (as of yet)
Conformance relation We Formally Define: • A hybrid trace = 0 a11 a2 …as a sequence of discrete actions and trajectories of continuous variables (alternating) • A after : the set of states in which A can be after executing trace • out(q): the set of output actions or trajectories possible in state q • Out(Q): the set of output actions or trajectories possible in the set of states Q Then implementation HIOA i is hiocospecification HIOAs if: i hioco s if traces(s): out(i after ) out(s after )
Examples !a !a !c Correct! !b !b specification implementation Correct! q q specification implementation
Examples !a !a !c Incorrect! !b !b specification implementation Incorrect! q q specification implementation
Test-Case generation & Execution • Input actions: Select and stimulate • Output actions: Observe and validate • Trajectories: • Select a time up to which to apply input • Select an input trajectory to use for stimulation • Stimulate and observe output variables : • Until the end of the trajectory, without observing a mismatch between observed and specified output trajectories, in this case continue • Until an output action is observed, in this case validate the output action and if correct, then continue • Until a mismatch between observed and specified output variables occurs • As long as verdict “fail” is given, the verdict “pass” can be given
Test-case generation fail ?Set_setpoint(SP) {SP > ST} pass fail !Print(“error”) !Print(“warning”) pass fail
Index • Introduction • Testing a Water Tank Controller • Hybrid Model-Based Testing • Conformance Relation • Test-case generation • Work in Progress • Questions
Coffee temperature “1”,”2” When no coffee comes out of the machine, temperature Is not continuously 0! While applying an input trajectory and observing the output, States change q q’ Work in Progress (1) • Does a continuous variable always have a value? • State Changes During a Trajectory
What does this mean? Work in Progress (2) • Quiescence • Is quiescence a property on the discrete actions of the system or also on continuous behavior of the system? • Notation
Current & Future Work • Further Develop these Concepts (Formalization, Theorems, Proofs) • Make Concepts Workable in Practice • Sampling of continuous behavior • Selecting Realistic Input Trajectories • Deal with Uncountable Many Possible Input and Output Trajectories and States • Deal with Inaccuracy of Observed Output • Implement Proof of Concept