Testing functionality independent on history of system-environment interactions

1. Study 2 Testing functionality independent on history of system-environment interactions

2. Specifications Спецификации Оракул Оракул Oracle Медиатор Медиатор Mediator Specifications Test sequence generator Test engine Steps of test development using the tool: Test actioniterator Testscenario • Specification development • Mediatordevelopment • Test scenario development Specifications Mediators on SeC Target system

3. Specification functiontrunc_spec specification mix_num trunc_spec(mix_num* mn) {pre {return mn->denom > 0 && ( mn->integral >= 0 && mn->num >= 0 ||mn->integral <= 0 && mn->num <= 0); }post {return !memcmp(&spec_trunc_spec, mn) && mn->denom == @(mn->denom) && abs(mn->num) < mn->denom && mn->integral * mn->denom + mn->num == @(mn->integral) * @(mn->denom) + @(mn->num); }}

4. signature Specification function specification<return type> <function name>(<type 1> <arg name 1>,...,<type n> <arg name n>) {pre { . . . }coverage { . . . }post { . . . }} body

5. Pointers as parameters of specification functions specification f( int* i, int* iarray ){ ... if(*i == 0) ... ... if(iarray[3] != 0) ... ... }

6. Specification types • void* create(&<type name>_type,…); • int compare(Object*,Object*);bool equal(Object*,Object*); • void* copy(Object*,Object*);void* clone(Object*); • String* toString(Object* ref) • <specification type> <var>;<specification type>* <ptr>; • Integer, Double, Char, String, Set, List, Map

7. Definition of specification type specificationtypedef <base type> <spec type> = {}; specificationtypedef <base type> <spec type> ={.init= pointer_to_initialization_function, .copy= pointer_to_finalization_function, .compare= pointer_to_comparing_function, .to_string= pointer_to_string_mapping_function, .enumerator = pointer_to_reference_counting_function, .destroy= pointer_to_destroying_function}; specificationtypedef <base type> <spec type>;

8. Specification typeMixNum intcompare_MixNum(MixNum* left, MixNum* right) { int l = left->num * right->denom , r = right->num * left->denom; if(left->integral != 0) l += left->integral * right->denom * left->denom; if(right->integral != 0) r += right->integral * left->denom * right->denom; return l – r;} specificationtypedef mix_num MixNum = {.compare = (Compare)compare_MixNum};

9. Signature of specification function trunc_spec specification MixNum* trunc_spec(MixNum* mn);

10. Structure of a body of specification function specification<return type> <function name>(<list of parameters>){pre { . . . }coverage<coverage name> { . . . }post { . . . }}

11. Precondition inputs (arg,*arg) call Target system outputs(*arg,result) specification <return type> <function name>(...){pre {if(...) /* values of inputs meet precondition requirements */return true;elsereturn false; }. . .}

12. Preconditionof specification function trunc_spec specificationMixNum* trunc_spec(MixNum* mn){pre {return mn->denum > 0 && ( mn->integral > 0 && mn->num > 0 || mn->integral < 0 && mn->num < 0 || mn->integral == 0 && mn->num == 0 );}...}

13. Type invariants /* invariant of C type */invariant typedef <base type> <type name>; invariant(<type name> value) {if(/* invariant are heldfor the value of value */)return true;elsereturn false;} /* invariant of specification type */invariantspecification typedef <base type> <spec type> ={...}; invariant(<spec type> *r) {...}

14. Automatic checking invariants forspecification function arguments inputs (arg,*arg) • Prevalues call outputs(*arg,result) • Postvalues invariant(<arg type> value);invariant(<res type> value);specification <res type><function name>(<arg type>* arg){pre{. . .}coverage {. . .}post{. . .}} Target system invariant(*arg); invariant(*arg); invariant(<return result>);

15. Invariant of specification typeMixNum specification MixNum* trunc_spec(MixNum* mn) {pre {return mn->denum >0 && ( mn->integral >= 0 && mn->num >= 0 || mn->integral <= 0 && mn->num <= 0 || mn->integral == 0 && mn->num == 0 );}...} invariant specification typedef mix_num MixNum; invariant(MixNum* mn) {return mn->denum > 0 && ( mn->integral >= 0 && mn->num >= 0 ||mn->integral<= 0 && mn->num <= 0|| mn->integral == 0 && mn->num == 0);}

16. Coverage criteria inputs (arg,*arg) call Target system outputs(*arg,result) specification <return type> <function name>(...){pre {. . .}coverage <coverage name> { . . .if(...) /*with given values of arguments the functionality branch <branch id> is covered*/return {<branch id>,<branch string>}; . . . }post {. . .}}

17. Coverage criteria completeness Any set of argument prevalues should belong to one of functionality branches defined be coverage criteria coverageС{if(...) /* Condition 1 */return { BRANCH_1, "1st branch" };elseif(...)/* Condition2*/...elsereturn { LAST_BRANCH, "Last branch" };}

18. Coverage criterionof specification function trunc_spec specificationMixNum* trunc_spec(MixNum* mn){coverageС {if(mn->integral == 0 && mn->denom > mn->num)return {PROP_FRACT, "Proper fraction"};if(mn->integral == 0 && mn->denom <= mn->num)return {IMPROP_FRACT, "Improper fraction"}; if(mn->integral != 0 && mn->denom > mn->num)return {PROP_MIXED, "Proper mixed number"};else/*(mn->integral!=0 && mn->denom < mn->num)*/return {IMPROP_MIXED, "Improper mixed number"}; }...}

19. Postcondition inputs (arg,*arg) call Target system outputs(*arg,result) specification<return type> <function name>(...){pre {. . .}coverage <coverage name> { . . . }post {if(...) /*values of outputs meet postcondition requirements */return true;elsereturn false; }} point of affecting

20. Prevalues of arguments and returned result in poctcondition inputs (arg,*arg) call Target system outputs(*arg,result) specification int func(int *u){post {return@(*u) != 0 && func == *u }}

21. Postconditionof specification function trunc_spec specificationMixNum* trunc_spec(MixNum* mn){coverageС {if(mn->integral == 0 && mn->denom > mn->num)return {PROP_FRACT, "Proper fraction"};if(mn->integral == 0 && mn->denom <= mn->num)return {IMPROP_FRACT, "Improper fraction"}; if(mn->integral != 0 && mn->denom > mn->num)return {PROP_MIXED, "Proper mixed number"};if(mn->integral != 0 && mn->denom < mn->num)return {IMPROP_MIXED, "Improper mixed number"}; }post {return trunc_spec->integral == mn->integral && trunc_spec->denom == mn->denom && trunc_spec->num == mn->num /* !memcmp(trunc_spec, mn) */ && mn->denom == @mn->denom && abs(mn->num) < mn->denum && equal(mn, @mn); }}

22. Mediators Оракул Оракул Oracle Mediators onSeC Медиатор Медиатор Mediator Test sequence generator Test engine Steps of test development using the tool: Test actioniterator Testscenario • Specification development • Mediatordevelopment • Test scenario development Specifications Mediators on SeC Target system

23. Definition of mediators onSeC mediator <mediator_name> forspecification <function name>(<argument list>) {call {/* mapspecification args to implementation ones */ ...if(.../* if something wrong */) setBadVerdict(<string>); /* implementation call */ ... /* map implementation results to specification ones */ ...if(.../* if something wrong */) setBadVerdict(<string>);return <specification result>;}}

24. Mediatorof specification function trunc_spec mediator trunc_media forspecification MixNum* trunc_spec(MixNum* mn){call {mix_num imn, res; imn.entire = mn->integral; imn.num = mn->num; imn.denom = mn->denom; res = trunc(&imn); mn->integral = imn.entire; mn->num = imn.num; mn->denom = imn.denom;return create(&MixNum_type, res.entire,res.num,res.denom); }}

25. Scenarios Scenario Оракул Оракул Oracle Медиатор Медиатор Mediator Test sequence generator Test engine Steps of test development using the tool: Test actioniterator Testscenario • Specification development • Mediatordevelopment • Test scenario development Specifications Mediators on SeC Target system

26. Scenario definition set_coverage_<specification function name>(<mediator name>); scenario dfsm <scenario name> = {.init = <function pointer of type bool(*Init)(int,char**)>,.actions = { <scenario function pointer> , <scenario function pointer> ... , NULL },.finish = <function pointer of type void(*Finish)()>}

27. Scenario functions scenario bool <function name>(){ ... <specification function name>(<values of args>); ...return <bool result>;}

28. Operatoriterate <type name> <iteration variable name> = <initail value> scenario bool <function name>(){ ...iterate( <iteration initialization> ; <iteration stop condition> ; <next iteration step evaluation> ; <iteration step filtration condition> ) { <build argument values of specification function call> <specification function call> } ...return <bool result>;}

29. Control flow in scenario function • scenariobool <function name>(){ ...iterate( <initialization expression> ; <stop expression> ; <next evaluation expression> ; <filtration expression> ) { ... } ... return <bool result>;} Test engine

30. Scenario of specification function trunc_spec bool trunc_scen_init (int argc, char **argv) { set_mediator_trunc_spec (trunc_media);return true;} scenario bool trunc_sf () {iterate (int i = -10; i < 10; i++;)iterate (int num = -10; num <= 10; num++; i*num >= 0)iterate (int denom = 1; denom <= 10; denom++;) { MixNum mn = create(&MixNum_type, res.entire, res.num, res.denom); trunc_spec (mn); }return true;} scenariodfsm trunc_scen = {.init = trunc_scen_init,.actions = { trunc_sf, NULL }};

31. Functionmain,starting scenario int main(int argc, char** argv){ <scenario name>( argc, argv );}

32. Functionmainof test for function trunc int main(int argc, char **argv){ trunc_scen(argc, argv);}