1 / 41

T.Y. Chen Swinburne University of Technology, Australia

Semi-Proving : an Integrated Method Based on Global Symbolic Evaluation and Metamorphic Testing. T.Y. Chen Swinburne University of Technology, Australia. T.H. Tse and Zhiquan Zhou The University of Hong Kong. ( speaker). Presentation Outline. Conventional Program Testing and Proving

ciaran-fry
Download Presentation

T.Y. Chen Swinburne University of Technology, Australia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Semi-Proving: an Integrated Method Based on Global Symbolic Evaluation and Metamorphic Testing T.Y. Chen Swinburne University of Technology, Australia T.H. Tse and Zhiquan Zhou The University of Hong Kong (speaker)

  2. Presentation Outline • Conventional Program Testing and Proving • Metamorphic Testing • Our method: Semi-Proving • Summary.

  3. Presentation Outline • Conventional Program Testing and Proving • Metamorphic Testing • Our method: Semi-Proving • Summary.

  4. Conventional Program Testing and Proving Given a bijective function f ; A Program:F_Sort (a1, a2, ..., an), n  2 Output: (a1’, a2’, ..., an’), such that 1. (a1’, a2’, ..., an’) is a permutation of (a1, a2, ..., an) 2. f (a1’) f (a2’)  ... f (an’).

  5. Conventional Program Testing and Proving • Testing 1. Design test cases: e.g. (2, 6, 3) for n=3 2. Run: F_Sort (2, 6, 3) = (6, 3, 2) 3. Check: f (6) <f (3) <f (2) ?

  6. Conventional Program Testing and Proving • Proving correctness 1. F_Sort terminates for any valid input; 2. The output is correct.

  7. Conventional Program Testing and Proving • Proving properties F_Sort (a1, a2, ..., an) = (a1’, a2’, ..., an’) Permutation.

  8. Metamorphic Testing • Metamorphic Testing • Employing relationships between different executions Fact: different permutations will produce same output F_Sort (a1, a2, a3) = F_Sort (a3, a1, a2) “ Metamorphic Relation ” ·

  9. Metamorphic Testing Metamorphic Test Cases: {(2, 6, 3), (3, 2, 6)} Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) No matter whether an oracle is available or not; Very useful when the oracle cannot be found. || 2. F_Sort (3, 2, 6) = (6, 3, 2) PASS

  10. || Metamorphic Testing Metamorphic Test Cases: {(2, 6, 3), (3, 2, 6)} Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) 2. F_Sort (3, 2, 6) = (3, 6, 2) Failure.

  11. Presentation Outline • Conventional Program Testing and Proving • Metamorphic Testing • Semi-Proving: Verifying Metamorphic Relations • Summary.

  12. Semi-Proving: Verifying Metamorphic Relations • Objective: • If the program does not satisfy a metamorphic relation onsome inputs, locate these inputs; • Otherwise prove the satisfaction of the metamorphic relation over all inputs.

  13. Semi-Proving: Verifying Metamorphic Relations • Why called “Semi”? • Proving necessary properties, which may not be sufficient for program correctness • Characteristics of Semi-Proving • Multiple symbolic executions • Testing and proving.

  14. Semi-Proving: Verifying Metamorphic Relations double GetMid (double x1, double x2, double x3) { double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; }

  15. Semi-Proving: Verifying Metamorphic Relations • Specification • “GetMid (X, Y, Z)” returns the medianof (X, Y, Z) • E.g. GetMid (3, 4, 1): “3”.

  16. Purpose: to verify Semi-Proving: Verifying Metamorphic Relations • Verifying “GetMid” by Semi-Proving • Identify a Metamorphic Relation GetMid ( X, Y, Z ) =GetMid ( permute(X, Y, Z) ) any numbers any permutation

  17. Semi-Proving: Verifying Metamorphic Relations • Basic concepts • Transposition • simple permutation that exchanges two elements (1, 2, 3)  (2, 1,3) ......... 1 (1, 2, 3)  (1, 3, 2) ......... 2

  18. 1 2   Semi-Proving: Verifying Metamorphic Relations • Basic concepts • Compositionof Transpositions A tuple (1, 2, 3) A permutation(2, 3, 1) (1, 2, 3) (2, 1,3) (2, 3, 1)

  19. Semi-Proving: Verifying Metamorphic Relations • Result from Group Theory • Any permutation of (X, Y, Z) can be achieved by compositions of transpositions (X, Z, Y) and (Y, X, Z).

  20. GetMid (X, Y, Z) = GetMid (X, Z, Y) • GetMid (X, Y, Z) = GetMid (Y, X, Z) Semi-Proving: Verifying Metamorphic Relations • Purpose • GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) • Only need to verify: Any permutation.

  21. GetMid (X, Y, Z) = GetMid (X, Z, Y) • GetMid (X, Y, Z) = GetMid (Y, X, Z) Semi-Proving: Verifying Metamorphic Relations • Purpose • GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) • Only need to verify:

  22. Semi-Proving: Verifying Metamorphic Relations • Global Symbolic Evaluation on GetMid (X, Y, Z) • Execute allthe possible paths.

  23. Semi-Proving: Verifying Metamorphic Relations double GetMid (double x1, double x2, double x3) { double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; }

  24. X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true C1: (Y X < Z) OR (Z < X Y) Path Conditions C2: (X < Y < Z) OR (Z Y < X) C3: (Y < Z X) OR (X Z Y) Semi-Proving: Verifying Metamorphic Relations

  25. ? GetMid (X, Z, Y) Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ?

  26. X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true X when C4 is true =Z when C5 is true Y when C6 is true GetMid (X, Z, Y) Semi-Proving: Verifying Metamorphic Relations ? ? PASS • C4: (Z X < Y) OR (Y < X  Z) • C5: (X < Z < Y) OR (Y  Z < X) • C6: (Z < Y X) OR (X  Y  Z)

  27. X when C4 is true =Z when C5 is true Y when C6 is true GetMid (X, Z, Y) Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ? ? ? PASS • C4: (Z X < Y) OR (Y < X  Z) • C5: (X < Z < Y) OR (Y  Z < X) • C6: (Z < Y X) OR (X  Y  Z) & C1: (Y X < Z) OR (Z < X Y)  Contradiction

  28. X when C4 is true =Z when C5 is true Y when C6 is true GetMid (X, Z, Y) Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ? ? ? • C4: (Z X < Y) OR (Y < X  Z) • C5: (X < Z < Y) OR (Y  Z < X) • C6: (Z < Y X) OR (X  Y  Z) X=Y<Z OR Z<Y=X & C1: (Y <= X < Z) OR (Z < X <= Y)

  29. X when C4 is true =Z when C5 is true Y when C6 is true GetMid (X, Z, Y) Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ? ? ? Yes. X=Y PASS • C4: (Z X < Y) OR (Y < X  Z) • C5: (X < Z < Y) OR (Y  Z < X) • C6: (Z < Y X) OR (X  Y  Z) X=Y<Z OR Z<Y=X & C1: (Y <= X < Z) OR (Z < X <= Y)

  30. verified Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ? GetMid (X, Z, Y)

  31. Semi-Proving: Verifying Metamorphic Relations • Conclusion X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ? GetMid (X, Z, Y)

  32. Semi-Proving: Verifying Metamorphic Relations • Conclusion X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true ? GetMid (X, Z, Y)

  33. Semi-Proving: Verifying Metamorphic Relations • Conclusion X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true GetMid (X, Z, Y)

  34. GetMid (Y, X, Z) Any Any. Semi-Proving: Verifying Metamorphic Relations • Conclusion X when C1 is true GetMid (X, Y, Z) = Y when C2is true Z when C3is true GetMid (X, Z, Y) Composition of transpositions GetMid (X, Y, Z) = GetMid ( Permute(X, Y, Z))

  35. Semi-Proving: Detecting Program Faults • Detecting Program Faults ·

  36. double GetMid (double x1, double x2, double x3) { double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; }

  37. ? AND Semi-Proving: Detecting Program Faults Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) || X when Y X < Z || Y when (Z < Y X ) OR (Y Z AND X Z)

  38. ? AND Semi-Proving: Detecting Program Faults Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) || X when Y X < Z || Y when (Z < Y X ) OR (Y Z AND X Z) (Y=X<Z) OR (Y<X<Z)

  39. AND Semi-Proving: Detecting Program Faults Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) || X when Y X < Z || Y when (Z < Y X ) OR (Y Z AND X Z) failure ? ? (Y=X<Z) OR (Y<X<Z) Can identify all the failure-causing inputs. Failure-causing input

  40. Summary • A proving technique: all the paths • A testing technique: • failure-causing inputs • selected path(s) • Characteristics • Metamorphic relations • Multiple symbolic executions • Employing global symbolic evaluation and constraint solving.

  41. Questions are welcome

More Related