1 / 16

Generating High-Quality Tests for Boolean Circuits by Treating Tests as Proof Encoding

Generating High-Quality Tests for Boolean Circuits by Treating Tests as Proof Encoding. Eugene Goldberg, Pete Manolios Northeastern University, USA. TAP-2010, July 1-2, Malaga, Spain. Summary. Introduction Test generation algorithm based on TTPE Experimental results and conclusion.

brooke
Download Presentation

Generating High-Quality Tests for Boolean Circuits by Treating Tests as Proof Encoding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Generating High-Quality Tests for Boolean Circuits by Treating Tests as Proof Encoding Eugene Goldberg, Pete Manolios Northeastern University, USA TAP-2010, July 1-2, Malaga, Spain

  2. Summary • Introduction • Test generation algorithm based on TTPE • Experimental results and conclusion

  3. Motivation • Testing is easy (scalable) but incomplete • Formal verification is complete but hard • How do we get the best of both worlds (e.g. by generating a small test set that is complete or close to such)? • We study testing by the example of combinational circuits • We describe circuits by propositional logic

  4. Main Idea TTPE: Treating Tests As Proof Encoding Q: Why does testing work at all? A: Short proof  small encoding test set Q: What is the best coverage metric? A: A formal proof is an ideal coverage metric (contains a complete set of corner cases) Q: When does one stop generating tests? A: When the test set encodes a proof

  5. Description of the Setup N is a combinational circuit, Itis correct if N  0, It has a bug if N(x) = 1. z N Y Circuit N  CNF formula FN(X,Y,z), N  0  (FNz)  0 …. X Point p is (x,y,z) i.e. a complete assignment to X Y  {z} Test t extracted from p is the assignment to X, i.e. t = x

  6. A Naive Testing Procedure Si = {p1,…,pi} z p1, t1, N(t1)=0, Enc(S1) = no N Y p2, t2, N(t2)=0, Enc(S2) = no …….. pi, ti, N(ti)=0, Enc(Si) = no …. X pi+1, ti+1, N(ti+1)=1, Enc(Si+1) = no pi+1, ti+1, N(ti+1)=0, Enc(Si+1) = yes

  7. Summary • Introduction • Test generation algorithm based on TTPE • Experimental results and conclusion

  8. High-Level Description • We use the resolution proof system • Checking if a set of points encodes a resolution proof is hard • Instead, we generate points (called boundary) that encode mandatory fragments of a proof • So, instead of trying to encode an entire proof we target essential parts of it • We stop when a counterexample is found or a resource is exceeded

  9. Encoding a Resolution Proof C = x1 x2  ~x5, C =~x1x7  C = x2  ~x5 x7 p, p legalize the resolution of C and C on v iff a) p and p are different only in v b) C (p ) = 0, C (p ) = 0 c) C(p ) = 0, C(p ) = 0, where C is the resolvent S={p1,..,pk} encodes a proof if the resolutions legalized by points of S are sufficient to derive an empty clause. Tests T encode a proof if they are extracted from S encoding a proof

  10. Small Complete Test Sets A proof of (FNz)  0 of k resolutions is encoded by 2k points (at most) A complete set of  2k tests for checking N  0. N with 1000inputs: 106 tests instead of 21000

  11. Boundary Points • Given a CNF formula F, an unsatisfying assignment to Vars(F) is a v-boundary point p iff • If C  F and C(p)=0, then v  Vars(C) • If p is a v-boundary point of F and F is unsatisfiable, any proof has to have a resolution such that • it is a resolution on v, • the resolvent is falsified by p Adding the resolvent of this resolution to F eliminates p as a v-boundary point. If F is satisfiable, there is a boundary point of F that cannot be eliminated.

  12. Extracting Tests from Boundary Points Circuit N , CNF formula F=FN  z, Is N(t)=1? Generate a v-boundary point pi of F (finding pirequires a SAT-check) Extract test ti from pi N(ti)=1? If so, stop. Add to F a resolvent on v (mandated by pi) Go to step 1 (to generatepi+1)

  13. Summary • Introduction • Test generation algorithm based on TTPE • Experimental results and conclusion

  14. Faults in Arithmetic Components G is a 24-input AND gate

  15. Comparing SAT, Random Tests and Tests from Boundary Points • 17 faults. Random tests failed (106 per fault) • Reused tests generated for previous faults • Precosat is a winner of SAT-2009 competition • Used Precosat for finding boundary points too

  16. Some Concluding Remarks • TTPE is not a trick. There is a deep relation between proofs and tests • Many ways to use TTPE (e.g. encoding mandatory parts of a proof) • TTPE for more expressive logics (describing sequential circuits and software)?

More Related