Efficient Functional Test Generation Techniques for Digital Systems Verification

1. Verification Introduction ECE 667Synthesis and Verificationof Digital Systems ECE 667 - Synthesis & Verification

2. ECE 667 - Synthesis & Verification Outline • Motivation: whatis verification, why we need it • Verification methods • Formal methods • Simulation-based functional verification • Deterministic test generation

3. ECE 667 - Synthesis & Verification  ? model Design 1 Design 2  ? RTL HDL / RTL behavior  ? Logic level Logic level function  ?  ?  ? structure Gate level Gate level  ? layout Mask level Mask level Verification • Design verification = ensuring correctness of the design • against its implementation (at different levels) • against alternative design (at the same level)

4. ECE 667 - Synthesis & Verification Why Verification • Verification crisis • System complexity, difficult to manage • More time, effort devoted to verification (70%) than to design • Need automated verification methods, integration • Consequences • Disasters, life threatening situations • Inconvenience (Pentium bug … ?) • Many more …

5. ECE 667 - Synthesis & Verification Formal Verification Verification Methods • Deductive verification • Model checking • Equivalence checking • Simulation - performed on the model • Emulation, prototyping – product + environment • Testing - performed on the actual product (manufacturing test)

6. ECE 667 - Synthesis & Verification Functional Verification Simulation-based: Validation • Goal: verify the design in the full operational context • RTL functional verification • Verify specification (HDL) of RTL model • No model to check against: must simulate • Functional simulation • Functional test generation • Automatically generate tests: high-level transactions on data, clocking, control • SAT based methods

7. ECE 667 - Synthesis & Verification Evaluating Test Coverage • Coverage metrics - facilities to measure the effectiveness of functional verification • Monitors: collect data about testing (coverage, profile) • Code coverage • low-level coverage statistics for states, transitions, HDL model line coverage • Functional verification coverage • statistics, monitors for events, state transition sequences (transactions), data sets • Self-checking tests

8. ECE 667 - Synthesis & Verification Functional Test Generation • Given an RTL design and a coverage metric, must reach the predefined coverage goal • Solution: run functional simulation • Directed tests • manual, often easy to generate (e.g. instruction set) • reliable (predictable coverage), but • not efficient (cover small portion of design) • Random tests • efficient (fast), but not reliable (unpredictable coverage) • Deterministic tests • Automatically generated • Constraints (user-defined, environment, coverage metrics) • Challenging to compute

9. ECE 667 - Synthesis & Verification Functional Verification - typical scenario 100 % Deterministic tests ? 100.0 95 % Normalized verification test cycles Pseudo-random directed tests (reliable and efficient) Coverage 1.0 50 % Manual directed tests (reliable, not efficient) Test development time

10. ECE 667 - Synthesis & Verification Functional Test Generation • Random and pseudo-random methods • Directed pseudo-random simulation • Deterministic Methods • SAT-based methods • Boolean satisfiability • Mixed, arithmetic/Boolean satisfiability • Symbolic simulation • ATPG-based methods

11. ECE 667 - Synthesis & Verification Test Generation using Boolean SAT • Given an RTL specification of a combinational circuit • Simulate the design (pseudo-random, targeted vectors) • Code coverage OK ? module input ports, output ports, internal signals begin . . . . . If (condition) then assign signal = function( ); end if; . . . . . end module If not - expand the underlying logic: Boolean function

12. ECE 667 - Synthesis & Verification Boolean logic + constraints B-SAT -Solving SAT using BDDs • Add constraints (modify the logic) • Build BDDs for each output, s.to constraints • Build the product BDD (AND of all BDDs) • If the set is empty, infeasible SAT instance • Otherwise: set of all satisfying assignments, test.

13. ECE 667 - Synthesis & Verification a u b c v w d A simple B-SAT example • Given: output value requirements for a circuit • Compute: satisfying assignments at the inputs • Output requirements: u=1, v=1, w=1 • SAT assignments: a,b,c,d = ?

14. ECE 667 - Synthesis & Verification a b ab c ab’c 1 0 How does B-SAT work ? • Boolean satisfiability analysis • H = product BDD • set of all satisfying solutions • to test for H = 1 (0), find a path in the BDD to terminal 1 (0) • the path, expressed infunction variables, gives a satisfying solution (test vector) H {1,1,-}, {1,0,1}

15. ECE 667 - Synthesis & Verification A=? + B=? < extract out * x=? c=? 0 1 a=? b=? Functional test generation using Symbolic Simulation • Deterministic test pattern generation • Formulate a SAT problem for a complex combinational design • Solve SAT: find a set of satisfying assignment Module DUT … always @ (clk) begin if (A+B < B*C) out = x; else out = a & b end

16. ECE 667 - Synthesis & Verification Formal Verification • Deductive reasoning (theorem proving) • Uses axioms, rules to prove system correctness • No guarantee that it will terminate • Difficult, time consuming: for critical applications only • Model checking • Automatic technique to prove correctness of concurrent systems: digital circuits, communication protocols, etc. • Practical tools become available, popular in industry • Equivalence checking • Check if two designs are equivalent • OK for combinational circuits, unsolved for sequential systems

17. ECE 667 - Synthesis & Verification Equivalence checking 0 D1 D2 Property checking p 0 D1 Formal Verification • Equivalence checking • Takes two designs and makes a miter (AIG) • Property checking • Takes design and property and makes a miter (AIG) • Prove that result is 0 (unSAT) The goals are the same: to transform AIG until the output is proved constant 0 ABC is very effective at doing it