CSE 497A Spring 2002 Functional Verification Lecture 2/3 Vijaykrishnan Narayanan

1. CSE 497ASpring 2002Functional VerificationLecture 2/3Vijaykrishnan Narayanan

2. Course Administration • Instructor Vijay Narayanan (vijay@cse.psu.edu) 229 Pond Lab Office Hours: T 10:00-11:00; W 1:00-2:15 • Tool Support Jooheung Lee (joolee@cse.psu.edu) • TA TBA • Laboratory 101 Pond Lab • Materials www.cse.psu.edu/~vijay/verify/ • Texts • J. Bergeron. Writing Testbenches: Functional Verification of HDL Models. Kluwer Academic Publishers. • Class notes - on the web

3. Grading • Grade breakdown • Midterm Exam: 20% • Final Exam: 25% • Verification Projects (~4): 40% • Homework (~3) 15% • No late homeworks/project reports will be accepted • Grades will be posted on course home page • Written/email request for changes to grades • April 25 deadline to correct scores

4. Secret of Verification (Verification Mindset)

5. The Art of Verification • Two simple questions • Am I driving all possible input scenarios? • How will I know when it fails?

6. Three Simulation Commandments Thou shalt stress thine logic harder than it will ever be stressed again Thou shalt not move onto a higher platform until the bug rate has dropped off Thou shalt place checking upon all things

7. Testcase Driver Testcase Simulator Environment Data Output Model Design Source General Simulation Environment Compiler (not always required) C/C++ HDL Testbenches Specman e Synopsis' VERA Event simulator Cycle simulator Emulator Initialization Run-time requirements Testcase results Event Simulation compiler Cycle simulation compiler .... Emulator Compiler VHDL Verilog

8. Run Foreground Simulation Run Background Simulation Configure Environment Release Environment Debug Fail Debug Environment View Trace Monitor Batch Simulation Specify Batch Simulation Answer Defect Redirect Defect Verify Defect Fix Regress Fails Create Defect Define Project Goals Release Model Project Status Report Transfer Testcase Logic Designer Environment Developer Verification Engineer Model Builder Project Manager

9. Facilities: a general term for named wires (or signals) and latches. Facilities feed gates (and/or/nand/nor/invert, etc) which feed other facilities. • EDA: Engineering Design Automation--Tool vendors. Some lingo

10. Behavioral: Code written to perform the function of logic on the interface of the design-under-test • Macro: 1. A behavioral 2. A piece of logic • Driver: Code written to manipulate the inputs of the design-under-test. The driver understands the interface protocols. • Checker: Code written to verify the outputs of the design-under-test. A checker may have some knowledge of what the driver has done. A check must also verify interface protocol compliance. More lingo

11. Snoop/Monitor: Code that watches interfaces or internal signals to help the checkers perform correctly. Also used to help drivers be more devious. • Architecture: Design criteria as seen by the customer. The design's architecture is specified in documents (e.g. POPS, Book 4, Infiniband, etc), and the design must be compliant with this specification. • Microarchitecture: The design's implementation. Microarchitecture refers to the constructs that are used in the design, such as pipelines, caches, etc. • Escape: An error that appears in test floor escaping verification Still more lingo

12. Checking framework Scoreboard Struct: Header Payload checking xlate predict DUT (bridge chip) Bus gen packet drive packet post packet Conv- ersation Errors Sequence Packet Protocol Typical Verification diagram Coverage Data Stimulus Device types FSMs latency conditions address transactions sequences transitions

13. Verification Cycle Develop environment Create Testplan Debug hardware Escape Analysis Regression Hardware debug Fabrication

14. Team leaders work with design leaders to create a verification testplan. The testplan includes: • Schedule • Specific tests and methods by simulation level • Required tools • Input criteria • Completion criteria • What is expected to be found with each test/level • What's not covered by each test/level Verification Testplan

15. Verification is a process used to demonstrate the functional correctness of a design. Also called logic verification or simulation.

16. Reconvergence Model • Conceptual representation of the verification process • Most important question • What are you verifying? Transformation Verification

17. What is a testbench? • A “testbench” usually refers to the code used to create a pre-determined input sequence to a design, then optionally observe the response. • Generic term used differently across industry • Always refers to a testcase • Most commonly (and appropriately), a testbench refers to code written (VHDL, Verilog, etc) at the top level of the hierarchy. The testbench is often simple, but may have some elements of randomness • Completely closed system • No inputs or outputs • effectively a model of the universe as far as the design is concerned. • Verification challenge: • What input patterns to supply to the Design Under Verification and what is expected for the output for a properly working design

18. Show Multiplexer Testbench

19. Importance of Verification • Most books focus on syntax, semantics and RTL subset • Given the amount of literature on writing synthesizeable code vs.. writing verification testbenches, one would think that the former is a more daunting task. Experience proves otherwise. • 70% of design effort goes to verification • Properly staffed design teams have dedicatedverification engineers. • Verification Engineers usually outweigh designers 2-1 • 80% of all written code is in the verification environment

20. Escape: A problem that is found on the test floor and therefore has escaped the verification process • The Line Delete escape was a problem on the H2 machine • S/390 Bipolar, 1991 • Escape shows example of how a verification engineer needs to think The Line Delete Escape

21. Line Delete is a method of circumventing bad cells of a large memory array or cache array • An array mapping allows for removal of defective cells for usable space The Line Delete Escape (pg 2)

22. 05 . . . The Line Delete Escape (pg 3) If a line in an array has multiple bad bits (a single bit usually goes unnoticed due to ECC-error correction codes), the line can be taken "out of service". In the array pictured, row 05 has a bad congruence class entry.

23. Data in ECC Logic 05 . . . ECC Logic Counters Data out The Line Delete Escape (pg 4) Data enters ECC creation logic prior to storage into the array. When read out, the ECC logic corrects single bit errors and tags Uncorrectable Errors (UEs), and increments a counter corresponding to the row and congruence class.

24. ECC Logic 05 . . . ECC Logic Counters Threshhold Service Controller The Line Delete Escape (pg 5) Data in When a preset threshhold of UEs are detected from a array cell, the service controller is informed that a line delete operation is needed. Data out

25. Data in ECC Logic Line delete control Storage Controller configuration registers 05 . . . ECC Logic Counters Data out Threshhold Service Controller The Line Delete Escape (pg 6) The Service controller can update the configuration registers, ordering a line delete to occur. When the configuration registers are written, the line delete controls are engaged and writes to row 5, congruence class 'C' cease. However, because three other cells remain good in this congruence class, the sole repercussion of the line delete is a slight decline in performance.

26. Data in ECC Logic Line delete control Storage Controller configuration registers 05 . . . ECC Logic Counters Threshhold Service Controller The Line Delete Escape (pg 7) How would we test this logic? What must occur in the testcase? What checking must we implement? Data out

27. Verification is on critical path

28. Want to minimize Verification Time!

29. Ways to reduce verification time • Verification can be reduced through: • Parallelism: Add more resources • Abstraction: Higher level of abstraction (i.e. C vs.. Assembly) • Beware though – this means a reduction in control • Automation: Tools to automate standard processes • Requires standard processes • Not all processes can be automated

30. System ... Chip Unit Macro Hierarchical Design Allows design team to break system down into logical and comprehendable components. Also allows for repeatable components.

31. Ways to reduce verification time • Verification can be reduced through: • Parallelism: Add more resources • Abstraction: Higher level of abstraction (i.e. C vs.. Assembly) • Beware though – this means a reduction in control/additional training • Vera, e are examples of verification languages • Automation: Tools to automate standard processes • Requires standard processes • Not all processes can be automated

32. Human Factor in Verification Process • An individual (or group of individuals) must interpret specification and transform into correct function. RTL Coding Specification Interpre- tation Verification

33. The verification engineer should not be an individual who participated in logic design of the DUT • Blinders: If a designer didn't think of a failing scenario when creating the logic, how will he/she create a test for that case? • However, a designer should do some verification on his/her design before exposing it to the verification team • Independent Verification Engineer needs to understand the intended function and the interface protocols, but not necessarily the implementation Need for Independent Verification

34. DO: • Talk to designers about the function and understand the design first, but then • Try to think of situations the designer might have missed • Focus on exotic scenarios and situations • e.g try to fill all queues while the design is done in a way to avoid any buffer full conditions • Focus on multiple events at the same time Verification Do's and Don'ts

35. Try everything that is not explicitly forbidden • Spend time thinking about all the pieces that you need to verify • Talk to "other" designers about the signals that interface to your design-under-test • Don't: • Rely on the designer's word for input/output specification • Allow RIT Criteria to bend for sake of schedule Verification Do's and Don'ts (continued)

36. Ways to reduce human-introduced errors • Automation • Take human intervention out of the process • Poka-Yoka • Make human intervention fool-proof • Redundancy • Have two individuals (or groups) check each others work

37. Automation • Obvious way to eliminate human-introduced errors – take the human out. • Good in concept • Reality dictates that this is not feasible • Processes are not defined well enough • Processes require human ingenuity and creativity

38. Poka-Yoka • Term coined in Total Quality Management circles • Means to “mistake-proof” the human intervention • Typically the last step in complete automation • Same pitfalls as automation – verification remains an art, it does not yield itself to well-defined steps.

39. Redundancy • Duplicate every transformation • Every transformation made by a human is either: • Verified by another individual • Two complete and separate transformations are performed with each outcome compared to verify that both produced the same or equivalent result • Simplest • Most costly, but still cheaper than redesign and replacement of a defective product • Designer should NOT be in charge of verification!

40. What is being verified? • Choosing a common origin and reconvergence points determines what is being verified and what type of method to use. • Following types of verification all have different origin and reconvergence points: • Formal Verification • Model Checking • Functional Verification • Testbench Generators

41. Formal Verification • Once the end points of formal verification reconvergence paths are understood, then you know exactly what is being verified. • 2 Types of Formal: • Equivalence • Model Checking

42. Equivalence Checking • Compares two models to see if equivalence • Netlists before and after modifications • Netlist and RTL code (verify synthesis) • RTL and RTL (HDL modificiations) • Post Synthesis Gates to Post PD Gates • Adding of scan latches, clock tree buffers • Proves mathematically that the origin and output are logically equivalent • Compares boolean and sequential logic functions, not mapping of the functions to a specific technology • Why do verification of an automated synthesis tool?

43. Equivalence Reconvergence Model Synthesis RTL Gates Check

44. Model Checking • Form of formal verification • Characteristics of a design are formally proven or disproved • Find unreachable states of a state machine • If deadlock conditions will occur • Example: If ALE will be asserted, either DTACK or ABORT signal will be asserted • Looks for generic problems or violations of user defined rules about the behavior of the design • Knowing which assertions to prove is the major difficulty

45. Steps in Model Checking • Model the system implementation using a finite state machine • The desired behavior as a set of temporal-logic formulas • Model checking algorithm scans all possible states and execution paths in an attempt to find a counter-example to the formulas • Check these rules • Prove that all states are reachable • Prove the absence of deadlocks • Unlike simulation-based verification, no test cases are required

46. Problems with Model Checking • Automatic verification becomes hard with increasing number of states • 10^100 states (larger than number of protons in the universe) but still does not go far beyond 300 bits of state variables. • Absurdly small for millions of transistors in current microprocessors • Symbolic model checking explores a larger set of states concurrently. • IBM Rulebase (Feb 7) is a symbolic Model Checking tool

47. Model Checking Reconvergence Model RTL Specification RTL Interpretation Model Checking Assertions

48. Functional Verification • Verifies design intent • Without, one must trust that the transformation of a specification to RTL was performed correctly • Prove presence of bugs, but cannot prove their absence

49. Functional Reconvergence Model Specification RTL Functional Verification

50. Testbench Generators • Tool to generate stimulus to exercise code or expose bugs • Designer input is still required • RTL code is the origin and there is no reconvergence point • Verification engineer is left to determine if the testbench applies valid stimulus • If used with parameters, can control the generator in order to focus the testbenches on more specific scenarios