Paper Report

1. Paper Report Design Fault Directed Test Generation for Microprocessor Validation Deepak A. Mathaikutty, Sandeep K. Shukla FERMAT Lab, Virginia Tech, Blacksburg Sreekumar V. Kodakara, David Lilja The University of Minnesota,Minneapolis AjitDingankar Validation Tools, Intel Corporation, Folsom Design, Automation & Test in Europe Conference & Exhibition, 2007. DATE '07 Presenter: Jyun-Yan Li

2. Abstract • Functional validation of modern microprocessors is an important and complex problem. One of the problems in functional validation is the generation of test cases that has higher potential to find faults in the design. • We propose a model based test generation framework that generates tests for design fault classes inspired from software validation. There are two main contributions in this paper. Firstly, we propose a microprocessor modeling and test generation framework that generates test suites to satisfy modified condition decision coverage (MCDC), a structural coverage metric that detects most of the classified design faults as well as the remaining faults not covered by MCDC.

3. Abstract (cont.) • Secondly, we show that there exists good correlationbetween types of design faults proposed by software validation and the errors/bugs reported in case studies on microprocessor validation. We demonstrate the framework by modeling and generating tests for the microarchitecture of VESPA, a 32-bit microprocessor. In the results section, we show that the tests generated using our framework's coverage directed approach detects the fault classes with 100% coverage, when compared to model-random test generation

4. What is the Problem • Simulation is widely used to validate large system • Depend on the quality of tests • Coverage metric measures the quality • Software validation • Statement, branch and path coverage • State machine representation • State, transition and path coverage • Functionality • Effectiveness of coverage directed test generation • Type of faults • Can occur • Strength of the coverage metric • Detect faults

5. Related work coverage random Extended fault class [7] Functional coverage 9 fault classes related to boolean expressions Graph-based [9] coverage of formal models [5] Micro architecture coverage [12] MCDC [6] Generate assembler test program directly for PowerPC Build graph model of processor and generate test program to detect fault For superscaler processor Strategy of coverage metric Design Fault Directed Test Generation for Microprocessor Validation This paper:

6. Fault classes [7] Correct

7. Coverage Metrics • Statement coverage • Invoke executable statement • Branch coverage • Execute both the true and false • Modified condition Decision Coverage (MCDC) • Test each of condition within a decision • Affect outcome of the decision independently • m conditions, m+1 test cases The same with T2 The same with T1

8. Motivation • Correlate the bug categories in [2, 13] and map into the fault class • Detect all the design fault except LIF • None of the errors observed was classified as a LIF

9. Proposal method • Modeling language as a metamodel • Describe architectural and microarchitectural Statement, branch, MCDC Generate golden result Validate the Microcode Simulation result Validate the RTL implementation

10. Architecture modeling • Processor is specified • Register/memory-level description • Registers, their relationship and memory schematic • Instruction set capture • Instruction behavior • Construct • Function blocks, if-else block, statement sequence and loop, register map and memory layout • Ex: jump-if-not-zero (JNZ) instruction Register map Register/memory reference or immediate value

11. Microarchitecture modeling • Construct • Configure pipe stage, provide ports with memory and registers, insert instruction registers and describe the control logic • Combined to complex stages by basic components as MUX, ALU, INC, etc • Ex: Z Instruction fetch stage 1 2 3 0 PC Instruction memory Increment by 4 1 0 PC2

12. Test generation Framework • Constraint Satisfaction Problem (CPS) • Build Program Flow Graph (PFG) • Static Single-Assignment(SSA) analysis for multiple incoming path • Coverage Constraint Generator (CCG) generates constrains for decisions • Test Case Generator (TCG) • ILOG solver generates test case • Binary generator initializes registers, memory locations and variables

13. Constraint Satisfaction Problem(CSP) • Convert the architectural and micro-architectural model into Program Flow Graph (PFG) • Architectural model • Function block -> hierarchical states • Statement -> state • If-else -> decision • Identifier, registers, memory location -> variables • Micro-architectural model • Basic components -> bunch of if-else sequences • If a state with multiple incoming transition • Resolve in the Static Single-Assignment (SSA) analysis

14. PFG example • JNZ instruction • IF stage variable decision state

15. Static Single-Assignment (SSA) analysis • Help CSP to generate constrains for complete run • Focus on states with multiple incoming transitions • If no change the value • the state will be ignored • Insert additional decisions and variables • Ex: SSA result Ignore S3 SSA analysis Rename variables

16. ILOG solver • ILOG is a commercial solver designed for performance and scalability • Generate test case that satisfying every constraint variable • Ex:

17. Coverage constraint generator (CCG) • Two set of inputs • Coverage annotation • expedite the reachability of the coverage goal • Setting the attribute MARK associated with a state and decision • Coverage type • Statement coverage • Branch coverage • MCDC • Design Fault coverage

18. MCDC Problematic decision point D Graph G Generate a case that the constraints for path p and decision is false (d=0) Generate a case that the constraints for path p and decision is true (d=1) Total test case =combine and Arrange D by graph depth Node d Generate a case that the constraints for path p and conditions are false Generate a case that the constraints for path p and conditions are true Total test case =combine and Find a path p form root to d Boolean operator? N Y Remove the duplicate test cases

19. Design Fault coverage • The fault detection capability of MCDC is 7/9 classes coverage • Literal omission fault (LOF) can be satisfied by MCDC • Every literal will affect the output of decision • Ex: • Literal Reference Fault (LRF) coverage by LRF_list • A list of variables to tell CCG which variable can be replaced for resulting in a fault • If LRF_list is not provided, the CCG replaces every variabled • Lead to large set of test case • Recommend LRF_list with a variable and mark it

20. Before Experiment • How many • test cases are generated • Not describes in the paper • Conditions in a decision • 330 unique decisions, 400 unique conditions • 1~10 conditions in a decision • Fault in each fault class

21. Experimental result • Environment • VESPA 32-bit processor, 5-stage pipeline • 50 decisions are chosen and replaced randomly for LRF • The number of test casesare the same for each coverage metrics

22. Conclusion • A metamodel-based modeling framework to capture structure and behavior • Architecture – registers, ISA • Micro-architecture – pipeline structure, control logic • TGF results in test case for covering design faults • CSP formulation • TCG • My comment • Overview the test case generating flow for code coverage • Some algorithm is not specification