SoC Verification Strategies for Embedded Systems Design

1. SOC Design Conference SoC Verification Strategies for Embedded Systems Design November 5-6, 2003/ Seoul Chong-Min Kyung, KAIST

2. Various Embedded Mobile Systems Data Processing Consumer Media Center Audio Desktop PC DVC DTV NotebookPC DSC DVD Game Console MP3Player SmartPhone PDA CreditCard CarNavigation CellularPhone Internet Telematics Automotive & etc. Communication Bluetooth

3. Difficulties in Embedded Systems Design(What’s special in ES design?) • Real operating environment of ES is difficult to reproduce. -> In-system verification is necessary. • Total design flow of ES until the implementation is long and complex. -> Verification must be started early. • Design turn-around time must be short as the life-time of ES itself is quite short. -> Short verification cycle

4. Three Most Important Issues in Verification • 1. Verify Right : • Always make sure you have correct specifications to start with. (Frequent interaction with SPECIFIER, customer, marketing, etc.) • In-System Verification • Check Properties in Formal Techniques. • 2. Verify Early • System-level, Heterogeneous Models, SW-HW • 3. Verify Appropriately • HW-SW Co-simulation • 4. Verify Fast • Hardware Acceleration, Emulation

5. Strongly Required Features of Verification • Accommodate Multiple Levels of Design Representation • Exploit Hardware, Software and Interfacing mechanisms as Verification Tools

6. Target board Chip Model;ISS Host computer as Virtual Chip cable Pin Signal Generator with Buffers Virtual Chip : “Verify Early, In-System” • Virtual Chip: Making Functional Models Work on Real Target System [DAC98] • Simulating ISS of a Processor Chip along with real target environment Chip Socket

7. Without CPU PC running ISS Picture taken in 1993 or 1994

8. Conventional design flow H/W prototype (H/W emulation) Architectural model RTL model Gate-level model H/W Emulation H/W System Board design idle Verification w/ H/W Application S/W design idle Virtual Chip design flow; EARLY ,IN-SYSTEM Design time is drastically reduced Architectural model RTL model Gate-level model H/W Emulation H/W System Board design Verification w/ H/W design Application S/W H/W prototype (Virtual Chip) Reducing TTM using Virtual Chip

9. more refined model CPU Model C Language HDL RTL Verilog Gate-Level Verilog Instruction Behavior In C (Polaris) Micro- architecture in C Virtual Chip FlexPC MCV PLI Real Mother-board H/W Virtual PC in C (VPC) Target Environment • MCV : Microcode Verifier • PLI : Programming Language Interface Microprocessor Design Verification Methodology

10. MS Office MaxPlus II MS Win. 3.1 HK386 (1995) • Design Specification • Instruction level, Pin-to-Pin compatible with i386 • Operation speed : 40 MHz • 0.8 m DLM CMOS ASIC • Test Programs • MS DOS 6.0, Windows 3.1, Office 4.0 • CAD tools, games, etc..

11. What about Emulation? • Swallowed a lot of Money • Required a separate, air-conditioned room • Needed a lot of time for Compile • Needed an expensive slowed-down PC ($20,000 for 500kHz clocking) • However…worked at the last minute • Finding bugs is not easy

12. Probe Module 500kHz Slowed-Down PC Target Interface Board Hardware Emulator x86 Emulation Configuration

13. Windows HDL Simulation Hardware Emulation HDL saver Attached DOS version update 1 version update 2 version update 3 setup Booting Windows 20M instructions on Marcia Simulation debugs, Emulation approves.

14. Agenda • Verify Early and Altogether (In-System) • Why Verification ? • Verification Alternatives • Verification with Progressive Refinement • SoC Verification • Concluding Remarks

15. Trend of Verification Effort in the Design • Verification portion of design increases to anywhere from 50 to 80% of total development effort for the design. 1996 300K gates Code Verify (30 ~ 40%) Synthesis P&R Code Verify (50 ~ 80%) Synthesis P&R 2000 1M SoC Verification methodology manual, 2000- TransEDA

16. Percentage of Total Flaws • About 50% of flaws are functional flaws. • Need verification method to fix logical & functional flaws From Mentor presentation material, 2003

17. Another recent independent study showed that more than half of all chips require one or more re-spins, and that functional errors were found in 74% of these re-spins. • With increasing chip complexity, this situation could worsen. • Who can afford that with >= 1M Dollar NRE cost?

18. Bug Fixing Cost in Time • Cost of fixing a bug/problem increases as design progresses. • Need verification method at early design stage Cost of Fixing a Problem Behavioral Design RTL Design Gate Level Design Device Production Verification methodology manual, 2000 – TransEDA

19. Verification Performance Gap; more serious than the design productivity gap • Growing gap between the demand for verification and the simulation technology offered by the various options. Verification Performance Gap SOC HW simulation acceleration (<10KCPS) Cycle-base simulation (<1KCPS) Verification complexity Simulation performance Event-base simulation (<10CPS) Complex ASIC Medium ASIC Small ASIC Design complexity System-on-a-chip verification, 2001 – P.Rashinkar

20. Completion Metrics; How do we know when the verification is done? • Emotionally, or Intuitively; • Out of money? Exhausted? • Competition’s product is there. • Software people are happy with your hardware. • There have been no bugs reported for two weeks. • More rigorous criteria; • All tests passed • Test Plan Coverage • Functional Coverage • Code Coverage • Bug Rates have flattened toward bottom.

21. Verification Challenges • Specification and/or Operating Environment is Incomplete/Undetermined. (Just like last-minute ECO!) • Cannot avoid using Yesterday’s tool for Today’s Design. • Design productivity grows faster than Verification productivity. (Verification productivity Gap) • Besides Functional Verification, Estimation on Performance, Power Consumption becomes crucial issues in SoC design.

22. Agenda • Verify early and in-system • Why Verification ? • Verification Alternatives • Simulation • Hardware-accelerated simulation • Emulation • Prototyping • Formal verification • Semi-Formal (Dynamic Formal) verification • Verification with Progressive Refinement • SoC Verification • Concluding Remarks

23. Overview of Verification Methodologies Prototyping Faster speed, closer to final product Emulation Hardware Accelerated Simulation Simulation Basicverificationtool Semi-formal Verification Formal Verification Bigger coverage

24. Software Simulation • Pros • The design size is limited only by the computing resource. • Simulation can be started as soon as the RTL description is finished. • Set-up cost is minimal. • Cons • Slow (~100 cycles/sec) ; Speed gap between the speed of software simulation and real silicon widens. (Simulation speed = size of the circuit simulated / speed of the simulation engine) • The designer does not exactly know how much percentage of the design have been tested.

25. Hardware-Accelerated Simulation • Simulation performance is improved by moving the time-consuming part of the design to hardware. • Usually, the software simulation communicates with FPGA-based hardware accelerator. Hardware Accelerator Simulation environment Testbench Module 2 is synthesized & compiled into FPGAs Module 0 Module 1 Module 2

26. Hardware-Accelerated Simulation • Pros • Fast (100K cycles/sec) • Cheaper than hardware emulation • Debugging is easier as the circuit structure is unchanged. • Not an Overhead : Deployed as a step stone in the gradual refinement • Cons (Obstacles to overcome) • Set-up time overhead to map RTL design into the hardware can be substantial. • SW-HW communication speed can degrade the performance. • Debugging of signals within the hardware can be difficult.

27. Emulation • Imitating the function of another system to achieve the same results as the imitated system. • Usually, the emulation hardware comprises an array of FPGA’s (or special-type processors) and interconnection scheme among them. • About 1000 times faster than simulation. Prototyping Emulation Hardware Accelerated Simulation Simulation

28. Emulation • Pros • Fast (500K cycles/sec) • Verification on real target system. • Cons • Setup time overhead to map RTL design into hardware is very high. • Many FPGA’s + resources for debugging  high cost • Circuit partitioning algorithm and interconnection architecture limit the usable gate count.

29. Emulation • Challenges • Efficient interconnection architecture and Hardware Mapping efficiency for Speed and Cost • RTL debugging facility with reasonable amount of resource • Efficient partitioning algorithm for any given interconnection architecture • Reducing development time (to take advantage of more recent FPGA’s)

30. Prototyping • Special (more dedicated and customized) hardware architecture made to fit a specific application. Prototyping Emulation Hardware Accelerated Simulation Simulation

31. Prototyping • Pros • 10X higher clocking than emulation • Components as well as the wiring can be customized. • Can be carried and dispatched for demo or customer evaluation • Cons • Not flexible for design change (Even a small change requires a new PCB.)

32. Overview of Verification Methodologies • Formal verification • Application of logical reasoning to the development of digital system • Both design and its specification are described by a language in which semantics are based on mathematical rigor. • Semi-formal verification • Combination of simulation and formal verification. • Formal verification cannot fully cover large designs, and simulation can come to aid in verifying the large design. Simulation Semi-formal Verification Formal Verification More complete verification

33. Formal Verification • Classes • Model Checking : verifies that the design satisfies properties specified using temporal logic • Equivalence Checking • Symbolic Simulation • Theorem Proving • Pros • Assures 100% coverage. • Run fast. • Cons • Works only for small-size finite state systems due to memory explosion. (so far, <500 flip-flops) • Uncomfortable, i.e., need to learn temporal logic for “property” description in Model Checking

34. Efforts by Formal People to reduce the Complexity • Reachability analysis • Design state abstraction • Design decomposition

35. Semi-Formal Verification - Assertion • Assertion-based verification (ABV) • “Assertion” is a statement on the intended behavior of a design. • The purpose of assertion is to ensure consistency between the designer’s intention and the implementation.

36. Semi-Formal Verification - Assertion • Simulation Quality of assertion-based verification Simulation with assertions Efficiency of assertion Number of bugs found Formal verification Simulation Time, Effort Setup testbench Describe assertions By IBM in “Computer-Aided Verification” 2000

37. Semi-Formal Verification - Coverage • Coverage-directed verification • Increase the probability of bug detection by checking the ‘quality’ of stimulus • Used as a guide for the generation of input stimulus Test Plan (Coverage Definition) Directives Random Test Generator Test Vectors Coverage Reports Coverage analysis Simulation

38. Semi-Formal Verification • Pros • Designer can measure the coverage of the test environment as the formal properties are checked during simulation. • Cons • Simulation speed is degraded as the properties are checked during simulation. • Challenges • Guiding the direction of test vectors to increase the coverage of the design. • Development of more efficient coverage metric to represent the behavior of the design.

39. Speed Comparison Speed (Cycles/sec, log scale) 10MHz 1~10MHz 500KHz 1MHz 100kHz 100 kHz 10 kHz 50-70Hz 100Hz 0 kHz Software Simulation Hardware- Accelerated Simulation (from Quickturn/Dynalith Presentation) Hardware emulation (from Quickturn presentation) Prototyping Semi-formal (Assertion-based verification)

40. Verification Time vs. Coverage (not to exact scale): Which best fits your application? Coverage Emulation /Simulation Acceleration Semi-formal Prototyping Simulation Verification Time Semi-formal setup Emulation /SA setup Prototyping setup Simulation setup

41. Agenda • Verify early and in-system • Why Verification ? • Verification Alternatives • Verification with Progressive Refinement • Flexible SoC verification environment • Debugging features • Cycle vs. transaction mode verification • SoC Verification • Concluding Remarks

42. Criteria for Good SoC Verification Environment • Support various abstraction levels • Support heterogeneous design languages • Trade-off between verification speed and debugging features • Co-work with existing tools • Progressive refinement • Platform-based design

43. Behavior model --- --- --- --- --- --- Transaction-Level Modeling • Model the bus system in transaction-level • No notion of exact time. • But precedence relation of each functional block is properly modeled. • Rough estimation on performance is possible. • Used as the fastest reference model by each block designer Software design Hardware design Core model Memory IP IP ISS Rough Performance Estimation ... ... transaction Transaction-level Bus model

44. Cycle-Accurate Bus Modeling • For more accurate modeling • Build a cycle-accurate system model in C or SystemC • Replace the transaction-level bus model with a cycle-accurate bus model • ARM released a “Cycle-Level Interface Specification” for this abstraction level. Core model Behavior model Memory RTL model IP ISS --- --- --- --- IP transaction Accurate Performance Estimation ... ... --- --- BFM BFM BFM Cycle-accurate Bus model

45. AMBA AHB CLI Specification • AMBA AHB Cycle Level Interface (CLI) Specification • CLI spec defines guidelines for TLM of AHB with SystemC. • Interface methods • Data structures • Header files for SystemC models • CLI spec leaves the detailed implementation of the AHB bus model to the reader. Read 10 data from slaveX starting Address 0x10 HADDR master master HWRITE HRDATA HTRANS slave slave Cycle-level modeling Transaction-level modeling

46. AMBA AHB CLI Specification • Example master implementation • Transactions are represented as methods in transaction-level modeling. • Abstraction levels of each method can be decided as needed such as cycle-count accurate, cycle-accurate, transaction accurate, etc. Header Body masterA.h masterA.cpp SC_MODULE(masterA) { ... ... void run(); }; void masterA::run() { bus_request(); has_grant(); init_transaction(); read_from_slave(); ... } The granularity of transactions are decided according to the verification needs.

47. Flexible SoC Verification Environment Algorithm • Build C reference model for the target application. • Setup of platform-level verification environment as early as possible JPEG decoding Functional block model HEAD VLD IDCT DISP Verification Environment Setup Platform-level model HEAD VLD IDCT DISP SW Model TRS TRS TRS TRS HW Model Memory Timer INTC

48. Flexible SoC Verification Environment Algorithm • Transactor connects various types of SW block models with HW bus system. • Several types of transactors must be prepared, for example • AHB  C model • AHB  HDL model • OPB  Testbuilder • PLB  SystemC JPEG decoding Functional block model HEAD VLD IDCT DISP Verification Environment Setup Platform-level model HEAD VLD IDCT DISP TRS TRS TRS TRS Memory Timer INTC

49. Flexible SoC Verification Environment • Socketize IP representation • HW: C  HDL  EDIF • SW: native C  ISS  Processor Core C to HDL Synthesis C HDL EDIF HW part model Transactor Transactor Transactor AHB AHB AHB Transactor Transactor Processor SW part model C ISS Cross-compiler

50. Cycle-Level Transactor • Generate stimulus at every clock cycle • Check the result of DUT at every clock cycle DUT Cycle-level transactor Testbench Device Driver PCI Controller DUT Testbench PCI Channel S/W simulation part FPGA part