1 / 22

Scoreboarding

Scoreboarding. FPGAworld 2018. Scoreboard. Advanced Testbench Scoreboard. Packet transmission. 213. 2. Received. Lost. 215. Expected. 4. 3. 1. Illegal dest. Illegal ctrl. Both. 1. 8. Frame error. CRC Error. 204. Correctly received. Normal, simple TB Scoreboard.

vgary
Download Presentation

Scoreboarding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Scoreboarding FPGAworld 2018

  2. Scoreboard Scoreboarding

  3. Advanced Testbench Scoreboard Packet transmission 213 2 Received Lost 215 Expected 4 3 1 Illegal dest Illegal ctrl Both 1 8 Frame error CRC Error 204 Correctly received Scoreboarding

  4. Normal, simple TB Scoreboard Frame transmission #Checked 215 #Total 213 6 #Failed 207 #Passed Scoreboarding

  5. Even simpler Let’s start withthe «simplest possible». Weneed:- Checks- Mismatch reports- Alert summary - Action on alerts- etc… Testbench summary 3 #Notes #Warnings 4 2 #Errors 0 #Failures Scoreboarding

  6. Simple data path TB check_value(out_val, exp_val, ERROR, "Byte #" & to_string(cnt)); May use UVVMUtility Library check_value(***); await_value(***); etc.... report_alert_counters(***) TB:=============================================== TB: ERROR: TB: 192 ns. filter_tb TB: value was: 'xFF'. expected 'x00'. TB: Byte #1053 TB:=============================================== p_main (test-sequencer) BFM ena_apply_data ena_fetch_data ============================================================== TB: *** SUMMARY OF ALL ALERTS *** TB: ========================================================= TB: REGARDED EXPECTED IGNORED Comment? TB: NOTE : 0 0 0 ok TB: TB_NOTE : 0 0 0 ok TB: WARNING : 0 0 0 ok TB: TB_WARNING : 0 0 0 ok TB: MANUAL_CHECK : 0 0 0 ok TB: ERROR : 1 0 0 Fail TB: TB_ERROR : 0 0 0 ok TB: FAILURE : 0 0 0 ok TB: TB_FAILURE : 0 0 0 ok TB: ========================================================== TB: >> No mismatch between counted and expected serious alerts TB: ========================================================== DUT(e.g. Filter) p_apply_data p_fetch_data BFM BFM in out Input stimuli Exp. Output Model Scoreboarding

  7. Simple data path TB Dead simple testbench. But a verygood start! Check check_value(out_val, exp_val, ERROR, "Byte #" & to_string(cnt)); May use Utility Library check_value(***); await_value(***); etc.... report_alert_counters(***) TB:=============================================== TB: ERROR: TB: 192 ns. filter_tb TB: value was: 'xFF'. expected 'x00'. TB: Byte #1053 TB:=============================================== p_main (test-sequencer) …… Recap …. Weneed:- Checks- Mismatch reports- Alert summary - Action on alerts- etc… BFM ena_apply_data ena_fetch_data Simple “Scoreboardish” ============================================================== TB: *** SUMMARY OF ALL ALERTS *** TB: ========================================================= TB: REGARDED EXPECTED IGNORED Comment? TB: NOTE : 0 0 0 ok TB: TB_NOTE : 0 0 0 ok TB: WARNING : 0 0 0 ok TB: TB_WARNING : 0 0 0 ok TB: MANUAL_CHECK : 0 0 0 ok TB: ERROR : 1 0 0 Fail TB: TB_ERROR : 0 0 0 ok TB: FAILURE : 0 0 0 ok TB: TB_FAILURE : 0 0 0 ok TB: ========================================================== TB: >> No mismatch between counted and expected serious alerts TB: ========================================================== DUT(e.g. Filter) p_apply_data p_fetch_data BFM BFM Mismatch report in out - Simple checkers - No autonomous checks  Pass vs Fail  Error++ count  No other statistics Input stimuli Exp. Output Model Alert summary Scoreboarding

  8. BFM Simple data communication BFM Simple “Scoreboardish” May use Utility Library and provided BFMs p_main (test-sequencer) SBI: Simple Bus Interface ~= Simple Avalon/AXI-lite Check uart_transmit(x"2A") sbi_check(C_RX, x"2A") - Simple checkers - No autonomous checks  Pass vs Fail  Error++ count  No other statistics Check sbi_write(C_TX, x"B3") Free, Open source BFMs: UART, AXI4-lite, SPI, I2C, Avalon MM, AXI4-stream, GPIO, SBI, ... uart_expect(x"B3") SBI DUT (UART) RX TX TB: 172 ns. uart_tbuart_transmit(x2A) on UART RX TB: 192 ns. uart_tb sbi_check(x1, ==> x2A) completed. From UART RX Progress report Alert summary ======================================================== TB: *** SUMMARY OF ALL ALERTS *** TB: ========================================================= TB: REGARDED EXPECTED IGNORED Comment? TB: NOTE : 0 0 0 ok TB: TB_NOTE : 0 0 0 ok TB: WARNING : 0 0 0 ok TB: TB_WARNING : 0 0 0 ok TB: MANUAL_CHECK : 0 0 0 ok TB: ERROR : 1 0 0 ok TB: TB_ERROR : 0 0 0 ok TB: FAILURE : 0 0 0 ok TB: TB_FAILURE : 0 0 0 ok TB: ========================================================== TB: >> No mismatch between counted and expected serious alerts TB: ========================================================== TB: 192 ns. uart_tb sbi_write(x2, ==> xB3) completed. To UART TX TB: ERROR: TB: 192 ns. uart_tb TB: value was: 'xB2'. expected 'xB3'. TB: (From uart_expect(xB3)) TB:============================================================== Mismatch report Scoreboarding

  9. UVVM entry level : Utility Library • A basic VHDL testbench infrastructure • Logging and verbosity control • Alert handling incl. optional positive acknowledge • Extended string handling • Simplified randomisation • check_value(), check_value_in_range() • check_stable(), await_stable() • await_change(), await_value() • report_alert_counters() • BFM support and examples • etc.... • Download from Github: 3 min • Include library in code: 1 min • Log + Check + Report: 3 min------------------------------------ •  Up and running: 7 min • Extremely low user threshold • Advanced funct. when needed • Well documented • Good overview in less than an hour Scoreboarding

  10. Multiple Scoreboard definitions DUT apply_data fetch_data in out Statistics 1:1 ??? seq FIFO Compare/checkers Frame transmission Model FIFO Queue 215 #Checked #Total 213 Input file Exp. file 6 #Failed 207 #Passed Scoreboarding

  11. "Scoreboard" includes more DUT apply_data fetch_data in out Statistics seq Scoreboard FIFO Compare/checkers Frame transmission Model FIFO Queue 215 #Checked #Total 213 Input file Exp. file Completion detection 6 #Failed 207 #Passed Scoreboarding

  12. Precap - Scoreboard in UVVM UART_SB Statistics Statistics Compare Actual data Expected data Queue Scoreboarding

  13. Multiple interfaces - Simple handling Simple distributed scoreboarding • Stimuli and checking distributed to VVCs • Scoreboarding distributed to VVCs DUT SBI uart 1 Seq. no-loss mux? A uart 2 UART_VVC,1 UART_1 C B SBI SBI_VVC,1 UART_2 UART_VVC,2 UART_3 UART_VVC,3 TB command & data Dedicated verification entity per interface  Better architecture and overview  Improves reuse  Allows simple individual interface control DUT aksess & data Scoreboarding

  14. Randomization & Functional Coverage write(SBI_VVCT,1, C_TX1, RANDOM, UVVM_FIFO_4, 300, "300 bytes"); • Shared data FIFO (slv) can be accessed by all • Allows local generation of data in VVCs • Destination VVC now checks against UVVM_FIFO • Shared data FIFO acts as a simple scoreboard • Less sequencer involvement DUT UVVM_FIFOshared_data_fifo expect(UART_VVCT,1, UVVM_FIFO, UVVM_FIFO_4, 300, "300 bytes"); uart 1incl. filter SBI no-loss mux? A uart 2 SimpleScoreboarding C B Seq. UART_VVC,1 UART_1 SBI SBI_VVC,1 UART_2 UART_VVC,2 UART_3 UART_VVC,3 TB command & data TB single command DUT aksess & data Scoreboarding

  15. Scoreboard Statistics Actual Expected Randomization & Functional Coverage write(SBI_VVCT,1, C_TX1, RANDOM, UVVM_FIFO_4, 300, "300 bytes"); • Shared data FIFO (slv) can be accessed by all • Allows local generation of data in VVCs • Destination VVC now checks against UVVM_FIFO • Shared data FIFO acts as a simple scoreboard • Less sequencer involvement DUT UVVM_FIFOshared_data_fifo expect(UART_VVCT,1, UVVM_FIFO, UVVM_FIFO_4, 300, "300 bytes"); uart 1incl. filter SBI no-loss mux? A uart 2 SimpleScoreboarding C B Seq. UART_VVC,1 UART_1 SBI SBI_VVC,1 UART_2 UART_VVC,2 UART_3 UART_VVC,3 UART_SB TB command & data TB single command DUT aksess & data Scoreboarding

  16. Scoreboard in UVVM Standard UVVM Quick Reference is provided Configuration record: • scope • allow_lossy • allow_out_of_order • mismatch_alert_level • etc... • logging/reporting • flushing queue • clearing statistics Statistics Statistics Compare Actual data Expected data Queue • generic data type • supports out of order • supports loss of data • tag and payload search Counting: • entered • pending • matched • mismatched • dropped • deleted • initial garbage • insert, delete, fetch • ignore_initial_mismatch • indexed on either entry or position • optional source element (in addition to expected + actual) Scoreboarding

  17. Advanced verification MP = My Protocol • Model • Models DUT behaviour • Receives transaction on DUT input • Generates expected data • Passes expected data onto Scoreboard MP_model SBI_SB Seq. DUT Scoreboard is directly connected to corresponding VVC SBI_VVC MP_VVC MP SBI SBI VVC? Some func. VVC? Scoreboard  Autonomous checks  Pass vs Fail  Error++ count  Multiple statistics Scoreboarding

  18. Using monitors MP = My Protocol • Monitor • Analyses transaction directly on the DUT interface • Passes transaction to the Model MP_model SBI_SB Seq. MP_MON,1 DUT SBI_VVC MP_VVC MP SBI SBI VVC? Some func. VVC? Scoreboarding

  19. Hierarchical Verification Components MP_SB MP = My Protocol • Challenge: • Packets sent to Word oriented receiver and scoreboard • Hierarchical VVCs • With their own Scoreboards • Sequencer can access any level at any time • Hierarchical VVCs will simplify complex TBs significantly MP_model MP_VVC SBI_SB Seq. MP_MON,1 DUT SBI_VVC MP_VVC MP SBI SBI VVC? Some func. VVC? Scoreboarding

  20. Quality and Efficiency enablers Structure & Architecture Structure & Architecture Simplicity UVVM with - VVCs - Scoreboards - Monitors - Controlled CR+FC  Yields all of this Overview, Readability Modifiability, Maintainability, Extensibility Debuggability Reusability Scoreboarding

  21. Pick and choose • Pick any Utility Library functionality: (from these plus more) • Pick any BFM - or any VVC - or any combination, with any cmd • Pick any FIFO, QUEUE, SCOREBOARD log() alert() error() manual_check() check_stable() await_stable() check_value() await_value() check_value_in_range() await_change() random() randomize() report_***() enable_log_msg() justify() fill_string() to_upper() replace() await_unblock_flag() clock_generator() await_barrier() GPIO SBI AXI4-lite SPI UART CLOCK_GENERATOR AVALON AXI4-stream I2C write() check() transmit() receive() insert_delay() Scoreboarding

  22. UVVM is gaining momentum github.com/UVVM bitvis.no/course-calendar • UVVM VVC Framework - Released February 2016 • Great feedback from users • Recommended by Doulos for Testbench Architecture • Supported by increasing number of tool vendors • ESA (European Space Agency) UVVM extensions • Included in EDA Playground • Major usage world-wide • Wilson Research will be presented Oct. 10 • Compare: Started from zero 2.5 years ago! All this - in only just over two years ... Scoreboarding

More Related