New Ways of Generating Large Realistic Benchmarks for Testing Synthesis Tools

1. New Ways of GeneratingLarge Realistic Benchmarksfor Testing Synthesis Tools Petr Fišer, Jan Schmidt Faculty of Information Technology Czech Technical University in Praguefiserp@fit.cvut.cz, schmidt@fit.cvut.cz

2. Outline • Motivation • New benchmark generation methods • Experimental results • Conclusions IWSBP 2010, Freiberg

3. Motivation … why another artificial benchmark generator? • To test logic synthesis tools • Capabilities of synthesis processes • Immunity to “bad” structures • Ability to discover “good” structures • Iterative power • Scalability • … IWSBP 2010, Freiberg

4. Motivation J. Cong, K. Minkovich: Optimality study of logic synthesisfor LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 • They created artificially large circuits, functionally equivalent to their small origins (70 LUTs) • Synthesis produced 10k – 30k LUTs IWSBP 2010, Freiberg

5. Motivation P. Fišer, J. Schmidt, J: Small but Nasty Logic Synthesis Examples, IWSBP'08 • XOR tree is appended to the circuit outputs and the circuit is collapsed • Synthesis produced>400 LUTs instead of 11 IWSBP 2010, Freiberg

6. Motivation Will my synthesis tool produce the same result for different descriptions (versions) of one particular circuit? (a.k.a. iterative power) Most probably not!(if things go bad) What went wrong? What descriptions are bad for me? What structures caused my failure? What should I do to perform better? IWSBP 2010, Freiberg

7. Proposed Benchmarks • Starting with seed circuit (could be small) • Functionally equivalent “big” circuit is created • The size of the benchmark circuit is adjustable Ideal case: Bench circuit 1 Transformation 1 Synthesis Result Seed circuit Bench circuit 2 Transformation 2 Synthesis Bench circuit 3 Transformation 3 Synthesis IWSBP 2010, Freiberg

8. Proposed Benchmarks • Starting with seed circuit (could be small) • Functionally equivalent “big” circuit is created • The size of the benchmark circuit is adjustable Real case: Bench circuit 1 Result 1 Transformation 1 Synthesis Seed circuit Bench circuit 2 Result 2 Transformation 2 Synthesis Bench circuit 3 Result 3 Transformation 3 Synthesis IWSBP 2010, Freiberg

9. Cong’s LEKU Benchmarks J. Cong, K. Minkovich: Optimality study of logic synthesisfor LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 LEKU = Logic Examples with Known Upper Bound • Based on elimination of the original circuit structure • … and bad decomposition IWSBP 2010, Freiberg

10. 1. Realistic LEKU Benchmarks • Any circuit may be used as a seed (instead of g25) • Possible chance of success • Global BDDs may be used instead of collapsing • Upper bound = size of the original circuit IWSBP 2010, Freiberg

11. 1. Realistic LEKU Benchmarks Size increase by collapsing 250 ISCAS and IWLS benchmarks Size increase in 61% of circuits IWSBP 2010, Freiberg

12. 1. Realistic LEKU Benchmarks Experimental results IWSBP 2010, Freiberg

13. 2. Parity Benchmark Circuits • XOR tree is appended to the circuit outputs, then the structure is destroyed (collapsing, BDD) • No guarantee of circuit size increase • Upper bound = size of the core circuit + XOR tree IWSBP 2010, Freiberg

14. 2. Parity Benchmark Circuits Size increase by appending parity & collapsing 100 ISCAS and IWLS benchmarks Size increase in 25% of circuits IWSBP 2010, Freiberg

15. 2. Parity Benchmark Circuits Experimental results IWSBP 2010, Freiberg

16. 3. Tautology Benchmarks • Large random SOP is generated • When the number of terms exceeds some threshold, the SOP is a tautology • Then, the big SOP is mapped into 2-input gates(SIS tech_decomp) • Big network • Upper bound = 0 • The benchmark size may be adjusted by • Number of input variables • Dimension of SOP terms IWSBP 2010, Freiberg

17. 4. Partial Collapsing Only parts of the network are collapsed • Choose one pivot gate • Extract its transitive fan-in and fan-out to a given radius • Collapse the extracted network part • Decompose into 2-input gates • Put it back • Iterate several times • Upper bound = size of the original circuit • The benchmark size may be adjusted by • Size of the extracted circuit • Number of iterations IWSBP 2010, Freiberg

18. 4. Partial Collapsing Example – c432 IWSBP 2010, Freiberg

19. 4. Partial Collapsing Example – big tautology IWSBP 2010, Freiberg

20. 4. Partial Collapsing Example – big tautology IWSBP 2010, Freiberg

21. 4. Partial Collapsing Experimental results IWSBP 2010, Freiberg

22. 5. Replicating Shared Logic Duplicate a part of the logic that is shared • Find a branching signal • Duplicate its transitive fan-in, to a given depth • Upper bound = size of the original circuit • The benchmark size may be adjusted by • Number of duplicated branches • Depth of duplication IWSBP 2010, Freiberg

23. 5. Replicating Shared Logic Experimental results IWSBP 2010, Freiberg

24. 6. Adding Inverters (special bonus – not included in the proceedings) • Add pairs of inverters to random locations • The network size may be arbitrarily expanded • And all the synthesis tools… Are completely immune to this! IWSBP 2010, Freiberg

25. Summary Experiments IWSBP 2010, Freiberg

26. Summary Experiments IWSBP 2010, Freiberg

27. Conclusions • Several new benchmark generation methods proposed • Artificially “big” circuits are generated from seed circuits • Benchmarks are functionally equivalent to the seed circuits  the complexity upper bound is known • Tested on ABC and 2 commercial tools Unfortunate result – the bigger the circuit going to synthesis, the bigger the result IWSBP 2010, Freiberg