New Ways of Generating Large Realistic Benchmarks for Testing Synthesis Tools

New Ways of GeneratingLarge Realistic Benchmarksfor Testing Synthesis Tools Petr Fišer, Jan Schmidt Faculty of Information Technology Czech Technical University in Praguefiserp@fit.cvut.cz, schmidt@fit.cvut.cz

Outline • Motivation • New benchmark generation methods • Experimental results • Conclusions IWSBP 2010, Freiberg

Motivation … why another artificial benchmark generator? • To test logic synthesis tools • Capabilities of synthesis processes • Immunity to “bad” structures • Ability to discover “good” structures • Iterative power • Scalability • … IWSBP 2010, Freiberg

Motivation J. Cong, K. Minkovich: Optimality study of logic synthesisfor LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 • They created artificially large circuits, functionally equivalent to their small origins (70 LUTs) • Synthesis produced 10k – 30k LUTs IWSBP 2010, Freiberg

Motivation P. Fišer, J. Schmidt, J: Small but Nasty Logic Synthesis Examples, IWSBP'08 • XOR tree is appended to the circuit outputs and the circuit is collapsed • Synthesis produced>400 LUTs instead of 11 IWSBP 2010, Freiberg

Motivation Will my synthesis tool produce the same result for different descriptions (versions) of one particular circuit? (a.k.a. iterative power) Most probably not!(if things go bad) What went wrong? What descriptions are bad for me? What structures caused my failure? What should I do to perform better? IWSBP 2010, Freiberg

Proposed Benchmarks • Starting with seed circuit (could be small) • Functionally equivalent “big” circuit is created • The size of the benchmark circuit is adjustable Ideal case: Bench circuit 1 Transformation 1 Synthesis Result Seed circuit Bench circuit 2 Transformation 2 Synthesis Bench circuit 3 Transformation 3 Synthesis IWSBP 2010, Freiberg

Proposed Benchmarks • Starting with seed circuit (could be small) • Functionally equivalent “big” circuit is created • The size of the benchmark circuit is adjustable Real case: Bench circuit 1 Result 1 Transformation 1 Synthesis Seed circuit Bench circuit 2 Result 2 Transformation 2 Synthesis Bench circuit 3 Result 3 Transformation 3 Synthesis IWSBP 2010, Freiberg

Cong’s LEKU Benchmarks J. Cong, K. Minkovich: Optimality study of logic synthesisfor LUT-based FPGAs, IEEE Trans. on CAD, vol. 26, 2007 LEKU = Logic Examples with Known Upper Bound • Based on elimination of the original circuit structure • … and bad decomposition IWSBP 2010, Freiberg

1. Realistic LEKU Benchmarks • Any circuit may be used as a seed (instead of g25) • Possible chance of success • Global BDDs may be used instead of collapsing • Upper bound = size of the original circuit IWSBP 2010, Freiberg

1. Realistic LEKU Benchmarks Size increase by collapsing 250 ISCAS and IWLS benchmarks Size increase in 61% of circuits IWSBP 2010, Freiberg

1. Realistic LEKU Benchmarks Experimental results IWSBP 2010, Freiberg

2. Parity Benchmark Circuits • XOR tree is appended to the circuit outputs, then the structure is destroyed (collapsing, BDD) • No guarantee of circuit size increase • Upper bound = size of the core circuit + XOR tree IWSBP 2010, Freiberg

2. Parity Benchmark Circuits Size increase by appending parity & collapsing 100 ISCAS and IWLS benchmarks Size increase in 25% of circuits IWSBP 2010, Freiberg

2. Parity Benchmark Circuits Experimental results IWSBP 2010, Freiberg

3. Tautology Benchmarks • Large random SOP is generated • When the number of terms exceeds some threshold, the SOP is a tautology • Then, the big SOP is mapped into 2-input gates(SIS tech_decomp) • Big network • Upper bound = 0 • The benchmark size may be adjusted by • Number of input variables • Dimension of SOP terms IWSBP 2010, Freiberg

4. Partial Collapsing Only parts of the network are collapsed • Choose one pivot gate • Extract its transitive fan-in and fan-out to a given radius • Collapse the extracted network part • Decompose into 2-input gates • Put it back • Iterate several times • Upper bound = size of the original circuit • The benchmark size may be adjusted by • Size of the extracted circuit • Number of iterations IWSBP 2010, Freiberg

4. Partial Collapsing Example – c432 IWSBP 2010, Freiberg

4. Partial Collapsing Example – big tautology IWSBP 2010, Freiberg

4. Partial Collapsing Experimental results IWSBP 2010, Freiberg

5. Replicating Shared Logic Duplicate a part of the logic that is shared • Find a branching signal • Duplicate its transitive fan-in, to a given depth • Upper bound = size of the original circuit • The benchmark size may be adjusted by • Number of duplicated branches • Depth of duplication IWSBP 2010, Freiberg

5. Replicating Shared Logic Experimental results IWSBP 2010, Freiberg

6. Adding Inverters (special bonus – not included in the proceedings) • Add pairs of inverters to random locations • The network size may be arbitrarily expanded • And all the synthesis tools… Are completely immune to this! IWSBP 2010, Freiberg

Summary Experiments IWSBP 2010, Freiberg

Conclusions • Several new benchmark generation methods proposed • Artificially “big” circuits are generated from seed circuits • Benchmarks are functionally equivalent to the seed circuits  the complexity upper bound is known • Tested on ABC and 2 commercial tools Unfortunate result – the bigger the circuit going to synthesis, the bigger the result IWSBP 2010, Freiberg

New Ways of Generating Large Realistic Benchmarks for Testing Synthesis Tools