Stratified Sampling for Fault Coverage of VLSI Systems

1. Stratified Sampling for Fault Coverage of VLSI Systems Vishwani D. Agrawal Agere Systems, Murray Hill, NJ 07974 va@agere.com http://cm.bell-labs.com/cm/cs/who/va September 26, 2001 Collaborators: Pradip Thaker, Acorn Networks, and Mona Zaghloul, GWU Agrawal: Stratified Sampling

2. VLSI System Design Register-transfer level (RTL) design and verification 90-100% stuck-at fault coverage required Logic synthesis Test generation Timing and physical design Design and test data for manufacturing Agrawal: Stratified Sampling

3. Problem • Accurately estimate the gate-level fault coverage for a VLSI system at the RT-level • Advantages: • Improve test • Improve design • Avoid expensive design changes • Previous approaches do not accurately represent gate-level fault coverage (function errors, mutation, statement faults, branch faults, etc.) Agrawal: Stratified Sampling

4. Solution • Model faults as representative sample of the targeted (gate-level stuck-at) faults. • Treat the coverage in an RTL module as a statistical sampling estimate. • For a multi-module VLSI system, combine module coverages according to the stratified sampling technique. Agrawal: Stratified Sampling

5. Outline of Talk • Introduction to fault sampling. • RTL fault model and application to modules. • Coverage in a multi-module system: • Need for stratified sampling • Stratum weights • Experimental results • Conclusion • References Agrawal: Stratified Sampling

6. Fault Sampling • A randomly selected subset (sample) of faults is simulated. • Measured coverage in the sample is used to estimate fault coverage in the entire circuit. • Advantage: Saving in computing resources (CPU time and memory.) • Disadvantage: Limited data on undetected faults. Agrawal: Stratified Sampling

7. Random Sampling Model Detected fault Undetected fault All faults with a fixed but unknown coverage Random picking Np = total number of faults (population size) C = fault coverage (unknown) Ns = sample size Ns << Np c = sample coverage (a random variable) Agrawal: Stratified Sampling

8. Probability Density of Sample Coverage, c (x--C )2 -- ------------ 1 2s2 p (x ) = Prob(x < c < x +dx ) = -------------- e s (2 p)1/2 C (1 - C) Variance, s 2 = ------------ Ns Sampling error s s p (x ) Mean = C x 1.0 C +3s C -3s x C Sample coverage Agrawal: Stratified Sampling

9. Sampling Error Bounds C (1 - C ) | x - C | = 3 [ -------------- ]1/2 Ns Millot, 1923 Solving the quadratic equation for C, we get the 3-sigma (99.8% confidence) estimate (Agrawal-Kato, 1990): 4.5 C3s = x ------- [1 + 0.44 Nsx (1 - x )]1/2 Ns Where Ns is sample size and x is the measured fault coverage in the sample. Example: A circuit with 39,096 faults has an actual fault coverage of 87.1%. The measured coverage in a random sample of 1,000 faults is 88.7%. The above formula gives an estimate of 88.7% 3%. CPU time for sample simulation was about 10% of that for all faults. Agrawal: Stratified Sampling

10. An RTL Fault Model(ITC-2000) • Language operators are assumed to be fault-free • Variables (map onto signal lines) contain faults • stuck-at-0 • stuck-at-1 • Only one fault is applied at a time (single fault assumption) Agrawal: Stratified Sampling

11. RTL Fault Injection • Not affected by faults: • Synthetic operators + - * >= <= == != • Boolean operators & | ^ ~ • Logical operators && || ! • Sequential elements (flip-flops & latches) • Faults introduced in signal variables (stems and fan-outs) • Separate faults for bits of data words Agrawal: Stratified Sampling

12. Fault Modeling for Boolean Operators Agrawal: Stratified Sampling

13. Stem and Fan-out Fault Modeling • RTL fan-out faults: if(X) then Z=Y; else Z=!Y; • Unique RTL fault is placed on each fan-out of each bit of a variable • Unique RTL fault on each stem Agrawal: Stratified Sampling

14. More RTL Faults Agrawal: Stratified Sampling

15. Observations and Assumption: RTL Faults • RTL faults may have detection probability distribution similar to that of collapsed gate-level faults • Statistically, an RTL fault-list approximates a random sample from the gate-level fault-list • Number of RTL faults vs. gate-level faults depends on • Level of RTL description • Synthesis procedure used to convert RTL to gate level Agrawal: Stratified Sampling

16. RTL Fault Simulation • Analogous to gate-level approach • Faults injected in RTL code of the design description by a C++ parser; a simulatable logic buffer element inserted at fault site • Fault report contains statistics on detected and undetected RTL faults • Cadence’s Verifault-XL used as RTL fault simulator Agrawal: Stratified Sampling

17. Estimation Error for Module Fault Coverage • RTL fault coverage assumed to be an estimate of the collapsed gate-fault coverage within statistical bound [Agrawal and Kato, D&T, 1990]: a = 3.00 for confidence probability of 99.8% c = ratio of detected to total number of RTL faults M = number of gate faults N = number of RTL faults, k = 1 - N/M Agrawal: Stratified Sampling

18. DSP Interface Module(3,168 Gates) Agrawal: Stratified Sampling

19. RTL Faults and VLSI System Coverage • Experimental results demonstrate RTL fault coverage of a module to be a good statistical estimate of the gate-level fault coverage • A VLSI system consists of many interconnected modules • Overall RTL fault-list of a VLSI system does not constitute a representative sample of the gate-level fault-list Agrawal: Stratified Sampling

20. Error at System Level Gate- level M1 150 faults 90% cov. RTL M1 100 faults 91% cov. RTL Coverage = (0.91 x 100 + 0.39 x 100) / 200 = 65% Gate Coverage = (0.90 x 150 + 0.40 x 400) / 550 = 54% • A correct estimation of gate-level fault coverage from RTL coverage: M2 400 faults 40% cov. M2 100 faults 39% cov. 91 x (150 / 550) + 39 x (400 / 550) = 53% Agrawal: Stratified Sampling

21. Application of Stratified Sampling • Fault population of a VLSI system divided into strata according to RTL module boundaries • RTL faults in each module are considered a sample of corresponding gate-level faults • The stratified RTL coverage is an estimate of the gate-level coverage: Wm = stratum weight of mth module = Gm/G cm = RTL fault coverage of mth module Gm = number of gate-level faults in mth module G = number of all gate-level faults in the system M = number of RTL modules in the system M C=SWmcm m=1 Agrawal: Stratified Sampling

22. Application of Stratified Sampling C + t s • Range of coverage, s2= --------cm(1 - cm) M Wm S where, rm- 1 m=1 rm = number of RTL faults in mth module t = value from tables of normal distribution The technique requires knowledge of stratum weights and not absolute values of Gm and G Agrawal: Stratified Sampling

23. Stratum Weight Extraction Techniques • Logic synthesis based weight extraction Wm = Gm/G • Floor-planning based weight extraction Wm = Am/A • Entropy-measure based weight extraction Agrawal: Stratified Sampling

24. Experimental Procedure • Technology-dependent weight extraction • Several unique gate-level netlists obtained by logic synthesis from the same RTL code • Each synthesis run performed using a different set of constraints, e.g., area optimization (netlist 1), speed optimization (netlist 2), or combined area and speed optimizations (netlists 3 and 4) • Strata weights calculated using gate-level fault lists of various synthesized netlists • Technology-independent weight extraction • Stratum weights calculated using area distribution among modules • Each set of stratum weights used to calculate RTL fault coverage and error bounds • Impact of estimation error investigated Agrawal: Stratified Sampling

25. Experimental Data: Weight Distributions Agrawal: Stratified Sampling

26. Experimental Data: RTL Fault Coverage Agrawal: Stratified Sampling

27. Experimental Data: Error Bounds Agrawal: Stratified Sampling

28. Timing Controller ASIC (17,126 Gates) Agrawal: Stratified Sampling

29. A DSP ASIC(104,881 Gates) Agrawal: Stratified Sampling

30. Conclusion • Main ideas of RTL fault modeling • A small or high-level RTL module contributes few RTL faults, but large statistical tolerance gives a correct coverage estimate • Stratified sampling accounts for varying module sizes and for different RTL details that may be used • Stratum weights appear to be insensitive to specific details of synthesis • Advantages of the proposed RTL fault model • High-level test generation and evaluation • Early identification of hard-to-test RTL architectures • Potential for significantly reducing run-time penalty of the gate-level fault simulation Agrawal: Stratified Sampling

31. References • V. D. Agrawal, “Sampling Techniques for Determining Fault Coverage in LSI Circuits,” J. Digital Systems, vol. V, no. 3, pp. 189-202, 1981. • V. D. Agrawal and H. Kato, “Fault Sampling Revisited,” IEEE Design & Test of Computers, vol. 7, no. 4, pp. 32-35, Aug. 1990. • P. A. Thaker, M. E. Zaghloul, and M. B. Amin, “Study of Correlation of Testability Aspects of RTL Description and Resulting Structural Implementation,” Proc. 12th Int. Conf. VLSI Design, Jan. 1999, pp. 256-259. • P. A. Thaker, V. D. Agrawal, and M. E. Zaghloul, “Validation Vector Grade (VVG): A New Coverage Metric for Validation and Test,” Proc. 17th IEEE VLSI Test Symp., Apr. 1999, pp. 182-188. • P. A. Thaker, Register-Transfer Level Fault Modeling and Evaluation Techniques, PhD Thesis, George Washington University, Washington, D.C., May 2000. • P. A. Thaker, V. D. Agrawal, and M. E. Zaghloul, “Register-Transfer Level Fault Modeling and Test Evaluation Techniques for VLSI Circuits,” Proc. Int. Test Conf., Oct. 2000, pp. 940-949. • This presentation is available from the website http://cm.bell-labs.com/cm/cs/who/va Agrawal: Stratified Sampling

32. Thank you Agrawal: Stratified Sampling