Functional Verification of HDL Models

1. Functional Verification of HDL Models Introduction to Verification Tehran University

2. Reference Book • Janick Bergeron [WTB] Writing Testbenches: Functional Verification Of HDL Models. • First Edition, Kluwer, 2000, ISBN: 0-7923-7766-4 • Second Edition, Kluwer, 2003, ISBN: 1-4020-7401-8 Tehran University

3. Other references • Comprehensive Functional Verification: The complete industry cycle • B. Wile, J. Goss, W. Roesner [IBM] • Principles of Functional Verification • Andreas Meyer • Functional Verification Coverage & Measurement and Analysis • Andrew Piziali • Advanced Verification Techniques: A SystemC Based Approach for Successful Tapeout • Leena Singh, L. Drucker, Neyaz Khan Tehran University

4. Course Goals Aim: To familiarise you with the routine tasks in verification management, and to give you the technical background plus some of the practical skills expected from a Design Verification Engineer. • Prerequisite: Hardware Language programming On successful completion of this unit, you will be able to: • understand the complexities and limits of verification; • carry out functional verification and determine its effectiveness; • set appropriate verification goals, select suitable verification methods and assess the associated risks; • compile a verification plan that fits into the flow of a design project Tehran University

5. 1974 Brian W. Kernighan • Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. Tehran University

6. Bugs: What is the Cost? • Time to Market is affected • Bugs found early in design have little cost! • Finding a bug at chip/system level has moderate cost: • Requires more debug time and isolation time. • Could require new algorithm, which could effect schedule and cause board rework. • Finding a bug in System Test (test floor) requires re-spin of a chip. • Finding a bug after customer delivery cost millons Tehran University

7. Cost of Bugs Over Time Number of bugs found Cost of finding bugs Time module chip system customer Tehran University

8. Importance of Verification • 70% of ASIC / IP / SoC design effort goes to verification • Is on the critical path • Has effects on Schedule, Cost, Quality • Fewer revisions through fabrication process means lower cost and faster time-to-market • Twice more Verification engineers than RTL designer • Re-spinning cost time and money • Getting the design right first time is an art • Due to chip complexity it is more and more difficult Tehran University

9. Silicon Validation times increase as features size decreases The difficulty is expected to increase at 65nm and below because existing ad-hoc methodologies do not scale with the unprecedented levels of SoC device complexity. Silicon Validation Tehran University

10. SoC Verification • Complete system-level verification of a complex SoC at 90 nm or below is not feasible pre-silicon • Even the most sophisticated SoC design methodology cannot fully account for all the parameters that impact silicon behavior, or for all logic "corner cases" that occur in the real life of a chip working at speed and in system [Miron Abramovici, Sep. 2007 ] • post-silicon validation has become an essential step in the design implementation methodology Tehran University

11. Importance of Verification (Cont.) • International Technology Roadmap for Semiconductors identified Verification as a crisis area (http://public.itrs.net/) • Shortfalls in qualified engineers and software-trained designers • Verification can be reduced through parallelism, abstraction and automation Tehran University

12. Course Outline Lecture Topics: • What is Verification? What is a Testbench? • Verification Tools (including Verilog basics) • Verilog HDL coding and Verification Plan • Assertion-based Verification • Advanced Testbench Design Methodology • Functional Formal Verification and Property Checking • Labs (Verilog, ModelSim, C/C++, SystemC) Tehran University

13. Introduction to IC Design Process • Specification • Behavioral Description Architectural Synthesis • RTL Description Logical Synthesis • Gate Level Description Physical Synthesis • Layout and tape-out In practice all steps almost start at the same time and run in parallel Tehran University

14. Steps in IC Design Process Architectural SynthesisInput: behavioral description • generates architecture for design at RTL • data path (interconnection of adders, multipliers, etc) • control unit (signals to control data path) • scheduling and binding Logic SynthesisInput: RTL description • converts into network of logic primitives (depending on design • style) • Two groups: • sequential/combinatorial Physical DesignInput: gate level description • generates geometric patterns that define physical layout of chip Tehran University

15. Verification Role in IC Design IC design is a complex process: • Conflict: design time (decreased) versus complexity (increased) • Aim: correct by construction, right-first-time design More and more time-consuming to obtain an acceptable level of confidence in correctness of design • Design time << Verification time • Needs its own trained verification engineers • 80% of written code is for the verification environment Tehran University

16. How does a designer know a circuit is correct? concept Verify: What you specified is what you envisioned Specification Verify: What you designed is what you specified HDL design (RTL) Verify: What you taped out is what the RTL described Tape out Verify: What you manufactured is what you taped out Silicon There is no single methodology that covers all! Tehran University

17. Some Definitions • What is Verification? • A process used to demonstrate the functional correctness of a design in its implementation  Hence the term Functional Verification • What is a Testbench? • Code used to generate predetermined input sequence to a design • Implemented using Verilog, VHDL, e or Open Vera, also include C routines • Always refers to test case/scenario Tehran University

18. Testbench and DUV • Completely closed systems • No inputs or outputs go in or out • Generic Structure of a testbench and DUV: Testbench Design Under Verification Stimulus Generator Checker Supply stimulus to checker for comparison with the specification Tehran University

19. Verification is on critical path Need to minimise verification time e.g. by using: • Parallelism: Add more resources • Abstraction: Higher level of abstraction (i.e. C vs Assembly, TLM) • This often means a reduction of control! • Automation: Tools to automate standard processes • Requires standard processes/methodology. • Usually a variety of functions, interfaces, protocols, and transformations must be verified. • Not all (verification) processes can be automated. • Randomization can be used as an automation tool Tehran University

20. Reconvergence Model • Conceptual representation of the verification process: Illustrate what is exactly being verified Transformation Verification Purpose of verification is to ensure that the result of some transformation is as intended or as expected. Tehran University

21. Source of ambiguity/error • Human Factor: • How a designer read and interpret the specification • Specification can be ambiguous • Designer can introduce errors by (mis-)interpretation (re-convergence model) DANGER: When a designer verifies her/his own design - then she/he is verifying her/his own interpretation of the design - not the specification! Tehran University

22. How to Reduce Human Errors Reduce human errors by: • Automation • Take the human out but not really feasible • Process not well defined enough, not formal enough • Make mistake-proof the human intervention by reducing it to simple steps (Poka-Yoka) • Redundancy • Two individuals check each other’s work • Costly mechanism but still cheaper than re-design and chip recall But never let a designer verify his own design Tehran University

23. What is being verified? • Choosing a common origin and the reconvergence points determines what is being verified. • Determined by the tool used to perform the verification • The following approaches have different origin and reconvergence points: • Formal Verification • Functional Verification • Testbench Generators Tehran University

24. Formal Verification This is not a tool that mathematically determines the correctness of a design without having to write testbenches! Build a formal mathematical model of some aspect of the system and calculate whether or not it has some desired property • 2 types of Formal Verification • Equivalence Checking • Model Checking Tehran University

25. Synthesis RTL Gate Equivalence Checking Equivalence Checking • Compares two models to check for equivalence • Mathematically proves that both are logically equivalent • Commonly used on lower level of design • Make sure that scan chain, clock tree insertion did not change the functionality • It is only interested in comparing boolean and sequential logic functions – no tech mapping • Why can’t we entirely trust the synthesis tool? Tehran University

26. Model Checking • Assertions or characteristics of a design are formally proven or disproved • Use to check for generic problems or violations of user-defined properties of the behavior of the design • Usually employed at higher levels of design process • Properties are derived from specification • Properties are expressed in some (temporal) logic • Checking often involves a (finite) state machine model Tehran University

27. Model Checking Problems: • Knowing which properties to check and how to express them needs expertise. • Very few designers are also good mathematicians. • (Verification engineers should be closing this gap.) • Very few user-friendly tools for model checking. • Tools that do exist can only check small designs. • There is a lot of research to make Model Checking more practically usable. For an introduction see ”Model Checking” by Clarke, Grumberg and Peled Tehran University

28. Functional Verification • Ensure that a design implements intended functionality • Without it, one must trust that the transformation of a specification into RTL code was performed correctly • We can only prove the presence of bugs, but we cannot prove their absence! Tehran University

29. Testbench Generators • Tools to generate stimuli to exercise code or expose bugs • Designer/Verification engineer input is still required. • RTL code is the origin and there is no reconvergence point • Verification engineer has responsibility to determine if the testbench applies valid stimuli • If used with parameters, one can control the generator in order to focus the testbench on more specific scenarios • 2 input ”and” gate - ... combinations of inputs (exhaustive) • Instruction stream generators to verify processor cores. Is exhaustive verification (always) feasible? Code coverage/proof Metrics RTL Testbench Testbench Generation Tehran University

30. Functional Verification Approaches There are 3 complementary approaches: • Black-Box Verification • White-Box Verification • Grey-Box Verification Tehran University

31. Black Box Verification • The black box has inputs, outputs, and performs some function. • The function may be well documented (or not ...). • To verify a black box, you need to understand the function and be able to predict the outputs based on the inputs. • The black box can be a full system, a chip, a unit of a chip, a module, etc. • No knowledge of the actual implementation is required • Independent of implementation. • Lacks controllability and observability. • Difficult to locate source of problem, only exposes effects • Example: round-robin arbiter Tehran University

32. White Box Verification • Opposite of Black Box • Full visibility and controllability of the internal structure and implementation of DUV • Easy to set up interesting conditions • But testbench is tied to a specific implementation – needs to be changed if design modified • Best for module or system level verification • Example: counter roll-over Tehran University

33. Grey Box Verification • Compromise between BB and WB • Like BB, it observes and controls top level interface signaling • But increases coverage metrics by adding some non-functional modification to increase visibility • Include additional software-accessible registers to observe internal states • In practice for SoC most environments are GB because prediction of results are sometimes impossible without looking at internal signals Tehran University

34. Perfect Verification • To fully verify a black box, you must show that the logic works correctly for all combinationsof inputs. This means: • Driving all permutationson the input lines. • Checking for proper results in all cases. • Full verification is not practical on large designs, but the principles are valid across all verification approaches Tehran University

35. HW Design Synthesis Manufacturing Spec HDL Gates Silicon Testbench Equivalence Checking Testing Verification versus Testing • Often Confused: • Test: show the design was manufactured correctly (post-silicon) – verifies internal nodes can be toggled by applying test vectors • Verification: Ensure the design meets its functional intent (pre-silicon) Tehran University

36. Verification versus Testing (Cont.) • One test method is scanning: Design For Test • Links all registers into a long shift register (chain) • Chain accessible from the I/O pins, observe and control internal states • Scan-based testing restricts the design (sync, no gated clock, single clock) but keeps cost low (reference book: Digital System Testing and Testable Design, Abramovici) Research topic: “Design for Verification”! Consider what the design is supposed to do? How will it be verified? Tehran University

37. Design and Verification Reuse • You won’t use what you don’t trust. • How to trust it? (Verify it) • For reuse, designs must be verified with more strict requirements • All claims, possible combinations and uses must be verified. • Not just how the design is (intended to be) used in one environment • Verification/testbenches re-usable to some degrees • BFMs have different requirements at block or system level • From project to project Tehran University

38. Cost of Verification • It is always too long and costs too much but is absolutely necessary • It does not generate revenue! • As number of bugs decreases, the cost and time for finding the remaining ones increases: When is the verification done? • Yet indispensable: • To create revenue, design must be functionally correct and provide benefits to customer. • Proper functional verification demonstrates trustworthiness of the design. • Right-first-time designs demonstrate professionalism and ”increase” reputation of design team Tehran University

39. When Is Verification Done? Never truly done on complex designs. • Remember: Verification can only show presence of errors, not their absence. • Skill and experience are needed to: • Determine WHAT needs to be verified. • Be selective/exhaustive. • Identify corner cases. • Given enough time, errors will be uncovered. • Critical question: • Is the error likely to be severe enough to warrant the effort spent to find/correct it? Verification is similar to statistical hypothesis testing. • Hypothesis ”under test” is: Is the design functionally correct? Tehran University

40. Hypothesis Matrix Fail Pass • Type I mistakes: Easy to identify - found error where none exists. • Type II mistakes: Most serious - verification failed to identify an error! • Can result in a bad design being shipped unknowingly! • Knowing where you are in the verification process is much easier to estimate than how long it will take to complete the job. Type II (False Positive) Bad Design Type I (False Negative) Good design Tehran University

41. Summary • Seen what a testbench is • Understood how important verification is and why it is so important. • Seen that parallelism, automation and abstraction are strategies to reduce the time necessary to implement testbenches. • Used reconvergence modelsto help us identify what is verified. • Been introduced to the various verification approaches that exist. • Investigated the importance of verification for design reuse. • Discussed the cost of verification • Next: Verification tools, especially simulators, waveform viewers and simulation languages (Verilog) Tehran University