The Evil’s in the Edits

1. The Evil’s in the Edits - Verifying “Golden” reused IPs William C Wallace Texas Instruments Nitin Jayaram Texas Instruments Vijay Kanumuri Texas Instruments Nitin Mhaske Atrenta Inc

2. Author Details William C Wallace: Senior Digital Designer, Texas Instruments 20+ years of experience, focused on using new methods to reduce risk and increase productivity when making modification on legacy IP Nitin Jayaram: Lead DV engineer, Texas Instruments 7+ years of experience, focused on new methods to increase DV productivity Nitin Mhaske: Sr Manager, Atrenta Inc nitinm@atrenta.com 13+ years of experience, leading verification for Semis and EDA

3. Abstract Time To Market makes IP reuse imperative in TI’s SoCs with expectations that reused IP development should be less efforts and less risks. However, most of the IPs need to go through some edits to meet current marketing requirement or meet constraints such as power, and timing. Lack of micro-architecture knowledge, poor documentation and rudimentary testing for reused IPs makes verification of new/modified RTL challenging and time consuming. Traditional quality control techniques fails to capture dependencies of these IP edits, risking the SoC. Assertion Synthesis techniques that learns design behavior based on simulation tests and RTL was used to solve the problem. These techniques automate process of finding functional coverage holes for the IP edits and guided test development to better coverage. With limited design knowledge, these coverage holes are harder to find using traditional coverage techniques. Assertions generated out of these techniques made sure that design edits did not affect the un-modified RTL and respected the SoC requirements.

4. Backdrop

5. SOCs are Heterogeneous Typical Internally Reused IP TI - SoC Typical Third Party Reused IP

6. The Problem Statement

7. Challenges With Reused IPs Almost never satisfies “current” SoC marketing requirements Other constraints such as timing, area etc SoC specific Power Intent Supported features, configurations Original (Micro-architecture) design Intent lost over time Poorly maintained documents Changed hand multiple times at multiple locations Original developers not available How much verification good enough? (Any design edits may affect many features) Lacks functional Coverage, assertions No standardization like UVM; makes reuse harder Existing Env often directed; so cannot trust quality

8. Verification Efforts For Reused IPs • Expectations: Reused IP should consume less time/resources • “Many unknowns” makes test development daunting • Modification of existing Env is challenging and time consuming • Regressions of un-modified feature tests may not work as-is (due to latency changes etc) • Poor documentation, lack of localized assertion makes debugging tougher • Summary: Verification of reused IPs, if not planned properly, may take as much efforts as new IP

9. Standard Quality Metrics Not Effective • IP sign-off to SoC primarily based on two metrics • Code coverage • Functional coverage • Code coverage consume lot of time for Reused IPs • Deprecated Features • Waivers Modification • Functional coverage monitors are subjective, error prone • DV writes high level coverage monitors for modified/new features • Important cross coverage due to micro-arch. dependencies of IP edits is difficult to plan and implement

10. Need For Improving The Methodology • Quality: Need automated checkers and coverage to find bugs at IP level; otherwise may be found during late stage in SoC verification. • Productivity: Need guidance to test development • Verifying reused IP from scratch is not practical • Coverage associated to IP edits and their dependencies highly important • Other verification techniques do not work • Formal Model checking difficult to setup and runtime too long • Sequential equivalence does not understand functional impact

11. Assertion Synthesis techniques and how to apply them

12. Solution: Assertion Synthesis(Generates assertions and coverage automatically from tests + RTL) Un-modified Modules Legacy Tests Assertion Synthesis New/Modified Modules New Tests IP RTL IP Verification Env “Feature_A == 1’b0” Tests never covers feature_A set “!(stall && wr) Test never asserts stall and wr same cycle “stall |->@cs == IDLE” In all Tests, when stall == 1, next cycle cs is always idle 1 to 2 cycle properties generated by analyzing all tests specific to control modules If signal “stall” is tied low in RTL, property “stall == 1’b0” or “!(stall && wr)” removed automatically Trivial un-reachable properties removed automatically If “feature_A == 1’b0”, redundant low imp properties such as “!(feature_a && feature_b) removed autmatically Redundant properties from others removed automatically L0 Properties: onehot0(sel), cnt < 4’ha L1 properties: !(rd && wr) Properties divided into different levels L0 (single signal), L1 (Two signal) and L2(rest) If design does not support rd & wr in the same cycle “!(rd && w)” is an assertion. Otherwise captures TB limitations; so a coverage Properties capturing design assumption/intent categorized as assert; otherwise as cover

13. Methodology on Reused IPs Before modification in IP1 starts, run all legacy tests and generate properties for unmodified modules M1, M2.. All properties converted as assertions automatically w/o review. Assumption is that for unmodified RTL legacy tests has complete coverage; no coverage holes M1 M2 Phase # 1 SoC SoC M4 M3 Integrated the properties back to IP simulations. So any assertion that fires indicates changes in module M4 that violation assumption of unmodified RTL; hence find bugs IP1 – RTL (Reused) IP2-RTL M5 IP3-RTL M6 Once sufficient tests are written for module M4, generated properties using all tests for M4 and review Phase # 2 Classified assertions are added back into simulation. Coverage holes targeted w/ additional tests. • Module M4 being new/modified • Rest of the modules in IP1 unmodified • IP1 has legacy verification env+tests When IP coverage matures(holes taken care), integrate all IP assertions into SoC simulations Phase # 3 Any assertion firing can indicate design bug or limitation of IP tests (did not cover fully as SoC intends too)

14. Example Property Stats From 2 IPs • 10-15 Lines/Property for control intensive modules • 20-50 Lines/Property for datapath intensive modules • Performance Numbers • PLI overhead during simulation when collecting properties 10-15% • When assertions are added back into simulation, overhead 5-10% • Classification of properties : Avg. 1 min per property

15. Assertions Found and Impact • Property: mrd_aes_data_in_index != mwr_aes_data_in_index Captures micro-architecture design intent: rd and wr index generated in the RTL based on algorithm; Should never be the same. • Property: mrd_aes_data_valid == 1’h1 |-> !isunknown(mrd_aes_data) Captures X propagation: If data driven unknown in SoC, it would localize failure • Property:!(rd_mac_start && rd_mac_data_out_pop) Captures code coverage waivers: Known coverage waiver; but an assertion will catch that it is truly not hit in any IP tests or SoC in any corner scenario • Assertion Synthesis captures design usage assumption and intent. • Integrated Assertion in IP simulation and SoC simulation

16. Coverage Holes Found and impact • Property: interrupts <= 3'h5 Finds holes missed by code coverage: Property indicates that higher values of interrupts possible; but missed. This hole could not be found by toggle coverage as all bits toggled • Property: mutex(cache_enable, roll_2k_page_active) Helps to understand & code micro-arch dependent functional coverage: Cache enable and roll_rk_page_active can be high in the same cycle; but DV missed important dependency • Property: emif_SDataAceept |-> n_mac_seq != MAC_WRT_CMD_STALL Finds BFM limitation: Data accept high and command accept low can happen in stall; but BFM did not support this functionality • Assertion Synthesis discovered individual and cross coverage holes missed • Analysis of coverage holes was efficient due to removal of un-reachable and redundant coverage holes • Guided our future test development based on the coverage holes found

17. Risk mitigation by Reusing Properties in SoC SoC Tests Simulation IP Tests Simulation + Assertion Synthesis Assertion Properties (Reviewed + non-reviewed) Assertion Fires • Assertions generated using IP tests are directly ported/bound to SoC • After review for modified RTL and W/O review for unmodified RTL • When assertions fire in SoC tests, it indicates either of following • Design bug: IP assumption/intent violated in SoC • Misconfiguration: IP is not used correctly in SoC; saves debug time • IP Coverage holes: IP is missing important coverage; important to particular SoC • Findings • When properties from IP used in SoC Context and no assertions fired implies: • SOC did not violate an IP requirement and/or • SOC did not exercises a unique state-space that IP DV did not cover • Performance overhead of running assertions in SoC was comparable as IP (5-10%) Assertion Holds True

18. Backup

19. Conclusions & Next Steps • Assertion synthesis guides test development for reused IPs by finding important functional coverage holes • Integrating assertions generated using Assertion Synthesis back to IP and SoC simulations give extra quality assurance • To-Do • Assertions generated from IP need to be ported to SoC Emulation Env • Find out how effective flow can be for even third party IP

20. Lessons Learned • Some learning curve involved in understanding properties • Aggravated due to less knowledge of RTL • Some properties need couple of minutes to review; improves over time • Review & target properties based on Level (Lx) to reduce property review time and prioritize test development • Deliver properties to SOC after coverage matures at IP; not early • DV should still code high level functional coverage for modified/new features • Co-relates to spec better & complement micro-architecture ones from Assertion Synthesis