Enhancing Post-Silicon Debugging with BackSpace: A Formal Analysis Approach

1. BackSpace: Formal Analysis for Post-Silicon Debug Flavio M. de Paula* Marcel Gort *, Alan J. Hu *, Steve Wilton *, Jin Yang+ * University of British Columbia + Intel Corporation

2. Outline • Motivation • Current Practices • BackSpace – The Intuition • Proof-of-Concept Experimental Results • (Recent Experiments) • Conclusions and Future Work

3. Motivation • Chip is back from fab! • Screened out chips w/ manufacturing defects

4. Motivation • Chip is back from fab! • Screened out chips w/ manufacturing defects • A bring-up procedure follows: • Run diagnostics w/o problems, everything looks fine!

5. Motivation • Chip is back from fab! • Screened out chips w/ manufacturing defects • A bring-up procedure follows: • Run diagnostics w/o problems, everything looks fine! • But, the system becomes irresponsive while running the real application…

6. Motivation • Chip is back from fab! • Screened out chips w/ manufacturing defects • A bring-up procedure follows: • Run diagnostics w/o problems, everything looks fine! • But, the system becomes irresponsive while running the real application… • Every single chip fails in the same way (1M DPM: Func. bugs)

7. Motivation • Chip is back from fab! • Screened out chips w/ manufacturing defects • A bring-up procedure follows: • Run diagnostics w/o problems, everything looks fine! • But, the system becomes irresponsive while running the real application… • Every single chip fails in the same way (1M DPM: Func. bugs) • What do we do now?

8. Current Practices Inputs Scan-out buggy state

9. Current Practices Inputs Scan-out buggy state But, cause is not obvious!!!

10. Current Practices Guess when to stop and single step Inputs ? ? ? Scan-out

11. Current Practices Guess when to stop and single step Inputs ? Problems: Single-stepping interference; Non-determinism; Too early/late to stop? Non-buggy path

12. Current Practices • Leveraging additional debugging support: • Trace buffer of the internal state

13. Current Practices • Leveraging additional debugging support: • Trace buffer of the internal state • Provides only a narrow view of the design, e.g., program counter, address/data fetches

14. Current Practices • Leveraging additional debugging support: • Trace buffer of the internal state • Provides only a narrow view of the design, e.g., program counter, address/data fetches • Record all I/O and replay • Solves the non-determinism problem, but… • Requires highly specialized bring-up systems

15. Current Practices • Leveraging additional debugging support: • Trace buffer of the internal state • Provides only a narrow view of the design, e.g., program counter, address/data fetches • Record all I/O and replay • Solves the non-determinism problem, but… • Requires highly specialized bring-up systems • Just having additional hardware • does NOT solve the problem

16. A Better Solution: BackSpace • Goal: • Avoid guess work • Avoid interfering with the system • Run at speed • Portable debug support • Compute an accurate trace to the bug

17. A Better Solution: BackSpace • Requires: • Hardware: • Existing test infrastructure and scan-chains; • Breakpoint circuit; • Good signature scheme; • Software: • Efficient SAT solver; • BackSpace Manager

18. A Better Solution: BackSpace Inputs 1. Run at-speed until hit the buggy state Non-buggy path

19. A Better Solution: BackSpace Inputs 1. Run at-speed until hit the buggy state Non-buggy path

20. A Better Solution: BackSpace Inputs 1. Run at-speed until hit the buggy state Non-buggy path

21. A Better Solution: BackSpace Inputs 1. Run at-speed until hit the buggy state Non-buggy path

22. A Better Solution: BackSpace Inputs 2. Scan-out buggy state and history of signatures

23. A Better Solution: BackSpace Inputs Off-Chip Formal Analysis Formal Engine

24. A Better Solution: BackSpace Inputs • Off-Chip Formal Analysis • - Compute Pre-image Formal Engine

25. A Better Solution: BackSpace Inputs Pick candidate state and load breakpoint circuit Formal Engine

26. A Better Solution: BackSpace Inputs Run until hits the breakpoint Formal Engine

27. A Better Solution: BackSpace Inputs Pick another state Formal Engine

28. A Better Solution: BackSpace Inputs Run until hits the breakpoint Formal Engine

29. A Better Solution: BackSpace Inputs Run until hits the breakpoint Formal Engine

30. A Better Solution: BackSpace Inputs Computed trace of length 2

31. A Better Solution: BackSpace Inputs Iterate Formal Engine

32. A Better Solution: BackSpace Inputs BackSpace trace

33. Outline • Motivation • Current Practices • BackSpace – The Intuition • Proof-of-Concept Experimental Results • Recent Experiments • Future Work

34. Proof-of-Concept Experimental Results Chip on Silicon BackSpace Manager SAT Solver

35. Proof-of-Concept Experimental Results Logic Simulator BackSpace Manager SAT Solver

36. Proof-of-Concept Experimental Results • Setup: • OpenCores’ designs: • 68HC05: 109 latches • oc8051 : 702 latches • Run real applications

37. Proof-of-Concept Experimental Results • Can we find a signature that reduces the size of the pre-image? • Experiment: • Select 10 arbitrary ‘crash’ states on 68HC05; • Try different signatures

38. Signature Size vs.States in Pre-Image

39. Proof-of-Concept Experimental Results • How far can we go back? • Experiment: • Select arbitrary ‘crash’ states: • 10 for each 68HC05 and oc8051; • Set limit to 500 cycles of backspace; • Set limit on size of pre-image to 300 states; • Compare the best two types of signature; • Hand-picked • Universal Hashing of entire state

40. 68HC05 w/ 38-Bit Manual Signature

41. 68HC05 w/ 38-Bit Manual Signature

42. 68HC05 w/ 38-Bit Universal Hashing

43. 8051 w/ 281-Bit Manual Signature

44. 8051 w/ 281-Bit Universal Hashing

45. Proof-of-Concept Experimental Results • Results • Signature: Universal Hashing • Small size of pre-images • All 20 cases successfullyBackSpaced to limit

46. Proof-of-Concept Experimental Results • Breakpoint Circuitry • 40-50% area overhead. • Signature Computation • Universal Hashing naïve implementation results in 150% area overhead.

47. Recent Experiments • OpenRisc 1200: • 32-bit RISC processor; • Harvard micro-architecture; • 5-stage integer pipeline; • Virtual memory support; • Total of 3k+ latches • BackSpace implemented in HW/SW • AMIRIX AP1000 FPGA board (provided by CMC) • Board mimics bring-up systems • Host-PC: off-chip formal analysis

48. Recent Experiments • BackSpacing OpenRisc 1200: • Running simple software application • Backspaced for hundreds of cycles • Demonstrated robustness in the presence of nondeterminism

49. Conclusions & Future Work • Introduced BackSpace: a new paradigm for post-silicon debug • Demonstrated it works • Main challenges: • Find hardware-friendly & SAT-friendly signatures • Minimize breakpoint circuitry overhead