Detection of Control Flow Errors Survey of Hardware and Software Techniques

1. Detection of Control Flow ErrorsSurvey of Hardware and Software Techniques Greg Bronevetsky

2. Scope of Lecture • No general techniques for detecting random faults • Approach: focus on faults that show up as control flow errors • Control flow well defined problem • Legal paths through program reasonably approximated via simple models • If control flow errors made detectable, may detect majority of faults • Will cover several papers in this field

3. Classes of Solutions • Watchdog Processors • Separate processor watches instruction stream • Blows whistle on erroneous control flow • Pro: • Small hit on application performance • Con: • Needs new hardware • Extra processor • Though might be doable via Thread-Level Speculation • Possibly, more bits in RAM

4. Classes of Solutions • Software-only solutions • Extra instructions inserted into program • Check that jumps are correct • Pro: • Works on existing hardware • Con: • Potential for significant overhead • Inserted instructions can also fail

5. Outline • Watchdog Processor • General overview • Concurrent Process Monitoring with No Reference Signatures (1994) • SEIS (1995) • Software • BSSC/ECI (1992) • CCA/ECCA (1999) • PECOS (2003) • Control-flow Checking for Security

6. Watchdog Processors • Watchdog processor acts as coprocessor • Can see all instructions loaded by CPU • If watchdog above cache, CPU involvement required • Signals error when sees invalid control flow RAM CPU Watchdog

7. Watchdog sees all instructions, recomputes SB At block end, watchdog compares computed signature with embedded signatureError raised if not equal Block Signatures • Each branch-free block gets a signature • Signature computed from block’s instructions • Signatures vary with scheme • Ex: sum of the instructions in block Compile-time Runtime ld r3[0xff] r1add r1, r5, r2……mult r2, r2, r3SBjmp label Block B, with signature SB Insert SB before end of block

8. Illegal jump Illegal Jump • If illegal jump happens • watchdog will have partially computed signature from prior block • jump to wrong block will lead watchdog to complete signature with wrong instructions • when watchdog sees next signature in code, will not match computed signature A B Legal control flows C D Computed SBEmbedded SB

9. Error Coverage • What if partial signature at point 1 equals partial signature at point 2? • Then at end of block D Computed SB = Embedded SB • Illegal jump not detected • Problem if many blocks start out the same A B Point 1 C D Point 2 Computed SB=Embedded SB

10. Error Detection Latency • Latency - number of clock cycles before error is detected • Blocks are typically small • 5-10 instructions • Thus, time to fault detection fairly low • Requires many signatures to be embedded • Memory overhead • Processor stalls while signature fetched • Some schemes use one signature for >1 blocks • Worse detection latency

11. Limitations • Approaches typically require knowledge of program control flow • Many cannot deal with programs that generate jump destinations at runtime • Only deal with errors manifested as control-flow errors • True for many errors • If error is originally a data error, long detection latency

12. Outline • Watchdog Processor • General overview • Concurrent Process Monitoring with No Reference Signatures (1994) • SEIS (1995) • Software • BSSC/ECI (1992) • CCA/ECCA (1999) • PECOS (2003) • Control-flow Checking for Security

13. Upadbyaya & Ramamurthy • Generalization of watchdog signatures approach • Requires no signatures to be embedded into program code • S.J. Upadbyaya and B. Ramamurthy, “Concurrent Process Monitoring with No Reference Signatures”, IEEE Transactions on Computers, 1994.

14. Outline of Scheme • SIG: signature accumulated by watchdog • INST: current instruction • F: some signature generating function • Ex: sum, XOR, etc. • Initialize SIG to some value (such as all 0’s) • For every instruction in block,SIG = F(SIG, INST)

15. Outline of Scheme • When SIG satisfies a certain property, stop and mark instruction as check-point • Can use parity bit to mark • At this point watchdog will notice check-point mark and will verify that its SIG also satisfies property • Paper favorite property: m-out-of-n codeword • m 1’s out of n bits • Easy to reason about error coverage

16. SIG = 0SIG = F(SIG, INST) SIG = F(SIG, INST) … SIG = F(SIG, INST) mult r2, r2, r6 m-out-of-n codeword! mark instruction as check-point Verify that SIG is now an m-out-of-n codeword SIG = F(SIG, INST) SIG = F(SIG, INST) SIG = F(SIG, INST) SIG = F(SIG, INST) . . . . . . An Single-Block Example Compiler Watchdog SIG = 0SIG = F(SIG, INST) SIG = F(SIG, INST) … SIG = F(SIG, INST) ld r3[0xff] r1add r1, r5, r2……… mult r2, r2, r6 … … … st r6, r3[0xb2]shl r1, r3[0xb2], 1

17. Branch Example • Two blocks with different final signatures • Join at block C • SIG will become legal codeword at different points depending on whether we came from block A or B • Must merge SIG1 and SIG2 into one A B SIG2 SIG1 C

18. Branch Example • Idea: make sure that when we get to C, SIG is valid codeword • But must still ensure that we correctly got to C A B SIG2 SIG1 C

19. TARGET • TARGET marked as a check-point At Compile-time • Let TARGET = first instruction in block C • Before end of blocks A&BSIG=F(SIG1, TARGET)EXIT_BYTE=value s.t. F(SIG, EXIT_BYTE) is avalid codeword … code … jmp label EXIT_BYTE … code … jmp label EXIT_BYTE SIG2= …SIG=F(SIG2, TARGET)Insert EXIT_BYTE SIG1= …SIG=F(SIG1, TARGET)Insert EXIT_BYTE TARGET

20. At Runtime • At branch • Load Target instruction • Compute SIG=F(SIG2, TARGET) • Compute SIG=F(SIG, EXIT_BYTE) • SIG should now be a valid codeword • TARGET is marked as check-point • Watchdog verifies that SIG is valid codeword … code … jmp label EXIT_BYTE SIG2= …SIG=F(SIG2, TARGET)SIG=F(SIG2, EXIT_BYTE) TARGET

21. Summary of Scheme • Scheme checks whether accumulated signature obeys given property • vs. ensuring that it matches a recorded signature • Thus, don’t store signatures for each block • Savings in memory and fewer noops • Need to store EXIT_BYTE • Paper suggests several mechanisms that don’t require extra noop • Can be generalized to any signature function and any property • Theoretical analysis given for m-out-of-n property

22. Performance Results • They give some indications of what detection latency would be • No information on • error coverage • performance overhead

23. Outline • Watchdog Processor • General overview • Concurrent Process Monitoring with No Reference Signatures (1994) • SEIS (1995) • Software • BSSC/ECI (1992) • CCA/ECCA (1999) • PECOS (2003) • Control-flow Checking for Security

24. Traditional Watchdog Techniques • Monitor every single instruction • Not possible in machines with caches (unless watchdog is part of processor) • External watchdog can’t see instructions fetches from Instruction cache (I-cache) • With low I-cache miss rate, most control flow invisible to watchdog • In multi-tasking systems task switches will confuse watchdog

25. Assigned Signatures to the Rescue • An Assigned Signature method • At various points in code, send unique signature to watchdog • Watchdog knows legal order of signatures • Pro: • Communication with watchdog explicit, will bypass cache • Con: • Since actual code invisible to watchdog, additional required watchdog intelligence can be expensive

26. SEIS • SEIS = Signature Encoded Instruction Stream • An Assigned Signature method • Aims to offer efficiency • Targets multiprocessing systems • Documented in:I. Majzik, A. Pataricza, W. Hohl, J. Hönig and V. Sieh, “A High Speed Watchdog Processor for Multitasking Systems”, 1994.

27. SEIS Basic Idea • Create control-flow graph of program • Nodes are jump-free blocks of code • Edges are possible control flows • Assign unique label to each node s.t. • Labels have constant length • Labels of adjacent nodes have easy to detect relationships • ex: label of each node contains IDs of valid successors

28. Label assignment • Find Eulerian circuit through control-flow graph • Contains all edges exactly once • Touches nodes multiple times • Guaranteed to exist if in all nodes in-degree = out-degree • Add phantom edges to graph to assure this

29. Label Assignment • Walk circuit, assigning numbers to nodes in increasing order • When walking over phantom edge, skip a number • Node’s label = concatenation of all assigned numbers (sublabels) • # sublabels per node = in-degree = out-degree • If two nodes neighbors in control-flow graph, node A followed by node B • Node A label <a1, … ak>, node B label <b1, … bl> • Then  ai, bj s.t. bj = ai + 1

30. Example

31. Example Initial Graph: in-degree  out-degree

32. Example Initial Graph: in-degree  out-degreeAdd Phantom Edges

33. Example Find Eulerian Circuit

34. ,12 7 ,10 ,8 11 Example 0 ,6 1 ,4 5 2 Assign Sublabels

35. Example 0,6,12 1,4 7,10 5 2,8 11 Note: If node A succeeds node B, they’re off by 1 along some sublabel

36. At runtime • In each block insert instruction to send block label to watchdog • Watchdog watches incoming labels, stores last label seen • If new label is neighbor of stored label along some sub-label, all OK • i.e. new label has some sub-label that is +1 relative to some sublabel of stored label • If new label NOT neighbor of stored label, control-flow error!

37. Unbounded Signatures • Method leads to efficient watchdog checks • Only problem: • Number of successor/predecessor nodes unbounded • Thus, label size unbounded • Must cut out some labels without sacrificing correctness

38. Removing Sub-labels . . . . . . . • k predecessors  k sublabels • Pick one predecessor sublabel: ai …,a1,… …,ak,… a1+1,…,ak+1

39. Removing Sub-labels . . . . . . . • k predecessors  k sublabels • Pick one predecessor sublabel: ai • Have all predecessors use ai • Node now needs one sublabel, not k …,ai,… …,ai,… ai+1

40. Sub-label Removal Problem • Problem: ajai but pre-predecessor still has aj-1 • Watchdog can’t tell if control-flow legal …,aj-1,… …,aj-1,… legal control flow? …,aj,… …,ai,… ai+1 a1+1,…,ak+1

41. Sub-label Removal Problem • Paper rather cryptic on this issue • Possible solutions: • More sublabels: • In predecessors replace single sublabel aj with sublabels <ai, aj> • In node itself, replace, <a1, … , ak> with just ai • Unclear if process will converge to specific label size bound • Propagate replacement: • If predecessor node gets aj replaced with ai, in pre-predecessor replace aj-1 with ai-1 and so on recursively • Many sublabels in program will be equal: reduced error detection coverage

42. Constant Label Size • Want label size constant, not just bounded • If label too large, use above procedure • If label too small, repeat a sublabel multiple times • Has no effect on watchdog’s accuracy • Paper’s target label size = 3 sublabels

43. Checking Procedure Calls • Procedures have enhanced checking • At procedure start and end insert special labels with unique procedure IDs • Watchdog maintains procedure stack • Mirrors call stack • Ensures correct procedure returns • Each block’s label contains ID of parent procedure • Allows for better&earlier detection of erroneous inter-procedure jumps

44. Multi-Tasking Support • Watchdog maintains several sets of last-label registers and procedure stacks • One per task • Switch between them during task switches • If more tasks than watchdog memory, caching scheme can probably be used • Program labels also contain task ID • Set at load time • Extra check for task scheduler and OS protection mechanisms

45. Experimental Evaluation • Program size overhead upto 30% • Execution time overhead can exceed 100% • Particularly for programs with tight loops • one signature send for only few instructions in loop body • Fault detection coverage: 50% • Authors implement several techniques to reduce number of checks • Overhead brought down to 10%-15% • Fault detection coverage drops to 20%

46. Outline • Watchdog Processor • General overview • Concurrent Process Monitoring with No Reference Signatures (1994) • SEIS (1995) • Software • BSSC/ECI (1992) • CCA/ECCA (1999) • PECOS (2003) • Control-flow Checking for Security

47. Checking Control Flow in Software • Modify program to contain instructions to check control flow • Requires no additional hardware • New instructions also vulnerable to faults • Notable performance penalty • blocks are small: many additional instructions

48. BSSC • BSSC = Block Signature Self Checking • G. Miremadi, J. Karlsson, U. Gunneflo, J. Torin, “Two software techniques for on-line error detection”, FTCS 1992.

49. Insert entry_proc(SB) before block end call entry_procSB entry_proc() saves SB in static varchanges return ptr to next instruction exit_proc() compares given SB to stored SB. If  : YellsIf = : changes return ptr to next instruction Insert exit_proc(SB) after block end call exit_procSB The Idea • Assign signatures to block as you wish • Upon block entry, record signature • Upon block exit, check signature Compile-time Runtime ld r3[0xff] r1add r1, r5, r2……mult r2, r2, r3 jmp label Block B, with signature SB

50. Optimization • If SB= block’s starting address, can eliminate one signature Compile-time • Address of instruction following entry_proc() call automatically pushed on stack before call • Code planted at specific virtual address Insert entry_proc() before block end call entry_proc ld r3[0xff] r1add r1, r5, r2……mult r2, r2, r3 jmp label Block B, with signature SB Insert exit_proc(SB) after block end call exit_procSB