Runtime Monitoring of C Programs for Security and Correctness

1. Runtime Monitoringof C Programsfor Security and Correctness Suan Hsi Yong University of Wisconsin – Madison Ph.D. Committee: Susan Horwitz (advisor), Thomas Reps, Charles Fischer, Somesh Jha, James Smith

2. Errors in Software • Software errors are undesirable • may produce incorrect results • may crash system • may corrupt data • may be vulnerable to attack • Software errors are difficult to detect • may be infrequently exercised • may not noticeably alter observable output

3. Memory and Type Safety • Memory safety: each dereference can only access ‘intended target’ • spatial access errors(e.g., out-of-bounds array access) • temporal access errors(e.g., stale pointer dereference) • Type safety: operations can only be applied to values of certain types

4. Memory and Type Safety • Useful for improving quality of software • Tradeoff between efficiency and flexibility • If too general, incurs a high runtime overhead to enforce • If too restrictive, limits expressiveness and utility of language • C language mandates but does not enforce memory and type safety • programmer’s responsibility, error prone

5. Approaches for Finding Errors • Static Approaches • imprecise, not scalable • Dynamic Approaches • incomplete coverage, high runtime overhead • This thesis: Dynamic approach, but use static analysis to improve overhead • for testing/debugging, and for use in deployed software

6. This Thesis Explores… • Runtime checking of memory and type safety in C programs • Three manifestations • Memory-Safety Enforcer (MSE): detects invalid dereferences • Sensitive Location Checker (SLC): detects invalid writes to security-sensitive locations • Runtime Type Checker (RTC): detects bugs manifested as type errors

7. Common Features • Tagged memory • each byte of memory is tagged at runtime with information used to detect errors • Source-level instrumentation • approach is portable, compatible with uninstrumented libraries • Static analysis • identifies and eliminates unnecessary instrumentation

8. Static Analysis classifications Architecture Csourcefile runtimesystem/libraries instru-mentedC sourcefile Instrumenter CCompiler instru-mentedexec-utable

9. Outline • Introduction • Memory-Safety Enforcer (MSE) • Sensitive-Location Checker (SLC) • Runtime Type Checker (RTC) • Related Work • Conclusion

10. Memory Safety p = &ap’s valid target is a *(p+i) valid only if it accesses a • Spatial access error: out of bounds • e.g., if i is negative or is too large • Temporal access error: stale pointer • e.g., if a had been freed prior to *(p+i)

11. Memory Safety Enforcer (MSE) • Debugging Tool • invalid access  programming error • e.g., buffer overrun, stale pointer dereference • Security Tool • most attacks require invalid write to succeed(e.g., stack smashing attacks) • halt execution when violation detected

12. Control Transfer Attacks • Idea: overwrite sensitive location with address of malicious code • Sensitive locations include • return address (stack smashing) • global offset table • function pointers • longjmp buffer • exec call argument • others…

13. Stack Smashing char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’); p buf return address

14. Detecting Invalid Access • Fat Pointer • Record information about what each pointer should point to • Safe-C, CCured, Cyclone • Tagged Memory (our approach) • Record information about which locations may be valid targets of some pointer dereference • also used by Purify

15. Fat Pointers • associate information with pointer: address and size of referent p buf 12 char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’);  buf  return address

16.                                              Tagged Memory • associates information with target rather than pointer p char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’);  buf  return address  = valid target of unsafe dereference

17. Fat Pointer vs. Tagged Memory • Fat Pointers • Guaranteed to catch all spatial errors • Difficult to catch temporal errors efficiently • e.g., CCured uses garbage collection • Tagged Memory • Can detect both spatial and temporal errors efficiently • Guaranteed only to catch invalid accesses to non-user memory • But can improve with static analysis

18. Improving MSE • Which dereferences to check? • if static analysis can guarantee that *p is always valid, then *p need not be instrumented. • classify dereferences into checked/unchecked • Which locations to tag valid at runtime? • if x can only be accessed directly or via unchecked dereference, then x need not be tagged valid • classify locations into tracked/untracked

19.                                                Checked Derefs/Tracked Locs • naively: all dereferences are checked;all user-defined locations are tracked char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’); p buf return address

20.                                                  Checked Derefs/Tracked Locs FN_PTRfp= &foo; char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’); (*fp)(); p buf fp • Static Analysis: identify fewerchecked derefs and tracked locations

21. Checked Dereferences • Writes Only vs. Read/Write • write-only checks catches most attacks; significantly improving overhead • Flow-insensitive analysis: • *p is checked if: • p is assigned a non-pointer value, or • p is the result of pointer arithmetic, or • p may point to stack or freed heap location a[i] *(a+i)

22. dereferences: *pchecked *fp unchecked Example: Checked Dereferences FN_PTR fp = &foo; char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’); (*fp)();

23. Tracked Locations • locations that may be validly accessed via checked dereference • fewer tracked locations means better performance and coverage • less overhead to mark and clear validtag • increase likelihood of catching invalid access • identify with points-to analysis [Das’00] • for each checked dereference *p,all locations in p’s points-to set are tracked

24. locations: points-to graph: dereferences: untracked *punsafe p p fp untracked *fp safe fp tracked buf foo buf Example: Tracked Locations FN_PTR fp = &foo; char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’); (*fp)();

25.                          locations:  points-to graph: dereferences:     untracked   *punsafe p   p fp   untracked   *fp safe fp    tracked  buf foo buf    Example: Tracked Locations FN_PTR fp = &foo; char buf[12]; char *p = &buf[0]; do { *p = getchar(); } while(*p++ != ‘\0’); (*fp)(); p buf   fp

26. Summary of MSE Coverage

27. Evaluation: Runtime Overhead

28. Flow-Sensitive Analyses • Redundant Checks Analysis *(p+i) = ...;*(p+i) = ...;//- don’t instrument • Pointer Range Analysis • track range of possible values for each pointer *p = ...; pa:char[10], [3,7] • if *p is definitely in-bounds, don’t instrument

29. Flow-Sensitive Analyses

30. Flow-Sensitive Analyses

31. Analysis Time Slowdown

32. MSE Static Analysis Summary • Unoptimized MSE • high runtime overhead • only catches invalid access to non-user memory • Flow-insensitive (Extended Points-To Analysis) • low runtime overhead, scalable analysis • Flow-sensitive Analyses • 20% improvement, but analysis not scalable • Write-only faster than read-write checking

33. Comparison with Other Tools(runtime overhead)

34. Summary of MSE • Tool for detecting invalid pointer dereferences that • has low runtime overhead • does not report false positives • is portable, and does not require programmer changes to source code • protects against a wide range of vulnerabilities, including stack smashing and erroneous free

35. Outline • Introduction • Memory-Safety Enforcer (MSE) • Sensitive-Location Checker (SLC) • Runtime Type Checker (RTC) • Related Work • Conclusion

36. Two Approaches to Security • MSE: Try to detect all invalid accesses • including invalid accesses that are not vulnerable to attack • SLC: Detect only invalid accesses to sensitive locations • return address, function pointers,longjmp buffers, exec call arguments • related work: StackGuard – protects only return address on the activation record

37. SLC vs MSE: classification MSE: identify unsafe dereferences, then compute tracked locations dereferences locations w *p x *q y *r z (points-to edges)

38. SLC vs MSE: classification SLC: identify sensitive locations, then compute unchecked dereferences dereferences locations w *p x *q y *r z (points-to edges)

39.      not sensitive      sensitive         unchecked   checked                Example char safe_buf[8]; char vuln_buf[8]; strcpy(vuln_buf, “ls”); gets(safe_buf); system(vuln_buf); safe _buf vuln _buf return address 

40. SLC vs MSE: instrumentation • SLC must set/clear tag of sensitive locations, while MSE must set/clear tag of tracked locations • In general, much fewer sensitive locations that tracked locations, so SLC is faster • SLC must set/clear tag of return address on activation record • may slow down SLC compared to MSE

41. Runtime Overhead: SLC vs MSE Average: SLC=37.7%, MSE=54.1%

42. SLC: The Bad News • In some of the benchmarks (ijpeg, li, perl, gap), over 90% of the dereferences were not checked • i.e., they may point to a sensitive location • due to imprecision of points-to analysis • could be improved with better points-to analysis • Good news: can tell from static analysis whether SLC will be effective for a given program

43. SLC vs MSE • Memory Safety Enforcer (MSE) • detects invalid accesses that may not be vulnerable to attack • may prevent new as-yet-undiscovered methods of attack • Sensitive Location Checker (SLC) • targets specific locations known (a priori) to be vulnerable to attack • better runtime overhead because of limited scope

44. Outline • Introduction • Memory-Safety Enforcer (MSE) • Sensitive-Location Checker (SLC) • Runtime Type Checker (RTC) • Related Work • Conclusion

45. Runtime Type Checking • Idea is to detect runtime type violations • value of one type is used in context of incompatible type • Scalar types only (structs and arrays broken down into components) • Debugging tool, for use during development/testing • Higher overhead acceptable (~20x) • Related tools: Purify, Insure++, Valgrind

46. Error Example 1: Union union U { int u1; int *u2; } u; int *p; u.u1 = 20; // write int into u.u1 p = u.u2; // copy int from u.u2 -- suspicious! *p = 0; // bad pointer deref -- error!

47. Example 2: Bad Pointer Access int *intArray = (int *) malloc(15 * sizeof(int)); int **ptrArray = (int **) malloc(15 * sizeof(int *)); User memory intArray ptrArray

48. i intArray ptrArray i intArray ptrArray padding PURIFY Example 2: Bad Pointer Access int *i, sumEven = 0; for(i = intArray; ...; i += 2)sumEven += *i; User memory ORIGINAL User memory

49. Example 3: Custom Allocator char * myMalloc(size_t size) { static char *myMemory, *current; ... if(first_time){ myMemory = (char *) malloc(BLKSIZE); } ... return &myMemory[current += size]; }

50. i i PURIFY myMemory Example 3: Custom Allocator int *intArray = (int *) myMalloc(10 * sizeof(int)); int **ptrArray = (int **) myMalloc(10 * sizeof(int *)); User memory intArray ptrArray ORIGINAL myMemory User memory