1 / 40

0x1A Great Papers in Computer Security

Learn about intrusion prevention, detecting system modifications, and various intrusion detection techniques such as misuse detection and anomaly detection.

slucia
Download Presentation

0x1A Great Papers in Computer Security

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS 380S 0x1A Great Papers inComputer Security Vitaly Shmatikov http://www.cs.utexas.edu/~shmat/courses/cs380s/

  2. After All Else Fails • Intrusion prevention • Find buffer overflows and remove them • Use firewall to filter out malicious network traffic • Intrusion detection is what you do after prevention has failed • Detect attack in progress • Discover telltale system modifications

  3. What Should Be Detected? • Attempted and successful break-ins • Attacks by legitimate users • Illegitimate use of root privileges, unauthorized access to resources and data … • Malware • Trojan horses, rootkits, viruses, worms … • Denial of service attacks

  4. Intrusion Detection Systems (IDS) • Host-based • Monitor activity on a single host • Advantage: better visibility into behavior of OS and individual applications running on the host • Network-based (NIDS) • Often placed on a router, firewall, or network gateway • Monitor traffic, examine packet headers and payloads • Advantage: single NIDS can protect many hosts and look for global patterns

  5. Intrusion Detection Techniques • Misuse detection • Use attack “signatures” (need a model of the attack) • Sequences of system calls, patterns of network traffic, etc. • Must know in advance what attacker will do (how?) • Can only detect known attacks • Anomaly detection • Using a model of normal system behavior, try to detect deviations and abnormalities • Can potentially detect unknown (zero-day) attacks • Which is harder to do?

  6. Misuse Detection (Signature-Based) • Set of rules defining a behavioral signature likely to be associated with attack of a certain type • Example: buffer overflow • A setuid program spawns a shell with certain arguments • A network packet has lots of NOPs in it • Very long argument to a string function • Example: denial of service via SYN flooding • Large number of SYN packets without ACKs coming back …or is this simply a poor network connection? • Attack signatures are usually very specific and may miss variants of known attacks • Why not make signatures more general?

  7. Extracting Misuse Signatures • Use invariant characteristics of known attacks • Bodies of known viruses and worms, RET addresses of memory exploits, port numbers of applications with known vulnerabilities • Hard to handle mutations • Polymorphic viruses: each copy has a different body • Big research challenge: fast, automatic extraction of signatures of new attacks

  8. Anomaly Detection • Define a profile describing “normal” behavior • Works best for “small”, well-defined systems (single program rather than huge multi-user OS) • Profile may be statistical • Build it manually (this is hard) • Use machine learning and data mining techniques • Log system activities for a while, then “train” IDS to recognize normal and abnormal patterns • Risk: attacker trains IDS to accept his activity as normal • Daily low-volume port scan may train IDS to accept port scans • IDS flags deviations from the “normal” profile

  9. Statistical Anomaly Detection • Compute statistics of certain system activities • Report an alert if statistics outside range • Example: IDES (Denning, mid-1980s) • For each user, store daily count of certain activities • For example, fraction of hours spent reading email • Maintain list of counts for several days • Report anomaly if count is outside weighted norm Problem: the most unpredictable user is the most important

  10. Y normal Compute % of traces that have been seen before. Is it above the threshold? N abnormal Raise alarm if a high fraction of system call sequences haven’t been observed before “Self-Immunology” Approach [Forrest] • Normal profile: short sequences of system calls • Use strace on UNIX … open,read,write,mmap,mmap,getrlimit,open,close … remember last K events … open,read,write,mmap read,write,mmap,mmap write,mmap,mmap,getrlimit mmap,mmap,getrlimit,open …

  11. Level of Monitoring • Which types of events to monitor? • OS system calls • Command line • Network data (e.g., from routers and firewalls) • Keystrokes • File and device accesses • Memory accesses • Auditing / monitoring should be scalable

  12. System Call Interposition • Observation: all sensitive system resources are accessed via OS system call interface • Files, sockets, etc. • Idea: monitor all system calls and block those that violate security policy • Inline reference monitors • Language-level: Java runtime environment inspects stack of the function attempting to access a sensitive resource to check whether it is permitted to do so • Common OS-level approach: system call wrapper • Want to do this without modifying OS kernel (why?)

  13. Janus [Berkeley project, 1996]

  14. Policy Design • Designing a good system call policy is not easy • When should a system call be permitted and when should it be denied? • Example: ghostscript • Needs to open X windows • Needs to make X windows calls • But what if ghostscript reads characters you type in another X window?

  15. Problems and Pitfalls [Garfinkel] • Incorrectly mirroring OS state • Overlooking indirect paths to resources • Inter-process sockets, core dumps • Race conditions (TOCTTOU) • Symbolic links, relative paths, shared thread meta-data • Unintended consequences of denying OS calls • Process dropped privileges using setuid but didn’t check value returned by setuid… and monitor denied the call • Bugs in reference monitors and safety checks • What if runtime environment has a buffer overflow?

  16. Incorrectly Mirroring OS State [Garfinkel] Policy: “process can bind TCP sockets on port 80, but cannot bind UDP sockets” 6 = socket(UDP, …) Monitor: “6 is UDP socket” 7 = socket(TCP, …) Monitor: “7 is TCP socket” close(7) dup2(6,7) Monitor’s state now inconsistent with OS bind(7, …) Monitor: “7 is TCP socket, Ok to bind” Oops!

  17. TOCTTOU in Syscall Interposition • User-level program makes a system call • Direct arguments in stack variables or registers • Indirect arguments are passed as pointers • Wrapper enforces some security condition • Arguments are copied into kernel memory and analyzed and/or substituted by the syscall wrapper • What if arguments change right here? • If permitted by the wrapper, the call proceeds • Arguments are copied into kernel memory • Kernel executes the call

  18. R. Watson Exploiting Concurrency Vulnerabilitiesin System Call Wrappers(WOOT 2007)

  19. Exploiting TOCTTOU Conditions [Watson] • Forced wait on disk I/O • Example: rename() • Page out the target path of rename() to disk • Kernel copies in the source path, then waits for target path • Concurrent attack process replaces the source path • Postcondition checker sees the replaced source path • Voluntary thread sleeps • Example: TCP connect() • Kernel copies in the arguments • Thread calling connect() waits for a TCP ACK • Concurrent attack process replaces the arguments

  20. TOCTTOU via a Page Fault [Watson]

  21. TOCTTOU on Sysjail [Watson]

  22. Mitigating TOCTTOU • Make pages with syscall arguments read-only • Tricky implementation issues • Prevents concurrent access to data on the same page • Avoid shared memory between user process, syscall wrapper and the kernel • Argument caches used by both wrapper and kernel • Message passing instead of argument copying • Why does this help? • Atomicity using system transactions • Integrate security checks into the kernel?

  23. D. Wagner, D. DeanIntrusion Detection via Static Analysis(Oakland 2001)

  24. Interposition + Static Analysis Assumption: attack requires making system calls 1. Analyze the program to determine its expected behavior 2. Monitor actual behavior 3. Flag an intrusion if there is a deviation from the expected behavior • System call trace of the application is constrained to be consistent with the source or binary code • Main advantage: a conservative model of expected behavior will have zero false positives

  25. Trivial “Bag-O’Calls” Model • Determine the set S of all system calls that an application can potentially make • Lose all information about relative call order • At runtime, check for each call whether it belongs to this set • Problem: large number of false negatives • Attacker can use any system call from S • Problem: |S| very big for large applications

  26. Callgraph Model [Wagner and Dean] • Build a control-flow graph of the application by static analysis of its source or binary code • Result: non-deterministic finite-state automaton (NFA) over the set of system calls • Each vertex executes at most one system call • Edges are system calls or empty transitions • Implicit transition to special “Wrong” state for all system calls other than the ones in original code; all other states are accepting • System call automaton is conservative • No false positives!

  27. NFA Example [Wagner and Dean] • Monitoring is O(|V|) per system call • Problem: attacker can exploit impossible paths • The model has no information about stack state!

  28. mysetuid myexec setuid log write log log exec Another NFA Example [Giffin] void mysetuid (uid_t uid) { setuid(uid); log(“Set UID”, 7); } void log (char *msg, int len) { write(fd, msg, len); } void myexec (char *src) { log(“Execing”, 7); exec(“/bin/ls”); }

  29. log e e write e e NFA Permits Impossible Paths mysetuid Impossible execution path is permitted by NFA! myexec setuid log log exec

  30. NFA: Modeling Tradeoffs • A good model should be… • Accurate: closely models expected execution • Fast: runtime verification is cheap Inaccurate Accurate Slow Fast NFA

  31. Abstract Stack Model • NFA is not precise, loses stack information • Alternative: model application as a context-free language over the set of system calls • Build a non-deterministic pushdown automaton (PDA) • Each symbol on the PDA stack corresponds to single stack frame in the actual call stack • All valid call sequences accepted by PDA; enter “Wrong” state when an impossible call is made

  32. log e e push B push A write e e pop B pop A PDA Example [Giffin] mysetuid myexec setuid log log exec

  33. Another PDA Example [Wagner and Dean]

  34. PDA: Modeling Tradeoffs • Non-deterministic PDA has high cost • Forward reachability algorithm is cubic in automaton size • Unusable for online checking Inaccurate Accurate PDA Slow Fast NFA

  35. Dyck Model [Giffin et al.] • Idea: make stack updates (i.e., function calls and returns) explicit symbols in the alphabet • Result: stack-deterministic PDA • At each moment, the monitor knows where the monitored application is in its call stack • Only one valid stack configuration at any given time • How does the monitor learn about function calls? • Use binary rewriting to instrument the code to issue special “null” system calls to notify the monitor • Potential high cost of introducing many new system calls • Can’t rely on instrumentation if application is corrupted

  36. log write B A Example of Dyck Model [Giffin] mysetuid myexec setuid B A exec Runtime monitor now “sees” these transitions

  37. CFG Extraction Issues [Giffin] • Function pointers • Every pointer could refer to any function whose address is taken • Signals • Pre- and post-guard extra paths due to signal handlers • setjmp() and longjmp() • At runtime, maintain list of all call stacks possible at a setjmp() • At longjmp() append this list to current state

  38. System Call Processing Complexity [Giffin] n is state count m is transition count

  39. Dyck: Runtime Overheads [Giffin] Execution times in seconds • Many tricks to improve performance • Use static analysis to eliminate unnecessary null system calls • Dynamic “squelching” of null calls

  40. Persistent Interposition Attacks [Parampalli et al.] • Observation: malicious behavior need not involve system call anomalies • Hide malicious code inside a server • Inject via a memory corruption attack • Hook into a normal execution path (how?) • Malicious code communicates with its master by “piggybacking” on normal network I/O • No anomalous system calls • No anomalous arguments to any calls except those that read and write

More Related