1 / 42

Enhancing Security with TxBox: Building Efficient Sandboxes through System Transactions

This paper presents TxBox, an innovative solution for securing untrusted applications through effective sandboxing. By leveraging system transactions, TxBox enforces strong security policies and guarantees automatic recovery from malicious actions. It improves upon traditional sandboxing methods by achieving isolation and preventing time-of-check-to-time-of-use (TOCTTOU) vulnerabilities. The design includes components for access control and transaction management, ensuring dynamic security measures without drastically affecting performance. The paper also evaluates TxBox's capabilities while addressing its limitations.

reece
Download Presentation

Enhancing Security with TxBox: Building Efficient Sandboxes through System Transactions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Suman Jana The University of Texas at Austin Donald E. Porter Stony Brook University VitalyShmatikov The University of Texas at Austin IEEE Security & Privacy 2011 TxBox: Building Secure, Efficient Sandboxes with System Transactions

  2. Outline • Introduction • Building a Better Sandbox • Related Work • System Transactions • Design and Implementation • Evaluation • Limitations • Conclusions

  3. Introduction • ACID: atomicity, consistency, isolation, and durability [wiki] • TxBox: • Speculative execution of untrusted application • Uncircumventable enforcement of system-call policies • Automatic recovery from the effects of malicious execution

  4. Introduction • TxBox components: • Grouping • Rollback • Access summary • Access check • Based on TxOS[link] • Linux with support for system transactions • Prevent race conditions

  5. Building a Better Sandbox • TOCTTOU example Public file 1. Link to public file Softlink Private file 2. Change to private file System-call monitor must know all possible system-call sequences that lead to a violation and the monitor must check every call ! Difficult and Slow !

  6. Building a Better Sandbox • TxBox is immune to TOCTTOU attacks • Sandboxed processes run inside separate transactions • Whitelist / blacklist access-control policies

  7. Building a Better Sandbox • Drawback of conventional sandbox • Violation can be detected only after multiple system calls, the damage may already have been done • TxBox: • Transaction is aborted, and the system automatically reverts

  8. Building a Better Sandbox • Parallelism (Multi-core processors) • Conventional sandbox or anti-virus: • Sandboxed program makes a system call, its execution must be paused • Split-personality malware • TxBox: • Sandboxed program run with close-to-native performance while performing checks in parallel • Only a single copy is executed

  9. Related Work • Speck, 2008 • Multi-core processor • Concurrently perform checks on an instrumented copy on another core • Cannot detect “split-personality” malware • Use Pin [link] • Pin can be detected by malicious program • Suffer TOCTTOU attacks

  10. Related Work

  11. Related Work • Software Transactional Memory (STM) [wiki] • Lock-based synchronization • STM-based systems • Enforce application-specific security policies • Cannot span system calls

  12. Related Work • Capsicum, 2010 • Capability-based sandboxing system • Help benign application to increase their trustworthiness • Native Client, 2009 • Require the code to be recompiled to a restricted subset of the x86 ISA and also confines it using segmentation

  13. System Transactions • ACID properties for sequences of updates to system resources • If the transaction aborted before it finish, all intermediate updates are rolled back • All accesses to the system within a transaction are kept isolated and invisible to the rest of the system until the transaction commits

  14. System Transactions • TxOS: • sys_xbegin – start transaction • sys_xend – end transaction • Maintain a transaction workset • Kernel objects (inodes, etc.) • Each entry in workset contains a pointer to the stable object, a pointer to the shadow copy • information: • read-only / read-write • Methods: commit / abort / lock / unlock / release

  15. Design and Implementation

  16. Design and Implementation • TxBox policy • BLACKLIST • Forbidden behavior • WHITELIST • Required behavior • DENY • Lists of system calls should be denied

  17. Design and Implementation • BLACKLIST and WHITELIST primitives • Regular expressions • System-call names and arguments, or objects • Objects • (I) inodes • Inode number • Mode: read or write • (S) sockets • Type: INET or UNIX • Attributes: IP address (for INET socket) or names (for UNIX datagram sockets)

  18. Design and Implementation • Example • (e)* means “match expression e any number of times” • Any deviation from a model-based whitelist policy means that the sandboxed program is no longer executing the original code (code injection attacks)

  19. Design and Implementation

  20. Design and Implementation • DENY primitives • DENY primitives is violated • Block calls and continue execution • Run a sandboxed program without letting it perform certain operations

  21. Design and Implementation • Policy enforcement • Character device driver • Send policy and pid to monitor • Only communicate with root

  22. Design and Implementation • System-call interposer • TxBox call wrappers • Symbolic link issue • rename calls • Enforcer • Force sandboxed process to run in transaction mode

  23. Design and Implementation • Policy decision engine • Invoked when sandboxed process attempts to perform a critical call

  24. Design and Implementation • Implementation • Based on Dazuko[wiki], by Avira • File Access Control • Can also be based on LSM • xbegin -> beginTransaction kernel function • beginTransaction checks if the current call is xbegin • Forced_transaction flag

  25. Design and Implementation • Handling external I/O by sandboxed process • Bidirectional communication cannot be buffered until the end of a transaction • Conventional system-call monitors • Call-by-call • TxBox • If DENY primitive exists • return error • Check WHITELIST and BLACKLIST • Inspects the trace of the process • Terminate and roll back

  26. Evaluation • Performance Test • TxBox with Linux 2.6.22.6 • Linux with Dazuko module • Quad-core Intel X5355 2.66GHz • 4GB RAM

  27. Evaluation • Micro-benchmarks • Policy • BLACKLIST WREGEX *I:1234* Worst system call for TxBox

  28. Evaluation • Application benchmarks

  29. Evaluation • Application benchmarks • PostMark: a file system benchmark • Due to the fact that the transaction commit groups all writes and presents them to the I/O scheduler all at once, thus improving disk arm scheduling

  30. Evaluation • Scalability • Opens 100 existing files • Intel Core Due 2.00 GHz with 2GB RAM

  31. Evaluation Original TxBox fd = open(“foo”, ..) read(fd, ..) sockfd = socket( .. ) sendto = (sockfd, .. ) close(sockfd) close(fd) … TX BEGIN fd = open(“foo”, ..) read(fd, ..) sockfd = socket( .. ) CHECK TX POLICY VIOLATION NONTRANSACTIONAL sendto(sockfd, ..) close(sockfd) close(fd) … CHECK TX POLICY VIOLATION TX END

  32. Evaluation • Non-transactional system call • sendto / reccvfrom / connect / send / recv /ioctl / read (from a socket) / write (to a socket) • IO-intensive applications • wget • System time increases by 30%-40%

  33. Evaluation • Overhead for trusted applications • A single non-transactional system call • 44% overhead • Compile a non-transactional Linux kernel • Less than 2% overhead

  34. Evaluation • Creating files in a protected directory • make install for vim • BLACKLIST WREGEX *I:164564* • 164564 -> /usr/local/bin/ • Correctly roll back all effects on the system

  35. Evaluation • Malicious MIME handler in a browser • Tarhandler with lynx browser for MIME type ‘application/x-tar’ • Tarhandler reads file from /home/secret and writes into /tmp/foo • BLACKLIST WREGEX *I:183145* • 183145 -> /home/secret

  36. Evaluation • Multimedia converter • ffmpeg: a popular codec • Create output files in the /home/user1/ • BLACKLIST WREGEX *I:181064* • 181064 -> /home/user1

  37. Evaluation • Javascript engine • Google V8 benchmark • SpiderMonkeyJavascript engine • Less than 5%

  38. Evaluation • On-access anti-virus scanning and parallelization of security checks • ClamAV • Small file -> the overhead of transactional mechanism dominates

  39. Evaluation • With 4 threads, TxBox is 2.3 times better than Linux+ Dazuko

  40. Limitations • Kernel-based security monitor is vulnerable to kernel atttacks • Transactional semantics may change the behavior of sandboxed processes • Transactional conflict • Auto-retry • Transactional state is not shared • Some anti-virus may need to observe the execution of the program

  41. Limitations • Colluding malware may evade security policies • No sandboxing mechanism can reliably prevent this • Processes tha generate very large worksets are killed • DoS Attack

  42. Conclusions • Use system transactions • Cannot be circumvented by TOCTTOU • Performance improvement (Multi-core)

More Related