1 / 19

setuid Demystified -- Examining the API of Security Operation in OS using Formal Models

setuid Demystified -- Examining the API of Security Operation in OS using Formal Models. Hao Chen, David Wagner UC Berkeley Drew Dean SRI International. Objective. Understand the semantics of security operation API in OS precisely Applications Using these system calls properly in programs

andrew
Download Presentation

setuid Demystified -- Examining the API of Security Operation in OS using Formal Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. setuid Demystified-- Examining the API of Security Operation in OS using Formal Models Hao Chen, David Wagner UC Berkeley Drew Dean SRI International

  2. Objective • Understand the semantics of security operation API in OS precisely • Applications • Using these system calls properly in programs • Verifying their documentations • Detecting inconsistency in OS kernels • Building security properties and checking them in programs automatically (e.g. by modelchecker)

  3. What is setuid • Access control in Unix is based the User ID model • Each process has 3 user Ids: • Real uid (ruid) • Effective uid (euid) • Saved uid (suid) • Uid-setting system calls • setuid() seteuid() setreuid() setresuid()

  4. The setuid Mystery • Uid-setting system calls are a semantic mess • Counter-intuitive semantics • Subtle differences among different calls • Incompatible semantics of the same call in different Unix systems (e.g.: Linux, Solaris, FreeBSD) • Incomplete, inaccurate, or even wrong documentation • Reason: historical artifacts

  5. Solution: Formal Model • Use a formal model to describe the user ID model • Build an FSA where • The states describe the user IDs of a process • The transitions describe the semantics of the uid-setting system calls

  6. Determine the States of the FSA • Each state is a tuple (ruid, euid, suid) • The range of user ID values determines the number of states • Example: • A process switches between a privileged user ID and an unprivileged ID • 2 user IDs: 0(root), x(non-root) • 8 states

  7. Problem: Difficult to Determine Transitions • Large number of transitions. E.g. • Range of user ID values: {0, x} where x!=0 • Number of states: 8 • Number of transitions per state: • setuid(uid): 2 transitions • seteuid(euid): 2 transitions • setreuid(ruid, euid): 4 transitions • setresuid(ruid, euid, suid): 8 transitions • Total transitions: 8*(2+2+4+8)=128 • A laborious, error-prone process

  8. Determine Transitions Automatically by Simulation • Idea: Exhaustively make all system calls at each state For each state s=(ruid, euid, suid) where ruid, euid, suid {0, uid1 , uid2 , …} For each system call c {setuid(e), seteuid(e), setreuid(r,e), setresuid(r,e,s)} { Make the system call c in the state s Observe the ensuing state s’ Add the transition }

  9. FSA for setuid() in FreeBSD

  10. FSA for setuid() in Linux

  11. FSA for setreuid() in Linux

  12. FSA for setresuid() in Linux

  13. Benefits • Correctness: the FSA reflects what programs experience • Efficiency: the automatic method is portable to • Different Unix systems • Different kernel versions

  14. Application: Understanding the semantics of the system calls • Find subtle semantic differences • Among different uid-setting system calls • Among the same system call on different Unix systems • Find surprising, counter-intuitive semantics

  15. Application: Verifying Man Pages • Incompete man page • Man page for setuid() in Linux fails to mention capabilities which affect how setuid() behaves • Wrong man pages • FreeBSD 4.4:Unprivileged users may change the ruid to the euid and vice versa • Redhat Linux 7.2:The setgid function checks if the egid of the caller and if it is the superuser, …

  16. Application: Detecting Inconsistency in OS Kernel • Linux has fsuid • Used for filesystem permission checking • Normally follows euid • Invariant in Linux 2.4.18 (kernel/sys.c) • fsuid is 0 only if at least one of ruid, euid, suid is 0 • Rationale • ensuring that an fsuid-unware cross-platform application can automatically drop root privilege in fsuid by dropping it in ruid, euid, suid

  17. Application: Detecting Inconsistency in OS Kernel (cont) • A bug breaks the invariant: • The invariant is satisfied in setuid(), seteuid(), setreuid() • But it is broken in setresuid() • We found the bug using the simulator • The bug has been confirmed by Linus and Alan and will be fixed using our patch.

  18. Application: Checking Proper Usage of Syscalls in Programs • Modelchecking security properties in programs • Model a program as a PDA • Intersect the PDA (program) with the FSA of uid-setting system calls to get a new PDA • Check reachability of risky states in the new PDA • Can answer questions like: • Can a uid-setting system call fail in this program? • Can this program fail to drop privilege? • Which part of this program run with privilege? • Result: Found known bugs in sendmail 8.10.1 and 8.12.0

  19. Conclusion • Formal models are useful in • Understanding the APIs of security operations • Verifying their documentations • Detecting inconsistency in OS kernels • Checking proper usage of security-relevant APIs in programs

More Related