Secure Operating Systems

Secure Operating Systems Lesson 7: The Application OS Interface

Where are we? • Time to start to delve deeper into security • Let’s look at how non-OS things talk to the OS…

How we talk… • Typically, a programmer will just call something – perhaps a function – in their library of choice • Example: fopen(); • This allows for portability of higher level languages, adds syntactic sugar, and allows for more portable code

Tracing through fopen… • If we trace through fopen we get to the actual Windows APIs used within the library • In this case, it’s CreateFileA/W()… • From here, we need to go through to the kernel – this requires a transition from Ring 3 to Ring 0 (remind me)

Ring What? • Unlike some processors (notably Motorola) the Intel processors don’t use a register or flag to determine mode • Instead, the privilege is determined by the GDT – the Global Descriptor Table • This table tells the processor if the block of memory currently being executed is in Ring 0 or 3 • Uses the LGDT command…

Obviously • Obviously (and let’s make sure we know why it’s obvious) lower privilege code cannot directly call higher privilege code • So, Intel provided a few mechanisms to do this: Int 2Eh and SYSENTER

Int 2Eh • But Int 2Eh in which mode… Aha! • DOS: DOS Reload Transient. Calls in Interrupt Service Routine, and allows the load and execution of a command to the command interpreter • But DOS runs in REAL mode

Int 2E: Protected Mode • A little more complicated… • Leverages an “Interrupt Gate Descriptor” – instead of a vector to the Interrupt Service handler • Offset in the segment of the ISR • The Entry in the GDT that owns this code • Hence: we can change mode, woohoo!

Security: Must use KERNEL stack • So this gets complicated • We cannot use the user-mode stack for parameter passing… • Why? • Solution: we switch to the kernel’s own stack • Which call are we calling? Defined in EAX • There’s a danger here too, in EBX (pointer to parms)

SYSENTER • Int 2E is slow: has to load two bits of memory, and then figure out where to jump to • The solution? Hardcode where we jump to from SYSENTER, in a MSR (model specific register – in this case, SYSENTER_EIP_MSR)

Okay. But… • The challenge is we’re getting a much higher privilege process to do something for us • Which means what?

Some Random Thoughts • 3 Laws: • Simplicity – we need to make the OS as simple as possible • Completeness – quoting Einstein “everything should be as simple as possible, but no simpler” • Efficiency – it needs to be efficient (slightly different to fast)

Everything is a file! • One possible way of accomplishing this is to make everything look like a file (does this sound familiar?) • This helps, as it provides a simple paradigm that once the programmer “gets” it’s easy to code

Unix v. Windows • exec(name, argp, envp) • Or CreateProcessA(10 parms, plus a struct with 18 (!) parts) • What’s the trade off here? • Remember, more code == more bugs

Don’t Hide Power • This quote is from Lampson • The basic idea is that abstractions are supposed to hide undesirable properties… not hide things we care about • This is slightly dangerous, but a good adage • But what of “public” versus “hidden” system calls? There are advantages to both approaches

Two Things to do • Thursday: You need to read “The Cake is a Lie” by Locasto • Come to class Thursday ready to present a discussion of this paper – our class Thursday will be ALL discussion on this paper, and I’ll be asking each of you to spend a few minutes talking. YOU WILL GET A GRADE FOR THIS. • Come prepared with something to say • Also, before the midterm, read http://www.pearsonhighered.com/educator/product/Modern-Operating-Systems/9780136006633.page Chapter 13

Questions & Comments • What do you want to know?

Secure Operating Systems