Understand stack buffer overflow attack and defense controls against program threats
1 / 15

Understand stack Buffer overflow attack and defense Controls against program threats - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Understand stack Buffer overflow attack and defense Controls against program threats. Computer Emergency Response Team (CERT). At the Carnegie Mellon University http://www.cert.org Formed after Morris Worm (after 1988)

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Understand stack Buffer overflow attack and defense Controls against program threats

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Understand stack

Buffer overflow attack and defense

Controls against program threats

Computer Emergency Response Team (CERT)

  • At the Carnegie Mellon University http://www.cert.org

  • Formed after Morris Worm (after 1988)

    • A long report on Morris Worm by Eugene Spafford of Purdue CERIAS http://portal.acm.org/citation.cfm?id=66093.66095

YearTotal vulnerabilities

2007 7,236













Program control – specification & verification

typical approach: specify a should-do list

security needs: a shouldn’t-do list

No silver bullet to achieve security effortlessly

1. exhaustive testing of all program states is infeasible

2. software engineering techniques evolve rapidly

Some definitions and Types of flaws

  • Program security flaw: unexpected behavior

  • Error: human mistake

  • Fault: an incorrect step, command, process, data definition in a program

  • Failure: departure from system’s required behavior

A taxonomy of program flaws

Landwehr [LAN93]

Inadvertent flaws

Intentional flaws

Validation, domain, serialization/aliasing,


Boundary condition violation

Logic errors



Java checks array bound – no buffer overflow

Buffer overflows

  • A buffer (or array or string) is a space in which data can be held

    • In memory, finite capacity

    • Checking bound takes time/space

char sample[10];




Replace code in system space with kernel privilege

Figure 3-1  Places Where a Buffer Can Overflow.

Process memory region






Text region: code, read-only data

Stack is for procedure call – jump and return

Use stack for:

dynamically allocate local vars

pass parameters to functions

return values from functions

Static variables. If data region expands or stack space runs out, new memory is added between data and stack segments

Stack: an abstract data type.

Last in, first out (LIFO)

PUSH: add an element at top of stack

POP: remove element at top of stack

Aleph One. Smashing the stack for fun and profit. 96.


How a process uses its stack: Call stack, stack pointer

  • Stack buffer overflow occurs when information is written into the memory allocated to a variable on a stack, but the size of this information exceeds what was allocated at compile time.

    • HEAP buffer overflow – used in many drive-by downloads

  • Run-time stack, call stack, control stack, execution stack (all the same):

    • What a process uses to keep track of the sequence of subroutines called and local variables encountered

  • Stack frame: the consecutive stack space for each calling function that has not yet finished execution

  • Top stack frame: for function that just got called and is being executed

  • Stack pointer: memory location of the top of the stack

    • Stored in a register

Avi Kak Computer Security lecture note

Another example


void function(int a, int b, int c) {

char buffer1[5];

char buffer2[10];


void main() {



sfp: saved frame pointer

Bottom of memory

Top of memory

buffer2 buffer1 sfp ret a b c

<------ [ ] [ ] [ ] [ ] [ ] [ ] [ ]

Bottom of stack

Top of stack

An example of stack and stack frame

// ex.c

int main() {

int x = foo( 10 );

printf( "the value of x = %d\n", x );

return 0;


int foo( int i ) {

int ii = i + i;

int iii = bar( ii );

int iiii = iii;

return iiii;


int bar( int j ) {

int jj = j + j;

return jj;


stack_ptr--> jj

return-address to caller




return-address to caller





stack frame for bar

stack frame for foo

stack frame for main

Partial assembly code of ex.c in ex.S

.type foo, @function


pushl %ebp

movl %esp, %ebp

subl $4, %esp

movl 8(%ebp), %eax

addl %eax, %eax

movl %eax, (%esp)

call bar



.size foo, .-foo

.globl bar

.type bar, @function


pushl %ebp

movl %esp, %ebp

movl 8(%ebp), %eax

addl %eax, %eax

popl %ebp


.size bar, .-bar

.globl foo


Compile with gcc -S –O ex.c –o ex.S

Inspecting the call stack – assembly code of ex.c


pushl %ebppush value stored in register ebp onto stack

movl %esp, %ebpmove value in register esp to ebp

subl $4, %espsubstract 4 from value in esp (stack grows)

movl 8(%ebp), %eaxmove i into an accumulator

addl %eax, %eaxi+i

movl %eax, (%esp)move accumulator content into stack location pointed to by the content of esp register – so that local var ii becomes the argument to bar

call barcall bar



To inspect the assembly code, run gcc -S -O ex.c -o ex.S

esp: stack pointer – top of the stack

ebp: base pointer (aka frame pointer) – param/local var of current statck frame

eip: instruction pointer – next CPU instruction to be executed

Intel X86 32 bit


Avi Kak Computer Security lecture note

void function(char *str) {

char buffer[16];



void main() {

char large_string[256];

int i; for( i = 0; i < 255; i++)

large_string[i] = 'A';



Another buffer overflow example

Return address is overwritten and becomes


You get segmentation fault

Worse, attacker can change flow of program

Overwritten by ‘A’ (0x414141…)

buffer sfp ret str*

<------ [ ] [ ] [ ] [ ]

Bottom of stack

Top of stack

Observing buffer overflow in action – try this at home

#include <stdio.h>

int main() {

while(1) foo();


int foo(){

unsigned int yy = 0;

char buffer[5]; char ch; int i = 0;

printf("Say something: ");

while ((ch = getchar()) != '\n') buffer[i++] = ch;

buffer[i] = '\0';

printf("You said: %s\n", buffer);

printf("The variable yy: %d\n", yy);

return 0;


gcc -fno-stack-protector buffover2.c -o buffover2

Avi Kak Computer Security lecture note

Heap overflow

  • Heap located above program code/global data

  • For use in dynamic data structures, e.g., linked lists

  • Can affect the memory following it

  • Unlike stack, there is no return address to overwrite

  • So aims to overwrite pointer to a function

    • E.g., a list of record containing data & their processing function

Another type of buffer overflow (usually in web applications)

Overflow when passing parameters to a routine


Web developer may just allocate 20 bytes for param1.

How does the program handle long phone number, e.g., 1000 digits?

Defense against buffer overflow

  • Boundary checking, sanity checking by developers

  • Canaries: a know value on the stack just before the return address – canary word

    • Check the canary when function is to return

    • Stack guard by Crispin Cowan (a gcc extension)

  • Non-executable stacks

  • Address randomization

  • Compiler boundary checking

    • In Java

    • Java JVM may still be susceptible to buffer overflow attacks

  • Login