Stack allocation and buffer overflow

1. Stack allocation and buffer overflow CSCE 531 Presentation by Miao XU xum@engr.sc.edu

2. Outline • Stack allocation in Windows • What is buffer overflow • How to exploit buffer overflow • Demo

3. Stack allocation in TAM SB SB = Stack base LB = Locals base ST = Stack top globals call frame Dynamic link LB call frame ST

4. Stack allocation in TAM (Contd.) LB link Link data return address locals Local data ST • A frame contains • A dynamic link: to next frame on the stack (the frame of the caller) • Return address • Local variables for the current activation

5. What’s going on inside Windows? • Initial stack state ESP Current frame EBP EBP: Extended Base Pointer ESP: Extended Stack Pointer

6. What’s going on inside Windows? • Before call f(arg1, arg2) • Push arguments arg1 agr2 ESP Current frame EBP EBP: Extended Base Pointer ESP: Extended Stack Pointer

7. What’s going on inside Windows? • Before call f(arg1, arg2) • Push next instruction address Ret. Addr. arg1 agr2 ESP Current frame EBP EBP: Extended Base Pointer ESP: Extended Stack Pointer

8. What’s going on inside Windows? • Enter into f(arg1, arg2) • Push current EBP Prev. EBP Ret. Addr. arg1 agr2 ESP Current frame EBP EBP: Extended Base Pointer ESP: Extended Stack Pointer

9. What’s going on inside Windows? • Enter into f(arg1, arg2) • Move EBP to ESP Prev. EBP Ret. Addr. arg1 agr2 ESP Current frame EBP EBP: Extended Base Pointer ESP: Extended Stack Pointer

10. What’s going on inside Windows? • Enter into f(arg1, arg2) Current frame Prev. EBP Ret. Addr. arg1 agr2 ESP Previous frame EBP EBP: Extended Base Pointer ESP: Extended Stack Pointer

11. What is buffer overflow? • Related with stack allocation • A buffer overflow, or buffer overrun, is an anomaly where a process stores data in a buffer outside the memory the programmer set aside for it. • Wikipedia

12. Buffer overflow void function(char *str) { char buffer[8]; strcpy(buffer,str); } void main() { char large_string[256]; for( int i = 0; i < 255; i++) large_string[i] = 'A'; function(large_string); }

13. An example

14. An example

15. An example

16. An example

17. An example

18. An example Return to 0x41414141

19. Problems with buffer overflow • A demo

20. Acknowledgement • The demo and part of this slides are from the training when the presenter was in Symantec, Chinese Development Center, Beijing • The example comes from the following reference: • Aleph One, Smashing the stack for fun and profit, Phrack Magzine, Vol. 7 (49) , 1996

21. Questions?

22. Thank you