Real Time Polymorphic Shellcode Detection

1. Real Time Polymorphic Shellcode Detection Evgeny Pinchuk (evgenyp@radware.com) Radware SOC Team

2. Introduction • Techniques for detecting buffer overruns • Protocol inspection for anomalies • Exploitation payload detection • What’s a shellcode • Pattern matching • Definition of polymorphism • In order to execute encrypted code, we must decrypt it first. • Differences between AV and IDS/IPS • Speed • Accuracy of executed code

3. Polymorphic vs. Regular • Regular Shellcode NOP Sled Shellcode Padding Return Address • Polymorphic Shellcode NOP Sled Decipher Engine Shellcode* Padding Return Address * Ciphered shellcode

4. Current techniques for detection • Counting NOP (or fake NOP) instructions • CPU consuming (making it not RT) • High false positives rate • Spectrum Analysis • High false positives • Beatable by four bytes encryption • Code emulation • CPU consuming (making it not RT) • Data Mining • Involves network learning mechanisms • High false positives rate • Preferred solution

5. The End Lets open the discussion !!!