html5-img
1 / 29

Malware Detection

Malware Detection. Slides courtesy of Mihai Christodorescu. The Rising Malware Tide. Malware is software with unwanted functionality. Viruses, trojans, backdoors, bots, adware, spyware, browser hijackers, downloaders, droppers, keyloggers, password stealers, ... “Blended” threats

zeal
Download Presentation

Malware Detection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Malware Detection Slides courtesy of Mihai Christodorescu

  2. The Rising Malware Tide • Malware is software with unwanted functionality. Viruses, trojans, backdoors, bots, adware, spyware, browser hijackers, downloaders, droppers, keyloggers, password stealers, ... • “Blended” threats 100,000,000 machines are infected. [Vint Cerf, World Economic Forum 2007] Mihai Christodorescu – “Behavior-based Malware Detection”

  3. Organized Cyber-Crime • Boom in online fraud: • Spamming • Trade in stolen data • Financial fraud • ID theft Malware is the tool of the trade. Mihai Christodorescu – “Behavior-based Malware Detection”

  4. The Changing Threat Landscape 1995: Hobby malware, for fun • Show programming prowess • Single author 2007: Professional malware, for profit • Collaborative development • Bug-fix releases, code reuse Botnets: distributed computing has finally arrived. Creator of the Melissa worm ? Mihai Christodorescu – “Behavior-based Malware Detection”

  5. Known Malware New Malware 1 New Malware 2 Failure of Signature Detectors Malware detectors still use signatures. Malware is obfuscated/transformed easily. Software diversity used successfully by malware. Virus Scanner Internet Paradigm shift in malware creation, yet no change in malware detection! ac028c0e86009d8edfac0ac075fb e81cfd72ef50b91000f7f15052b9 0:*:504b03040a0001000800*... *:188420:181779:*:8 ad6900f5088cab9356678e43c... 3:*:3e3c623e6c696e6b3c2f6... Mihai Christodorescu – “Behavior-based Malware Detection”

  6. Focus On Behavior New malware & malware families Number of variants grows exponentially. 100,000 86,876 53,950 31,726 20,731 11,136 8,821 10,000 1,000 Number of families stays constant. 325 335 274 202 (est.) 100 Family = malware with a common code base. 10 1 Time 2001 2002 2003 2004 2005 2006 A family is a collection of behaviors. A behavior can be shared by many families. [Kaspersky Labs, Symantec] Mihai Christodorescu – “Behavior-based Malware Detection”

  7. Main thesis Detection of obfuscated malware requires a semantic analysis of program behavior. Program verification provides the techniques necessary to perform malware detection effectively and efficiently. Mihai Christodorescu – “Behavior-based Malware Detection”

  8. Syntactic Semantic “ Execution of program M causes the system to reach a state where a copy of M has been sent by email. ” Specifying Behavior Byte signatures allow for fast detection. • But not resilient to obfuscation. High-level descriptions require expensive detection. • Resilient to obfuscation. Mihai Christodorescu – “Behavior-based Malware Detection”

  9. Malspec: Self-Propagation by Email push 10h push eax push edi call connect push esi push eax push [ebp+hMem] call wsprintfA add esp, 0Ch push [ebp+hMem] call lstrlenA push 0 push eax push [ebp+hMem] push ebx push eax push ecx push edi call send Connect Send Netsky.B Mihai Christodorescu – “Behavior-based Malware Detection”

  10. Malspec: Self-Propagation by Email push 10h push eax push edi call connect push esi push eax push [ebp+hMem] call wsprintfA add esp, 0Ch push [ebp+hMem] call lstrlenA push 0 push eax push [ebp+hMem] push ebx push eax push ecx push edi call send X := Arg1 Arg1 = X & Arg2= “EHLO.*” Connect Send Syntactic component describes temporal constraints. = + Semantic component describes dependency constraints. Netsky.B Mihai Christodorescu – “Behavior-based Malware Detection”

  11. X:=socket() connect(X) S:=process_name() send(X,“EHLO”) Z:=open(S) send(X,“DATA”) Y:=read(Z) send(X,T) Building a Real Malspec “Send Email” “Read Own Exe. Image” Mihai Christodorescu – “Behavior-based Malware Detection”

  12. send(X,T) Building a Real Malspec “Send Email” “Read Own Exe. Image” X:=socket() connect(X) S:=process_name() send(X,“EHLO”) Z:=open(S) send(X,“DATA”) Y:=read(Z) send(X,T) Mihai Christodorescu – “Behavior-based Malware Detection”

  13. Dependence constraint: X after socket = X before connect Automating Malspec Creation: Malspec Mining Local constraint Malware Sample Benign Program Benign Program Dependence constraint — Benign Program Benign Program Malspec Constraints X:=socket() connect(X) S:=process_name() send(X,“EHLO”) Z:=open(S) send(X,“DATA”) Y:=read(Z) send(X,T) Mihai Christodorescu – “Behavior-based Malware Detection”

  14. X:=socket() connect(X) S:=process_name() Z:=open(S) send(X,“EHLO”) send(X,“DATA”) Y:=read(Z) send(X,T) Malspecs Benefits • Choice of security-sensitive operations • Constraint-based execution order • Dependences free of obfuscation artifacts Expressive to describe even obfuscated behavior. Mihai Christodorescu – “Behavior-based Malware Detection”

  15. X:=socket() connect(X) S:=process_name() Z:=open(S) send(X,“EHLO”) send(X,“DATA”) Y:=read(Z) send(X,T) Malspec Detection Strategies • Static analysis • Dynamic analysis • Host-based IDS • Inline Reference Monitors Malspecs are independent of detection method. Mihai Christodorescu – “Behavior-based Malware Detection”

  16. X:=socket() connect(X) S:=process_name() Z:=open(S) send(X,“EHLO”) send(X,“DATA”) Y:=read(Z) send(X,T) Detection of Malicious Behavior Binary File Malware Detector Goal: Find a program path that matches the malspec. Mihai Christodorescu – “Behavior-based Malware Detection”

  17. X:=socket() connect(X) S:=process_name() Z:=open(S) send(X,“EHLO”) send(X,“DATA”) Y:=read(Z) send(X,T) Find A Malicious Program Path Interprocedural Control-Flow Graph Mihai Christodorescu – “Behavior-based Malware Detection”

  18. X:=socket() connect(X) S:=process_name() send(X,“EHLO”) Z:=open(S) send(X,“DATA”) Y:=read(Z) send(X,T) Match Malspec Operations Mihai Christodorescu – “Behavior-based Malware Detection”

  19. X:=socket() connect(X) S:=process_name() send(X,“EHLO”) Z:=open(S) send(X,“DATA”) Y:=read(Z) send(X,T) Match Malspec Constraints Program Constraint: The program fragment preserves the program expression bound to Z. Malspec Constraint: Z after open = Z before read Like a semantic def-use constraint. Mihai Christodorescu – “Behavior-based Malware Detection”

  20. Match Malspec Constraints Semantic nop wrt E = program fragment preserving an expression E. Program Constraint: The program fragment preserves the program expression bound to Z. Mihai Christodorescu – “Behavior-based Malware Detection”

  21. Match Malspec Constraints Need an Oracle... Program Constraint: The program fragment preserves the program expression bound to Z. Mihai Christodorescu – “Behavior-based Malware Detection”

  22. Advances in Decision Procedures Dramatic improvements in SAT solvers: • SATO[Zhang, CADE 1997] • GRASP[Marques-Silva & Sakallah, 1999] • zChaff[Moskewicz et al., DAC 2001] • BerkMin [Goldberg & Novikov, DATE 2002] SAT-based Bounded Model Checking: [Clarke et al., FMSD 2001] • SAT-specific speedups [Strichman, CHARME 2001] • Richer logics[Seshia et al., DAC 2003] A decision procedure can approx. an Oracle. Mihai Christodorescu – “Behavior-based Malware Detection”

  23. P add esp, 0Ch push [ebp+hMem] £ Using Decision Procedures Decision procedure Program Constraint: The program fragment preserves the program expression bound to Z. True/False Mihai Christodorescu – “Behavior-based Malware Detection”

  24. CFG Constraint satisfaction Malspec Simplify UCLID Semantics-Aware Detector Binary File Malspec operations Malspec constraints Semantics-Aware Malware Detector Disassembler CFG constructor Graph matching IDA Pro [Detlefs et al., “Simplify,” 2004] [Lahiri & Seshia, CAV 2004] Yes / No Mihai Christodorescu – “Behavior-based Malware Detection”

  25. Effective Detection With hard-coded semantic-nop patterns: With decision procedures: Mihai Christodorescu – “Behavior-based Malware Detection”

  26. Semantic-Nop features: Flow sensitivity Binding procedure Decision procedures Rich constraints Obfuscation resilience: Code reordering Register renaming Junk code Code substitution Semantic-Nop Detection Benefits Mihai Christodorescu – “Behavior-based Malware Detection”

  27. Detection Performance Powerful decision procedures are expensive. SAFE pattern matching 1–9 s Idea: Use expensive decision procedures only if cheap decision procedures do not provide a definitive answer. Simplify theorem prover 300–800 s UCLID bounded model checker Mihai Christodorescu – “Behavior-based Malware Detection”

  28. Program fragment Constraint No Yes Yes Yes/No Stack of Decision Procedures SAFE pattern matching ? Random execution “No, code does not satisfy constraint!” ? Simplify theorem prover ? UCLID bounded model checker Average cost, same decision power. Mihai Christodorescu – “Behavior-based Malware Detection”

  29. Performance Results Detection times in seconds Test setup: 1 GHz CPU, 1 GB RAM Comparison: Commercial signature-based detector: <1s Decision procedure-based detector: >300s Mihai Christodorescu – “Behavior-based Malware Detection”

More Related