210 likes | 235 Views
Explore a layered approach for detecting malicious activities such as keystroke logging, DoS attacks, and more, using behavior-based models. Learn about sophisticated solutions and behavior graphs for accurate threat identification.
E N D
A Layered Architecture for Detecting Malicious Behaviors Lorenzo Martignoni , Elizabeth Stinson, Matt Fredrikson , Somesh Jha , John Mitchell RAID 2008
Bot-infected Computers • Botnets are used to perform nefarious tasks, such as: • keystroke logging, • spyware installation, • denial-of-service (DoS) attacks, • hosting • phishing web sites or command-and-control servers, • spamming, • click fraud, • license key theft
Disadvantage of Traditional Method • Even the most effective malware detectors fail to detect more than 30% of malware seen in the wild. • Traditional malware detectors are based on syntactic signatures • Malware producers can easily generate malware variants capable of evading existing signatures. • Malware detectors have a finite set of syntactic signatures, but malicious programs have in infitely mutable syntax
Behavior-based malware detection • Detect high-level actions that financially motivate malware development & distribution • keystroke logging • data leaking • proxying • program download and execute
Semantic gap between models and monitored events • Monitor execution of the program using an emulator • Lowest level events in behavior specifications are system calls • Malicious behaviors are described as sequences of essential actions • E.g. What we see NtDeviceIo. . . NtOpenFile NtCreateSe. . . NtMapView . . . is dierent from the essential actions we need to identify download a file and execute it Low-level events Behaviors
Solutions • Complex & high-level behaviors are decomposed into multiple layers. • The lowest layer represents system call invocations. • Upper layers have a richer semantics. • E.g. Hierarchy of events used to specify download_exec
Contributions • A behavior-speciation language that can be used to describe novel, semantically meaningful behaviors. • A detector that identifies when a process performs a specified high-level action, regardless of the process's source-code implementation of the action. • Our evaluation demonstrates that our detector can distinguish malicious execution of high-level behaviors from benign.
Behavior Graphs • A behavior graph is a directed graph of a form that is adapted from and extends AND/OR graphs.
Behavior Graphs • Internal nodes represent events (with formal parameters)
Behavior Graphs • Edges represent predicates on events arguments
Behavior Graphs • ORed edges represent events of which at least one has to occur
Behavior Graphs • ANDed edges represent events that all have to occur (but can occur in any order)
Behavior Graphs • Annihilator and replicator nodes represent events that destroy and duplicate resources
Behavior Graphs • Acceptor nodes represent actions taken by our system when behaviors are matched
Matching Malicious Behaviors OS events are passed to the lowest layer
Construction of behavior graphs • They developed our graphs manually and iteratively through domain knowledge and analysis of tens of gigabytes of execution traces, obtained from multiple runs of • 1. around fifteen standard applications • 2. over one hundred specially-crafted programs, • 3. several malicious programs.
Architecture of the system • Customized Qemu that instruments the guest code to monitor system call invocations, to perform taint analysis, and to track local user input. • A behavior matcher that receives events in real-time and tries to match each behavior graph loaded.
Spec. of Malicious Behavior • RI stands for Remotely Initiated • Tainted refers to data received over the network
Result on Malicious bots • Blank entries denote behaviors not matched because the bot did not implement them
Results on Benign Applications • Under 2 scenarios: UI refers to an experiment in which user input tracking was not used, and “UI” to one which enabled • User input tracking is very important to distinguish between behaviors triggered by the user and behaviors triggered automatically
Comments • False Positive Problem!! • E.g. Automatic Windows Update