Download
1 / 13
150 likes | 449 Views
Malware Detection via Virtual Machine Monitoring. Wenke Lee. Host-Based Monitoring. Monitor host activities to detect the presence of malware Compromised applications, malware, etc. Run malware on host (a “controlled” environment”, e.g., a sandbox) to understand its behavior
E N D
Host-Based Monitoring • Monitor host activities to detect the presence of malware • Compromised applications, malware, etc. • Run malware on host (a “controlled” environment”, e.g., a sandbox) to understand its behavior • Analyze malware runtime data to understand malicious activities, input conditions, etc. • The host monitors and malware analyzers need to be tamper resistant and transparent to malware
Host-Based Monitoring (cont’d) • Virtual machine provides the separation required for tamper resistance and transparency • Security VM monitors the application VMs • Challenges: timely and fine-grained • Current research projects • Memory monitoring framework • Malware analysis tools/platform
Why Monitor Memory? • The only reliable source on the current state of a computer system is memory • Nearly endless data for security, forensics, etc • Running and (some) killed processes • Encryption keys and decrypted data • Network sockets and data • OS-level accounting information • User input (e.g., key strokes, mouse movement) • Screen captures and graphical elements • And much more!
Monitoring Memory onProduction-Level Systems • Passive Monitoring: Viewing memory in Ai from S without any timing synchronization between the two virtual machines • Active Monitoring: Viewing memory in Ai from S with event notification being sent from Ai to S to permit monitoring at relevant times • Locating Valuable Data: Applying models obtained from supervised learning to find critical data structures within the raw memory view
Passive Monitoring Monitoring application periodically views memory from another virtual machine -- technique known as VM introspection Mapping “raw memory view” to virtual addresses and symbols requires the steps shown in figure below. Address and symbol mapping can be performed by a VM introspection library (e.g., XenAccess) BD Payne, M Carbone, and W Lee. Secure and Flexible Monitoring of Virtual Machines. In ACSAC 2007.
The XenAccess Library • The only open source VM introspection library • Access to virtual addresses, kernel symbols, and more • Works with Xen and dd-style memory image files • Released in Spring 2006 • Maintained by GTISC to encourage more research • http://www.xenaccess.org
Active Monitoring Monitoring application receives event notification from Guest VM when code execution reaches one of the hooks installed in the Guest VM kernel. Hooks and all associated code are protected from tampering using hypervisor-enforced memory protections (i.e., User VM can not modify these security-critical components). Hooks invoke trampoline, which transfers control to the security application. BD Payne, M Carbone, M Sharif, and W Lee. An Arch for Secure Active Monitoring Using Virtualization. In Oakland 2008.
Ether • Use Intel VT hardware virtualization extensions to provide instruction execution on actual hardware • Extend the Xen hypervisor to leverage Intel VT for malware analysis • Provides for both instruction-by-instruction examination of malware, and also coarser grained system call-by-system call examination • System Diagram:
Ether: Experiments • We created two tools to test the Ether framework: • EtherUnpack: extracts hidden code from obfuscatd malware • EtherTrace: Records system calls executed by obfuscated malware • We then compared both of these tools to current academic and industry approaches • EtherUnpack: we compared how well current tools extract hidden code by obfuscating a test binary and looking for a known string in the extracted code • EtherTrace: we obfuscated a test binary which executes a set of known operations, and then observe if they were logged by the tool
