Malware detection based on application behavior modeling
This presentation is the property of its rightful owner.
Sponsored Links
1 / 77

Malware Detection based on Application Behavior Modeling PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on
  • Presentation posted in: General

Malware Detection based on Application Behavior Modeling. NWMTD’11 Jun 20–21, 2011. Mrs P.R.Lakshmi Eswari C-DAC, Hyderabad. Evolution of Malware Attacks. Malware Definition (Wikipedia). A software which is designed to infiltrate a computer system without the owner’s informed consent

Download Presentation

Malware Detection based on Application Behavior Modeling

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Malware detection based on application behavior modeling

Malware Detection based on Application Behavior Modeling

NWMTD’11

Jun 20–21, 2011

Mrs P.R.Lakshmi Eswari

C-DAC, Hyderabad


Evolution of malware attacks

Evolution of Malware Attacks


Malware definition wikipedia

Malware Definition (Wikipedia)

A software which is designed to infiltrate a computer system without the owner’s informed consent

Refers to a variety of forms of hostile, intrusive, annoying software code

MALicious softWARE


Threat from the malware

Threat from the Malware

  • A code

    • which collects the credit card number or any other personal info

    • Which makes an application do the buffer overflow and crash

    • Loosing the private and sensitive information

    • which shows annoying advertisements without your consent

    • Which encrypts the data and asks for money to decrypt it


Malware categories

Malware Categories


A typical malware

A Typical Malware

Exploit Logic

  • Motivational Logic

    • Spam

    • Data theft

    • Ransom

    • Disrupt the routine

  • Protection Logic

    • Packing

    • Anti Debugging

    • Anti Virtualization

Propagation Logic

Mails

USBs


Attacks classified

Attacks - Classified

  • Untargeted attacks

    • Attacking websites

    • Infecting portable storage devices

    • Attacking social networking websites

    • Wild malware (worms etc)

  • Botnets

  • Targeted Attacks


Targeted attacks

Targeted Attacks


A typical attack

A Typical Attack

Originally a executable

Doc file

Opens the file,

and executes the malware

Whenever updates windows, also downloads the malware, sends the data out etc.

Malware

Changes the windows update program


Botnet

Botnet

IRC Server

4. Attacker will also join this channel

(preferably through a program) and

issue commands (for e.g. update)

3. Join a channel on IRC

Receives the command (update)

1. Exploit / Attack

Victim

2. Download malware (bot)


Botnet1

Botnet

DDoS (distributed denial of service attacks)

Collecting lot of bank related data

Spidering attacks (on websites)

Spams

Using victim for other sensitive attack

Shutdown the computer etc


Motivation and business

Motivation and Business


Motivation and business1

Motivation and Business

14


Vulnerability exploit and race

Vulnerability, Exploit and Race


Vulnerability exploit and race1

Vulnerability, Exploit and Race


Malware detection techniques

Malware Detection Techniques

  • Black listing

    • Anti Virus

    • Intrusion Detection System

    • Behavior Based Malware Detection

  • White listing

    • Specification Based Detection

    • Anomaly Detection


Commercial solutions

Commercial Solutions


End system security suites

End System Security Suites

  • Centralized configuration on all clients

  • Centrally controlled

    • Firewall

    • Encryption

    • Device Control

    • Anti Malware

    • Security policies


White listing solutions

White listing Solutions

Core Trace Bouncer

Bit9 Parity

Robot Genius

Microsoft App Locker

McAfee Application Control


Don t want to pay

Don’t want to pay ? !

Free Anti Virus [AVG, AVIRA, AVAST]

Free Firewall [Zone Alarm]

URL Scanner [AVG, WOT, RG Guard]

Trend Micro Web Protection Add on

Disable Auto runs

Returnil Virtual System / Windows Steady State

Wehn-Trust HIPS [MUST for Windows XP – ASLR Tool]

Win-pooch HIPS [Windows XP]

OSSEC HIDS

WinPatrol [BillP Studios]


How anti malware works

How anti malware works?

Behaviors database

Behavior Based Engine

(On Process Activities)

Basic Activity Scanning *

Malware Signature database

Anti Virus Scanning

(On file content)

Known Applications database

White listing

(On process creation)

( * Process activity, file read or write )


Malware prevention system mps

Malware Prevention System (MPS)


Mps approach

MPS - Approach

Each application makes sequence of system calls for accessing various OS resources through multiple control paths (normal behaviour)

When the application is infected with malware, its behaviour changes

User Process n

User Process 1

User Process 2

……………..

User Space

System Calls

Kernel Space

Operating System

Detects malicious activity before it causes damage to end system i.e. before the system calls are executed by the operating system


Mps architecture

MPS - Architecture


Flowchart

Flowchart


Malware detection based on application behavior modeling

Malware Prevention System

1. Application Profiling and Model Generation Process in a Sandbox

4. Client

Protection against overall threats -

Process Execution Control

Model Enforcement Module

2. Server Manages the models and admin can set the policies here

Server communication module

3. Based on the policies the model gets pushed to clients

27


Malware detection based on application behavior modeling

Resource - A

System calls : {1,2,4}

Resource - B

System calls: {1,3,4,2}

Resource - C

System calls: {1,2,4}

Model Generation


Operations hooked in mps

Operations Hooked in MPS

File System Calls

Process hooks

Network Calls

Registry Calls


Deployment scenario

Deployment Scenario


System architecture

System Architecture


Database structure @ server

Database Structure @ Server


Database structure @ client

Database Structure @ Client


Index file @ server

Index File @ Server


Update request

Update Request

MPS Server

MPS Client

UPDATE_REQUEST

UPDATE_RESPONSE

Major No,

Minor No,

OS type,

ModelUpdate,

Db Major No,

Db Minor No

No.of Model Files,

Model File names,

ModelFile Path


File transfer request

File Transfer Request

MPS Server

MPS Client

TRANSFER_REQUEST

TRANSFER_RESPONSE

Model File

Name with

path

Contents of

the Model File


Log message request

Log Message Request

Application name,

OS type,

Date,

IP,

Operation,

Path

Success

or

Fail


Client and server technologies used

Client and Server – Technologies used

Server on Linux

Apache Server 2.2

Virtual Machine

Windows XP, Vista and 7 images

Linux 2.6.23 kernel image

Java runtime environment

PHP

HTTP message format

XML, OpenSSL

Windows Client

Mini Filter Driver

Call out Drivers

Win32 programming

C, C++ programming

PE Executable format Open SSL

Linux Client

Linux Security Modules

C, C++ programming

Qt Programming

OpenSSL


Server gui

Server GUI


Malware detection based on application behavior modeling

Client GUI


Malicious pdf

Malicious Pdf

Creation of Axsle.dll

Creation of Icucnv34.dll

Write file on cvs.exe

The malware repeatedly tries to write cvs.exe file and it gets blocked. The document doesn’t open until the write file operation on cvs.exe is completed.


Malware detection based on application behavior modeling

Malicious Pdf


Malware detection based on application behavior modeling

Stuxnet

  • Behaviors Detected

    • Hides view of system files

    • Hidden image file

    • File has system attribute

    • Creates logon entry

    • Unsigned binary

    • Drops executable

    • Modifies internet settings

    • Spawns process


Malware detection based on application behavior modeling

Stuxnet


Malware detection based on application behavior modeling

Stuxnet


Att27390 doc file

ATT27390 doc file

  • Activities blocked

    • Dropping of zipfldr.dll in system32 folder

    • Dropping of wuaueng.dll in system32 folder


Field testing report

Field Testing Report

  • MPS is compared with similar best commercial tools available in the market like NovaShield, Mamutu, Malware Defender, Sana Security Primary Response, Safe Connect, Threat fire etc.


Field testing report1

Field Testing Report

  • MPS is found sensitive against blended MS office and PDF documents wherein the MPS solution alone identified the malicious activity as the other industry product remain silent

  • Application has a tendency to raise false alarm against benign documents as it might match the enforcement policies defined

  • Overall it is felt that the solution is detecting high level targeted malware behaviours, but there is a need to improve the capabilities by suppressing the false alarms.


Malware resist simplifying and strengthening security

Malware ResistSimplifying and Strengthening Security

  • Detection Based on Runtime Behaviour. All running programs are monitored for a set of critical behaviors that could affect the normal functioning

Salient Features

Detection Based on Runtime Behavior

Small memory footprint and high detection rate

Co-exists with Anti Virus Solutions

Low False Positive Rate

Easy to Deploy and Use


Malware prevention system mps1

Malware Prevention System (MPS)


Ongoing research @ c dac hyderabad

Ongoing Research @ C-DAC Hyderabad

Design and Development of Anti Malware Solution for Web Applications and Mobiles


Malware analysis

Malware Analysis


The approach to analyze the malware

The approach to analyze the Malware

Run the malware in isolated lab

Monitor network and system connections

Understand the program’s code

Repeat until satisfied with gathered info


How to

How to?

  • Manual

    • Dedicated system (ready to be compromised)

    • Virtualized System

  • Automated Analysis


Automated analysis

Automated Analysis

Anubis [analyzing unknown binaries]

  • http://anubis.iseclab.org/

Virus total [analyze suspicious file]

  • http://www.virustotal.com/

Bit-Blaze [Malware Analysis Service]

  • https://aerie.cs.berkeley.edu/

Norman Sandbox

Joe Box Sandbox

Sunbelt CWSandBox

Comodo [Comodo Instant Malware Analysis]

  • http://camas.comodo.com/


Two steps phases

Two Steps / Phases

Behavioral (Dynamic) Analysis

Code (Static) Analysis

Gather as much as from behavioral analysis

Fill the gaps from the code analysis


Analysis

Analysis


Malware analysis1

Malware Analysis

To analyze malware, we requires basic and advanced knowledge in Windows and Linux concepts (depends)

For example: while doing behavioral analysis of the malware, we find malware modifies file A. – To get more out of it, we must know what is the significance of file A


Prepare the system

Prepare the System

  • Use VMWare and use the snapshot feature to restore state after malware execution

  • Use Virtual PC – execute the malware – Close and Delete changes

  • Physical System State Restore

    • Returnil Virtual System

    • Windows Steady State


Behavioral analysis

Behavioral Analysis

  • Activate various monitoring tools

  • Execute the malware

  • Terminate / suspend the malware process

    • Sometimes malware process comes again and again

  • Observe the results of monitoring tools


Process explorer

Process Explorer

  • Free from Microsoft TechNet

  • Super Task Manager

  • Shows process tree

    • We can know if malware created the new processes

  • Also shows files which a process is using

  • Can see the strings also


Process monitor

Process Monitor

  • Free from Microsoft TechNet

  • Monitors the following activities

    • Process creation

    • File related

    • Registry

    • Network related

  • Captures for all the process

    • Best is to do it for all and then apply the filters


Regshot

Regshot


Using idapro

Using IDAPro

Can reveal a lot of information

Great tool if user can reverse the C/C++ code


Use ollydbg

Use OllyDbg

  • OllyDbg is a great debugger

  • Open the sample using OllyDbg


Snort

Snort

  • Either use snort in a separate virtual machine to monitor its network activity

  • Or use tools like wire shark

  • Find

    • IRC server to whom this sample connects

    • Web servers?

  • May notice DNS queries


Packed malicious executables

Packed Malicious Executables

  • Packers compress / encrypt the executable

  • This is used

    • Difficult to analyze

    • Smaller size on hard disk

  • However runs unpacked and original in memory


How it executes

How it executes?

Small Decryptor extracts

the packed code and

executes the code

Executable Decryptor

Unpacked program in memory

Packed program stored as data


Pe format

PE Format

IMAGE_DOS_HEADER

MS-DOS Stub Program

IMAGE_NT_HEADERS

Signature

IMAGE_FILE_HEADER

IMAGE_OPTIONAL_HEADER

IMAGE_SECTION_HEADER

IMAGE_SECTION_HEADER

SECTION

SECTION


If it is packed

If it is packed

IMAGE_DOS_HEADER

IMAGE_DOS_HEADER

MS-DOS Stub Program

This is Decryptor code

MS-DOS Stub Program

IMAGE_NT_HEADERS

IMAGE_NT_HEADERS

IMAGE_SECTION_HEADER

IMAGE_SECTION_HEADER

IMAGE_SECTION_HEADER

IMAGE_SECTION_HEADER

SECTION

SECTION

SECTION

Original PE

SECTION


Packers availiable

Packers Availiable

UPX

ASPack

Themida

Petite

VMProtect


Malware detection based on application behavior modeling

PEiD


Process dumping with lordpe

Process dumping with LordPE

  • LordPE shows all the processes and can dump there images from memory

  • We can run the process from packed executable

    • Anyways it has to unpack itself in the memory

  • We can dump from memory using LordPE


Thank you

Thank You


  • Login