An overview of malware packing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 88

An Overview of Malware Packing PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

An Overview of Malware Packing. Jenna Kallaher Costas Akrivoulis Raul Gonzalez. Packers. "Programs that transform an input binary's appearance without affecting its execution semantics" [1]. Packers.

Download Presentation

An Overview of Malware Packing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


An overview of malware packing

An Overview of Malware Packing

Jenna Kallaher

Costas Akrivoulis

Raul Gonzalez


Packers

Packers

"Programs that transform an input binary's appearance without affecting its execution semantics"

[1]


Packers1

Packers

"Programs that transform an input binary's appearance without affecting its execution semantics"

... For the purposes of

[1]


Packers motivation

Packers - Motivation

  • Frustrate reverse engineer!

  • Increase barrier to entry for analysis

    • Cost, Technical understanding

  • Avoid signatures

  • Avoid classifications/triage


Outline

Era 3

Block-based Packing

Instruction Virtualization

Era 2

Metamorphism

Anti-RE

Outline

Era 1

Encryption

Polymorphism


Era 1

Era 1

Alpha Pack


Encryption

Encryption

Alpha Pack


Encryption motivation

Encryption - Motivation

Goal

Evade static signature

Idea

  • Encrypt the binary

  • Different keys == different signatures


Encryption1

Encryption

What one thinks about when they hear "packer"

Packer encrypts all of the data and/or text,

then decrypts at runtime


Encryption key generation

Encryption – Key Generation

  • Embed key

  • Fetch key

  • Derive key


Encryption embed key

Encryption – Embed Key

B = Original Binary

K = Encryption Key

E = Encryption Function

Be = Ek(B)

D = Decryption Code

Malware = Be | K | D


Encryption fetch key

Encryption - Fetch Key

Decryptor code "phones home”

  • Retrieves key from master

    Master can authenticate malware

  • Verify environment

  • Attestation


Encryption derive key

Encryption - Derive Key

No phones

  • Evade attribution

  • No network constraints

    Targeted

  • Wrong environment == wrong key


  • Encryption limitations

    Encryption - Limitations

    Requires decryption before run

    • Decryptor can be signatured

      Encrypt the decryptor?

    • Need decryptor' for the decryptor...

      Some stub code needs to be "plain text"


    Polymorphism

    Polymorphism

    Alpha Pack


    Polymorphism motivation

    Polymorphism - Motivation

    Goal

    Make every signature unique

    Idea

    Mutate the encrypted binary

    • Re-key

    • Obfuscate decryptor code


    Polymorphism techniques

    Polymorphism - Techniques

    • Dead Code Insertion

    • Variable Renaming

    • Register Reordering

    • Code Reordering

    • Code Transposition

    • Instruction Substitution

    • Branch Inversion

    • ...


    Polymorphism dead code

    Polymorphism - Dead Code

    popebx

    popecx

    push.haha

    ret

    db0xab

    db0xbc

    db0xcdff

    db0xe8

    db0xa9

    .haha:

    xor eax, eax

    mov ecx, 1

    nop

    xchg eax, ecx

    push edx

    pop edx

    ret


    Polymorphism register reorder

    Polymorphism - Register Reorder


    Polymorphism limitations

    Polymorphism - Limitations

    Unencrypted binary is constant

    • Evades static signatures

      Emulators

    • Run malware, freeze upon decryption

    • Memory dump can be signatured

      Not robust enough


    Era 2

    Era 2

    Alpha Pack


    Metamorphism

    Metamorphism

    Alpha Pack


    Metamorphism motivation

    Metamorphism - Motivation

    Goal

    Make every signature unique

    Idea

    Mutate the plaintext binary

    • Physically different

    • Semantically equivalent


    Metamorphism techniques

    Metamorphism - Techniques

    • Dead Code Insertion

    • Variable Renaming

    • Register Reordering

    • Code Reordering

    • Code Transposition

    • Instruction Substitution

    • Branch Inversion

    • ...


    Metamorphism code reorder

    Metamorphism - Code Reorder


    Metamorphism substitution

    mov $0, %eax

    test %eax, %eax

    jz .hello

    Metamorphism - Substitution

    • A * 4 == A << 2

    • jmp .hello ==


    Metamorphism branch inversion

    Metamorphism - Branch Inversion

    if (false)

    bar()

    else

    foo()

    if (true) ==

    foo()

    else

    bar()


    Metamorphism limitations

    Metamorphism - Limitations

    Hidden Markov Models:

    System being modeled is assumed to be a Markov Process with hidden states.

    "State is not directly visible, but output, dependant on the state, is visible.“

    [2]


    Metamorphism limitations1

    Metamorphism - Limitations

    Hidden Markov Models:

    Use output, to reason about the properties of the hidden states in the system.

    Allows for Statistical Pattern Analysis


    Metamorphism limitations2

    Metamorphism - Limitations

    • Treat variants of a single binary as a family

    • Train a model, use HMM to represent the statistical properties of that family

    • Use trained model to decide whether a new variant belongs to the family or not - based on its statistical properties


    Metamorphism limitations3

    Metamorphism - Limitations

    [3]


    Metamorphism limitations4

    Metamorphism - Limitations

    Similarity Index:

    Pair program P with malware m, of family M

    Find their similarity index, Spm

    If somewhat similar,

    • Compare mwith its variants to find average similarity index, Sm, amongst family M

      If Sm < Spm, then P is in M


    Metamorphism limitations5

    Metamorphism - Limitations

    Detectable, variants are:

    • Too similar to each other

    • Too different from normal programs

      Good metamorphism requires high degree of:

    • Metamorphic variant differences

    • Regular programs similarities

      Defensive techniques exist (and work)


    Anti reversing

    Anti-Reversing

    Alpha Pack


    Anti re motivation

    Anti-RE - Motivation

    Enemies of malware:

    • Virtual Machines

    • Debuggers

    • Disassemblers

    • Emulators

    • Memory Dumpers

      Any of the above imply:

    • “Non-production" settings

    • Reverse engineering likely occurring


    Anti re motivation1

    Anti-RE - Motivation

    Goal

    Run ONLY on production systems

    Idea

    Find metrics for answering:

    • "Am I in a VM?"

    • "Am I in a debugger?"

    • "Am I being disassembled?"

    • "Am I being emulated?"

    • "Am I being dumped?"


    Anti re in malware

    Anti-RE - in Malware

    • [4]


    Anti re techniques

    Anti-RE - Techniques

    • [4]


    Anti disassembly

    Anti-Disassembly

    Alpha Pack


    Anti disassembly prevalence

    Anti-Disassembly - Prevalence

    • [4]


    Anti disassembly push ret

    Anti-Disassembly - Push Ret

    .code

    start:

    popeax

    popebx

    popecx

    push.continue

    ret

    db0xab

    db0xbc

    db0xcd

    db0xff

    db0xaa

    db0x1

    db0x2

    db0xff

    db0xef

    db0xe9

    db0xab

    db0xe8

    db0xa9

    .continue:

    xoreax,eax

    movecx,1

    nop

    xchgeax,ecx

    pushedx

    popedx

    ret

    .end start

    • [4]


    Anti disassembly push pop math

    Anti-Disassembly - Push Pop Math

    • Push known immediate to stack

    • Immediately pop to a register

    • Do some math on the register

    • Register holds desired value, but that value never appears directly in the code


    Anti virtualization

    Anti-Virtualization

    Alpha Pack


    Anti virtualization techniques

    Anti-Virtualization - Techniques

    Common x86 Instructions for VM Detection

    • cpuid

    • sidt (Red Pill)

    • sldt

    • sgdt

    • smsw

    • str

    • in

      System Artifacts

      Invalid Instructions


    Anti virtualization prevalence

    Anti-Virtualization - Prevalence

    • [4]


    Anti virtualization in

    Anti-Virtualization - in

    #define VMWARE_HYPERVISOR_MAGIC 0x564D5868

    #define VMWARE_HYPERVISOR_PORT 0x5658

    #define VMWARE_PORT_CMD_GETVERSION 10

    #define VMWARE_PORT(cmd, eax, ebx, ecx, edx) \

    __asm__("inl (%%dx)" : \

    "=a"(eax), "=c"(ecx), "=d"(edx), "=b"(ebx) :\

    "0"(VMWARE_HYPERVISOR_MAGIC), \

    "1"(VMWARE_PORT_CMD_##cmd), \

    "2"(VMWARE_HYPERVISOR_PORT), "3"(UINT_MAX) : \

    "memory");

    int hypervisor_port_check(void) {

    uint32_t eax, ebx, ecx, edx;

    VMWARE_PORT(GETVERSION, eax, ebx, ecx, edx);

    int in_hyp = (ebx == VMWARE_HYPERVISOR_MAGIC);

    printf(“%sVMWare\n”, in_hyp ? “”: “Not ”);

    return in_hyp;

    }

    • [6]


    Anti virtualization cpuid

    Anti-Virtualization - CPUID

    mov $0x1, %eax

    cpuid

    and $0x80000000, %ecx

    test %ecx, %ecx

    jnz virtualized

    mov $0x0, %eax

    ret

    virtualized:

    mov $0x1, %eax

    ret


    Anti virtualization artifacts

    Anti-Virtualization - Artifacts

    VMWare

    • VMWare Tools Process

    • 300+ registry references to VMWare in WinXP

    • 50+ file system references to VMWare or VMX in WinXP

      Virtual Hardware

    • MAC Address

    • USB Controller Type

      Anomalies in clock synch between host and guest

    • [7]


    Anti debugging

    Anti-Debugging

    Alpha Pack


    Anti debugging techniques

    Anti-Debugging - Techniques

    • Windows

      • IsDebuggerPresent

      • CheckIsRemoteDebuggerPresent

      • NtQueryProcessInformation

      • PEB

    • Linux

      • Ptrace of Self


    Anti debugging techniques1

    Anti-Debugging - Techniques

    • Windows + Linux

      • Rogue 0xCC/0x2D/0xF1

      • 0xCC Scanning

      • popf and trap

      • SS Register

      • Signal Handling


    Anti debugging prevalence

    Anti-Debugging - Prevalence

    • [4]


    Anti debugging 0xf1

    Anti-Debugging - 0xF1

    static void catch_function(int signal) {

    puts(“Signal caught.”);

    }

    int main(void) {

    if (signal(SIGTRAP, catch_function) == SIG_ERR) {

    fputs(“An error occurred! \n”, stderr);

    return EXIT_FAILURE;

    }

    puts(“Ice breakpoint added.”);

    asm __volatile__ (.byte 0xf1);

    puts(“This will never print if this program is being debugged.”);

    return 0;

    }


    Anti debugging rdtsc

    Anti-Debugging - RDTSC

    if(dbg() == 0)

    puts("No Debugger");

    else

        puts("Debugger");

    Int dbg(){

    asm volatile("rdtsc\n\t"

    "mov %%eax,%%ecx\n\t"

        "rdtsc\n\t"

        "sub %%ecx,%%eax\n\t"

        "cmp $0xfff,%%eax\n\t"

        "jae debugged\n\t"

        "mov $0x0,%%eax\n\t"

        "jmp done\n\t"

        "debugged:\n\t"

        "mov    $0x1,%%eax\n\t"

        "jmp done\n\t"

        "done:\n\t"

        "mov %%eax,%0\n\t"

        : "=g" (debugged)

        :: "eax");

    }


    Anti emulation

    Anti-Emulation

    Alpha Pack


    Anti emulation1

    Anti-Emulation

    • Emulators aren't perfect

    • Some instructions aren't handled correctly and the differences can be used to detect emulators

    • Finding these differences is oftentimes done by fuzzing


    Anti dumping

    Anti-Dumping

    Alpha Pack


    Anti dumping1

    Anti-Dumping

    • Nanomites

    • SizeOfImage

    • Guard Pages

    • Removing the PE Header


    Anti dumping nanomites

    Anti-Dumping - Nanomites

    Used by Armadillo

    Replaces branch instructions with 0xCCs

    Stores all jump information in an encrypted table

    • Jump Destination

    • Flags

      Uses self-debugging to handle 0xCCs:

    • Determines if it is a debug or nanomite breakpoint

    • Skips to jump destination if the jump should be taken


    Anti dumping sizeofimage

    Anti-Dumping - SizeOfImage

    // Any unreasonably large value will work say for example

    // 0x100000 or 100,000h

    void ChangeSizeOfImage(DWORD NewSize)

    {

    __asm {

    mov eax, fs:[0x30] // PEB

    mov eax, [eax + 0x0c] // PEB_LDR_DATA

    mov eax, [eax + 0x0c] // InOrderModuleList

    mov dword ptr [eax + 0x20],NewSize // SizeOfImage

    }

    }

    • [8]


    Anti dumping guard pages

    Anti-Dumping - Guard Pages

    • Not all code is decrypted immediately

    • These pages are replaced by guard pages

    • Guard pages generate exceptions

    • Upon exception, relevant code is decrypted


    Anti dumping remove pe header

    Anti-Dumping - Remove PE Header

    // This function will erase the current images

    // PE header from memory preventing a successful image

    // if dumped

    inline void ErasePEHeaderFromMemory()

    {DWORD OldProtect = 0;// Get base address of module

    char *pBaseAddr = (char *)GetModuleHandle(NULL);

    // Change memory protection (assume x86 page size)

    VirtualProtect(pBaseAddr, 4096, PAGE_READWRITE, &OldProtect);

    // Erase the header

    ZeroMemory(pBaseAddr, 4096);

    }

    • [8]


    Anti re limitations

    Anti-RE - Limitations

    VM assumptions no longer hold

    • Amazon EC2

      Defenders can hook specific "anti" checks

    • No general techniques...

      Endless cat and mouse game


    Era 3

    Era 3

    Alpha Pack


    Era 3 motivation

    Era 3 - Motivation

    Past techniques pack payload statically

    Reverse Engineers

    • Assume payload will at some point be present and unprotected

    • Wait until payload unpacks => analysis

      Era 3 packers violate this assumption

    • Never present entirety of code

    • Must now RE payload before applying SA


    Block based packing

    Block-based Packing

    Alpha Pack


    Block based packing motivation

    Block-based Packing - Motivation

    Goal

    Entire payload should never be present in memory at any given moment

    Idea

    Unpack small block of malware at a time

    • Re-pack previous blocks

      Only small code block inspectable by AV

    • not enough to meaningfully signature


    Block based packing limitations

    Block-based Packing - Limitations

    Performance

    • Smaller blocks need to be (re-)packed more often

      • code that loops across block boundaries

        Binary disassembly can be difficult[BIRD paper]

        Not all binary present for scan at once

    • ... but can save incrementally unpacked code


    Instruction virtualization

    Instruction Virtualization

    Alpha Pack


    Instruction virtualization motivation

    Instruction Virtualization - Motivation

    Goal

    Obfuscate malware through emulation

    Idea

    • Translate input binary into bytecode

    • Packed binary := emulator + bytecode payload

    • At runtime, emulator interprets bytecode

      Themida, VMProtect, Code Virtualizer


    Instruction virtualization process

    Instruction Virtualization - Process

    • Construct virtual instruction set arch (V-ISA)

      • (compatible with x86 for what payload requires)

    • Take input binary and translate x86 instructions to bytecode for V-ISA

    • Generate emulator (virt. processor) for V-ISA

      • "implementation" of ISA


    Instruction virtualization process1

    Instruction Virtualization - Process

    • [9]


    Opportunities to frustrate res

    Opportunities to Frustrate REs

    • Construct virtual instruction set arch (V-ISA)

      • Compatible ISA => randomize opcodes

      • Superfluous ISA features

        • Duplicate instructions

        • Unused instruction parameters

    • Generate emulator (virt. processor) for V-ISA

      • Can implement an ISA in many different ways

        • Put on your Computer Architect hats...

        • Single-cycle, pipelined, OoO, VLIW, dataflow

          Generated signatures are meaningless


    Instruction virtualization detection

    Instruction Virtualization - Detection

    Even if REs suspect that an emulator is used:

    Must detect all 3 parts of the process

    • Virtual ISA (semantics, syntax)

    • Bytecode (location)

    • Emulator (behavior)

      Remember: packed binary is a black box!


    Instruction virtualization rotalum

    Instruction Virtualization - Rotalumé

    • Emulator (behavior)

      • Dynamic instruction tracing

      • Abstract variable binding (forward, backward)

      • Read clustering

    • Bytecode (location)

      • Cross-testing clusters to determine VPC

    • Virtual ISA (semantics, syntax)

      • Dynamic taint analysis on read instructions


    Instruction virtualization limitations

    Instruction Virtualization - Limitations

    • Hard to identify emulator

      • Heuristics employed to detect emulator

        • Traditional decode/dispatch architecture

        • Abstract variable lifetimes

    • Hard to identify V-ISA fully (semantics,syntax)

      • Bytecode does not traverse all emulator paths

    • Performance


    Instruction virtualization limitations1

    Instruction Virtualization - Limitations

    Goal (as the unpacker)

    • Determine original, unpacked malware

    • Perform traditional static analysis

      Can't identify + understand V-ISA

      => Can't convert bytecode to native instructions

      => Can't analyze malware with static techniques


    Instruction virtualization thoughts

    Instruction Virtualization - Thoughts

    "We argue that this trend will continue and that a large portion of malware in the near future will be emulation based."

    "There is no existing technique that can reliably counter an emulation-based obfuscation technique."


    Miscellaneous

    Miscellaneous

    Alpha Pack


    Masquerading motivation

    Masquerading - Motivation

    Goal

    • Mislead malware triage

    • Confuse Reverse Engineers

      Idea

      Mimic statistical features of a different packer


    Masquerading techniques

    Masquerading - Techniques

    • Signature Injection

    • Statistical Feature Spoofing


    Masquerading signature injection

    Masquerading - Signature Injection

    Majority of packer signatures are regex

    Inject others' signatures in our malware

    [Armadillo 3.00a -> Silicon Realms Toolworks]

    signature =

    60 E8 00 00 00 00 5D 50 51 EB 0F ?? EB 0F ?? EB 07 ?? EB 0F ?? EB 08 FD EB 0B F2 EB F5 EB F6 F2 EB 08 FD EB E9 F3 EB E4 FC ?? 59 58 50 51 EB 0F ?? EB 0F ?? EB 07 ?? EB 0F ?? EB 08 FD EB 0B F2 EB F5 EB F6 F2 EB 08 FD EB E9 F3 EB E4 FC ?? 59 58 50 51 EB 0F

    ep_only = true

    • [10]


    Masquerading feature spoofing

    Masquerading - Feature Spoofing

    Use Instruction Virtualization

    as previously described

    DO NOT generate random ISA

    DO generate ISA that's statistically similar to the output of other packers.


    Future techniques

    Future Techniques

    • What's next?


    What it looks like

    What It Looks like

    Alpha Pack


    An overview of malware packing

    • [11]


    References

    References

    • Guo, Fanglu, Peter Ferrie, and Tzi-Cker Chiueh. "A study of the packer problem and its solutions." Recent Advances in Intrusion Detection. Springer Berlin Heidelberg, 2008.

    • Rabiner, Lawrence R. "A tutorial on hidden Markov models and selected applications in speech recognition." Proceedings of the IEEE 77.2 (1989): 257-286.

    • Wong, Wing, and Mark Stamp. "Hunting for metamorphic engines." Journal in Computer Virology 2.3 (2006): 211-229.

    • Branco, Rodrigo Rubira, Gabriel Negreira Barbosa, and Pedro Drimel Neto. "Scientific but Not Academical Overview of Malware Anti-Debugging, Anti-Disassembly and Anti-VM Technologies."

    • Garfinkel, Tal, et al. "Compatibility is not transparency: VMM detection myths and realities." Proceedings of the 11th Workshop on Hot Topics in Operating Systems (HotOS-XI). 2007.

    • http://kb.vmware.com/kb/1009458

    • http://handlers.sans.org/tliston/ThwartingVMDetection_Liston_Skoudis.pdf

    • http://www.codeproject.com/Articles/30815/An-Anti-Reverse-Engineering-Guide

    • Sharif, Monirul, et al. "Automatic reverse engineering of malware emulators."Security and Privacy, 2009 30th IEEE Symposium on. IEEE, 2009.

    • http://reverse-engineering-scripts.googlecode.com/files/UserDB.TXT

    • https://corkami.googlecode.com/files/packers.pdf


    An overview of malware packing

    Questions?


  • Login