Analysis Avoidance Techniques of Malicious Software: Threats and Solutions

1. Analysis Avoidance Techniques of Malicious Software Murray Brand Edith Cowan University

2. Panda Labs Statement from 2010 • One third of all malware in existence was created in the first 10 months of 2010. • Daily virus signature files can be up to 100MB in size. • Systems struggling to handle the load in terms of downloads and scan times. • 48 hrs minimum time to create and distribute new virus definitions. New threats as much as 48 days. • Panda Security. (nd). Collective Intelligence. Retrieved 30 July 2011 from http://www.pandasecurity.com/usa/technology/cloud/collective-intelligence.htm

3. McAfee Q1 Threat Report 2011 • Malware – busiest quarter in history. • Identified more than six million unique samples in Q1 alone. • Expect 75 million samples in the “malware zoo” by end of 2011. • McAfee Labs, (2011). McAfee Threats Report: First Quarter 2011. Retrieved 30 July 2011 from http://www.mcafee.com/us/resources/reports/rp-quarterly-threat-q1-2011.pdf

4. Malware Analysis Body Of Knowledge (MABOK)

5. Taxonomy of Analysis Avoidance Techniques • Anti Emulation • Anti Online Analysis • Anti Hardware • Anti Debugger • Anti Disassemblers • Anti Tools • Anti Memory • Anti Process • Anti Analysis • Packers and Protectors • Rootkits

6. Analysis Avoidance Techniques are very effective • 80 techniques examined • A number of these implemented in standalone programs • All found to be effective • Can be used in various combinations/variations • Use can be detected and mitigated

7. Analysis Tools have Deficiencies • Various plugins available, but do not cover all techniques • Focus on hiding the tool • Do not necessarily log the detection of the technique • However, tools can be extended

8. Detection and Mitigation can be Effective • Scripting for debuggers and disassemblers can extend the functionality of the tools.

9. Packers and Protectors are extensively used by Malware • Malware invariably Packed/Protected • Measures of Entropy as good Detector • Packer signatures useful so appropriate unpacking technique can be used. • Packer signatures can vary just like AV signatures. • Custom Packers and Protectors

10. Derivation of an Appropriate Analysis Methodology

11. An Alternative Paradigm for Malware Detection is Required • Signatures and heuristics can be defeated • May not be prudent to submit samples for analysis • Sandboxes can be limiting and can be defeated • Malware invariably uses anti analysis techniques and deception techniques – could be a very good indicator of malicious software.

12. For the Analyst / Incident Responder • Do not totally rely on AV signatures • Malware is full of anti analysis techniques • Detailed malware analysis is very technically difficult and manually intensive • There are significant deficiencies in the tools • Anti analysis techniques can be detected and mitigated, but very manually intensive and extensive technical competency required. • Discovery of the intent of Deception

13. Existing Threats : Crimeware Toolkits

14. Protectors - Themida

15. Code Virtualizers

16. Social Engineer Toolkit

17. Threat Horizon • A Malware Rebirthing Botnet • Break existing AV?

18. Premises • Recognition of malware highly dependant upon exiting signatures. • Malware employs anti-analysis techniques to avoid detection and hinder analysis. • Open source software for collecting malware freely available. • Botnets – a collection of compromised computers directed by a C&C mechanism, used for a variety of nefarious purposes.

19. Moore’s Law / Malware Growth Rate • 1965 – Gordon Moore predicted that the number of transistors on an IC would double every two years. • Inference, processing power doubles every two years. • Malware Growth Rate • Non linear, increasing growth rate • Existing AV paradigm • signatures and heuristics • algorithms • Is there going to be a cross over point? • Will there come a time where the processing required to scan for malware overwhelm the capability of the computer?

20. Botnets in Perspective • CyberCrime (now, long established) • Mail relays for spam • DDoS • Malware distribution • ID theft • Phishing sites • Click Fraud • CyberWar (now and on the threat horizon) • Mobile Botnets (on the threat horizon)

21. The Idea behind the MRB • Integrate • Honeynets • Botnets • Exploitation frameworks • Anti analysis techniques • Exploit the way AV algorithms work • Exploit deficiencies in AV engines • Availability of AV signature files • Availability of online AV scanners/sandboxes • Test the hash

22. Malware Rebirthing BotnetRebirthing Suite

23. Malware Rebirthing BotnetFunctional Flow Block Diagram

24. Implications • A Win / Win Opportunity • For the bad guys  • Detected or not Detected • Concepts of operation for both scenarios

25. Salting the Earth • Salting the earth, or sowing with salt, is the ritual of spreading salt on conquered cities to symbolize a curse on its re-inhabitation. • Ridley, R.T. (1986). "To Be Taken with a Pinch of Salt: The Destruction of Carthage". Classical Philology81 (2)

26. Concepts of OperationPrinciple of Salting the Earth • Attack systems with rebirthed malware that is not detected by AV systems. • Compromise new systems, add nodes to the botnet, farm out for profit.

27. Concepts of OperationPrinciple of Salting the Earth • Attack systems with rebirthed malware that is eventually detected by AV systems. • Infect the entire network with as much stealthy, rebirthed malware as possible (then time release, or engage trigger mechanism to reveal obfuscated but known signature within the code) • A Denial of Confidence • Compromised network no longer trustworthy, take entire critical infrastructure network offline, snow ball effect on other services.

28. Concepts of OperationPrinciple of Salting the Earth • Inject known malware signatures into good network traffic, or into good code. • Overload Intrusion Detection Systems or other Sensors • Engage other attack whilst resources are diverted, or sensors are recalibrated or taken off line.

29. Concepts of OperationPrinciple of Salting the Earth • Analysing previously undetected malware is very manually intensive. • Hide the really malicious code amongst other code that triggers AV scanners. • Hide in plain sight • Generate so much malware that processing and scanning by existing AV software gets to point of no return.

30. Mitigations? • New paradigm for malware detection required. • Point of no return with existing paradigms sooner rather than later? • Detection of analysis avoidance techniques should raise a flag. • Whitelisting • Back to basics (keep it simple) • Constraints (patching etc) • Human behaviour modification • But management of technology is complicated enough! • Keep a finger on the pulse • Risk management • There is a need to keep an eye on the threat horizon. • Further research required on this front

31. Questions?