Rootkits – Avoiding detection

1. Rootkits – Avoiding detection Tillmann Werner, werner@bonn.eduSeminar Computer Security, B-IT 2006-11-27

2. Agenda Motivation and definition, short history of rootkits Entering the kernel: Hiding and starting Operating system internals: How things get executed Modern rootkit techniques Covert channels for stealth communication Countermeasures against rootkits

3. Motivation Hackers want to keep access to a successfully compromized box. At the same time, they want to remain undetected and thus need to hide their presence and traces. Using conventional ways to remotely access a hacked box is often much too noisy. Once a system is under control, an intruder normally wants to install his own invisible backdoor. All hacker activities and data related to those activities shall be invisible to legitimate users. Any permanent trace should be avoided, if possible.

4. Definition „A rootkit is a set of programs and code that allows a permanent or consistent, undetectable presence on a computer.“ Source: G.Hoglund, J. Butler: „Rootkits“, ISBN 0-321-29431-9 „A rootkit is a set of software tools intended to conceal running processes, files or system data from the operating system.“ Source: Wikipedia Encyclopedia, http://en.wikipedia.org/wiki/Rootkit

5. Privileged access and stealth In a nutshell: It‘s all about permanent access and stealth. Once a hacker has administrativeprivileges, assuring permanent accessis only a matter of her creativity. This presentation focusseson how to stay stealthy. Filtering I/O between two layerscould conceal the presenceof a rootkit. User Program Operating System Hardware

6. From early rootkits… First rootkits came up in the late eighties. They were written forUNIX-like operating systems (Windows was not yet around). Early versions consisted of a bunch of modified programs that replaced the original instances on a compromized box. Typical candidates are programs that are used to examine the current system status, like ls, ps, who, netstat, etc. Also, the login program was often modified to accept login attempts for a specially crafted user. Invoked by telnet, it enabled attackers to come back at any time they wanted.

7. … to modern rootkits The early rootkits were easily detectable: modified binaries differ in size or cryptographic hashes from the original versions. There are 1001 ways to explore the system status – one could even write own programs. Comparing their results reveals the presence of rootkit binaries. Modern rootkits step into a lower layer, the kernel. If they run in kernel mode, any userland program is under their control as well. User input to a program and its output can then already be filtered on the kernel level. Patching binaries is not necessary at all.

8. A rootkit design example User Space Kernel Space StealthProtection UsermodeProgram KernelDriver Keyboard Sniffer Packet Sniffer Main OS Kernel Modifications TCP portfor remote control

9. Entering the kernel: Hiding rootkit resources A common approach for hiding a rootkit is prefix-based filtering:A resource whose name matches a given prefix is considered to belong to the rootkit and is lurked to other programs. Such a prefix can be prepended to file names, directory names or program names (which are inherited by corresponding processes). A similar method allows for hiding network traffic from other processes. Relevant packets are equipped with a magic value in their payload. This simple technique enables an attacker to hide most of his activities quite successful.

10. Entering the kernel: Starting a rootkit A common technique is to implement the rootkit as kernel module or driver that can be loaded during runtime. Modification of a present driver is also possible. The kernel code itself can be altered to start a rootkit. Both the on-disk image and the running kernel must be changed (e.g. by altering /dev/kmem on Linux). A rootkit program can also get hooked into the system to load automatically during operating system startup, e.g. by patching/sbin/init or by using .ini files. The boot loader can be modified to apply patches to the kernel just before the start phase.

11. i386 architecture The i386 architecture provides four differentprotection domains in hierarchical order,so called „rings“. This allows for enforcinga security model on the hardware layer. Most operating systems only use the rings0 and 3 for compatibility reasons. Ring 0 is also known als „protected mode“ oder „kernel mode“. On modern operating systems only the kernel is allowed to enter it. Ring 3 is generally called „user mode“ and is used to execute code in an unprivileged level, i.e. with memory protection. 0 1 2 3

12. Entering ring 0 So, how can a program execute privileged instructions? The operating system provides system calls to a userland process. Common examples are functions like read(), write(), or open(). System calls are not used directly. Instead, they are encapsulated in libraries that can be used in high level languages. Examples are the libc or glibc on UNIX-like systems. Some platforms, like Microsoft Windows, do not want a program to use the low-level libraries directly. They provide API functions that simplify common programming tasks, like controlling a GUI.

13. Operating system internals – how things get executed Program : flow of execution : memory pointer Import Address Table User Space Kernel Space Interrupt Descriptor Table System Call Table

14. Operating system internals – how things get executed Program : flow of execution : memory pointer Import Address Table API FunctionLibrary Function User Space Kernel Space Interrupt Descriptor Table System Call Table

15. Operating system internals – how things get executed Program : flow of execution : memory pointer Import Address Table API FunctionLibrary Function User Space Choose IHfrom IDT Kernel Space Interrupt Descriptor Table System Call Table

16. Operating system internals – how things get executed Program : flow of execution : memory pointer Import Address Table API FunctionLibrary Function User Space Choose IHfrom IDT Choose Syscall from SCT Kernel Space Interrupt Descriptor Table System Call Table

17. Operating system internals – how things get executed Program : flow of execution : memory pointer Import Address Table API FunctionLibrary Function User Space Choose IHfrom IDT Choose Syscall from SCT System Call Kernel Space Interrupt Descriptor Table System Call Table

18. Operating system internals – how things get executed Program : flow of execution : memory pointer Import Address Table API FunctionLibrary Function User Space Return Choose IHfrom IDT Choose Syscall from SCT System Call Kernel Space Interrupt Descriptor Table System Call Table

19. Possible locations for intervention 1 Program Import Address Table API FunctionLibrary Function User Space Return 4 Choose IHfrom IDT Choose Syscall from SCT System Call 5 2 Kernel Space 3 Interrupt Descriptor Table System Call Table

20. IAT Hooking 1 Program On Windows platforms, theimport address table (IAT) ofa program contains pointersto API or library functions. A rootkit can modify the tableto make a pointer reference a modified function. This function at first execute hostile instructions and then eventually calls the original API function if revelation is not feared. But: Every application has ist own import address table and the rootkit has to place a DLL with its wrapper functions in the filesystem. Import Address Table API Function Rootkit Function

21. Modifying System Calls 2 Return Return Specially crafted system calls affect allcalling programs. The system call could check given argumentsin order to decide how to operate. If a detection of the rootkit is feared, an alternative routine is executed that returns a different, harmless value. Modification of system call code requires runtime kernel patching on multiple locations. This is dangerous as one single error could crash the operating system. ? Patched Syscall

22. System Call Table Hooking 3 System Call Choose Syscall from SCT Altering the system call tableis the most common rootkittechnique. Some system calls arereplaced with modifiedversions by changing the corresponding pointers in the syscall table. A rootkit system call can behave different depending on the calling process, e.g. an open() fails if the argument is a rootkit resource. Problem: System call tables are backed up multiple times. Integrity checks could easily reveal a rootkit‘s presence. System Call ? System Call Table Rootkit Function

23. Altering the System Call Selector 4 Choose Syscall from SCT Instead of altering the syscalltable, use your own! The kernel must knowwhere to find therootkit‘s syscall table. This can easily be done by changing the code of the system call chooser routine such that it consults the rootkit‘s table instead of the original one. But again, altering kernel functions during runtime would be necessary. System Call Table Rootkit System Call Table

24. Interrupt Descriptor Table Hooking 5 Choose IHfrom IDT SyscallSelector The pointer to the syscallselector code in the IDTcan be changed to pointto a modified instance. This function basicallybehaves like the original syscall selector but uses a modified instance of the system call table which references certain hostile syscalls. Modification of the original system call selector is not necessary. In theory, one could even go further and use an alternative IDT. The address in the Interrupt Descriptor Table Register (IDTR) must then be set to the location of the rootkit table. Rootkit SyscallSelector Interrupt Descriptor Table

25. … and without execution flow modification? 6 Hiding processes is also possible: Unlinking a rootkit from lists used by the kernel to manage active processes can fool tools like ps or top. Files can be camouflaged by placing data in the file system‘s slack space. While this is not totally safe, studies show that chances are good for such data to survive a very long time. Rootkit Process File System

26. Stealth communication: Covert channels (1) A rootkit is only of value if it allows remote access. To hide remote logins, covert channels are used. This method abuses another legitimate communication channel in order to transmit commands and their responses by manipulating certain properties. Timing channels use the relative timing of events to code information.Storage channels code information into existing data. The information transmitted via a covert channel can also be cryptographically encoded or obfuscated using steganography.

27. Stealth communication: Covert channels (2) Information can be stored in the identification field of IP packets. DNS queries can also contain information that can be interpreted differently. The protocol does not even require a direct connection. Timing channels can be formed by representing bits by the duration of a session, the interval between two arriving packets, … Information theory teaches that the possibility of covert channels cannot be completely eliminated. However, a statistical analysis leads to detection in many cases.

28. Countermeasures against Rootkits Mandatory access control can be implemented to reduce the danger of a system level compromize. Anti rootkits can control kernel data and the execution flow process themselves. Regular checks could detect modifications. Host-based intrusion detection and prevention systems can check system integrity by maintaining a list of cryptographic hashes for important resources. Integrity checks are only trusted when executed offline, that means from a trusted media while the suspect system is not running.

29. The Future of Rootkits Operating systems like Windows Vista and up-to-date Linux versions make use ring -1 on modern hardware to implement virtualization. A recent area of research is the development of so-calledvirtual machine based rootkits. The projects Blue Pill and SubVirt published working proof-of-concept methods that move a running operating system into a virtual environment. This environment is controlled from the outside. Hence, it is impossible to detect the rootkit from within the compromized operating system instance.

30. References Greg Hoglund, James Butler, Rootkits – Subverting the Windows KernelAddison-Wesley, 2006, ISBN 0-321-29431-9 Andreas Bunten, Rootkits – Techniken und Abwehr,Proceedings of 10. DFN-CERT/PCA-Workshop, 2003, ISBN 3-8330-0097-X Andrew S. Tanenbaum, Modern Operating Systems,Second Edition,Prentice Hall, 2001, ISBN 0-13-031358-0 Daniel P. Bovet, Marco Cesati, Understanding the Linux Kernel,O'Reilly, 2002, ISBN 0-596-00213-0 Greg Hoglund, A *REAL* NT Rootkit, patching the NT Kernel, Phrack 55-05, 1999 Black Tie Affair, Hiding out under Unix, Phrack 25-06, 1989 Halflife, Abuse of the Linux Kernel for Fun and Profit, Phrack 50-05, 1997 Invisible Things, http://invisiblethings.org Rootkit – Share your old stuff, keep your good stuff, http://rootkit.com US DoD, Covert Channel Analysis of Trusted Systems (Light Pink Book),Rainbow Series, 1993 Craig H. Rowland, Covert Channels in the TCP/IP protocol suite,1997, First Monday 05/97