Operating System Security :

1. Operating System Security : A Study of Windows Rootkits David Phillips

2. What Is a “Rootkit”? • The term “Root” (super user) originates from the Unix operating system and “kit” is the program that grants an attacker super-user abilities. • Different Applications • A Rootkit can be a tool that an attacker installs into a system in order to deceive the other processes that are running on the system. • A Rootkit can be a tool that hides in memory and snoops on system activities while not engaging in deception.

3. What is NOT a Rootkit? • A Rootkit is not a computer virus (although a virus might deploy a Rootkit to assist in an attack). • A computer virus is self-replicating and generally advertises its presence on the system by causing damage and attempting to infect other programs. • A Rootkit does not attempt to replicate itself nor does it typically engage in damaging the system. Its role is more covert in nature and focused on information snooping or modifying the rules of the system.

4. How Does A Rootkit Work? • A Rootkit “program” is not a typical executable file. • The Rootkit logic is typically implemented within a kernel module (device driver) or shared library (Dynamic Link Library). • An executable user-space program called the “loader” program will load the Rootkit module/library into the system using well known Application Programmer Interface (API) system calls. • Once the Rootkit module is loaded, the module’s entry point initialization routine is called by the operating system. The Rootkit uses the initialization routine to install itself.

5. Rootkit Classification • Rootkits can be classified into two different types: • User Space Rootkits (User Application) • Kernel Space Rootkits (Device Driver)

6. User Space Rootkits • Pros: Easier to write (access to lots of user-space libraries and API’s that cannot be accessed from Kernel space). No kernel module required. Relatively easy to “inject” into other processes. • Cons: Does not have root privileges on system. Easier for anti-Rootkit software to detect. Affects only the behavior of a single process. No access to kernel data structures.

7. Kernel Space Rootkits • Pros: Affects ALL processes running on the system. Has root privileges. Harder for anti-Rootkit software to detect. Able to access kernel data structures. • Cons: More difficult to write. System user must have adequate rights to install.

8. Rootkit Implementation Phase • The Rootkit implementation phase of the project consisted of the creation of two Rootkits: 1.) A kernel space Rootkit that hides system process information by installing a hook function in the kernel’s System Service Dispatch Table in place of the kernel’s ZwQuerySystemInformation function. 2.) A user space Rootkit that hides system files by installing an inline function hook into Windows API FindNextFile function. 

9. Kernel Space Rootkit Implementation • Implemented using the Windows Device Driver Kit for Windows XP. • The Rootkit logic was compiled into a kernel driver/module named “rootkit_driver.sys”. • All driver modules export a function called ‘DriverEntry’. When the module is loaded into the kernel, the operating system automatically calls this routine.

10. Kernel Space Rootkit Driver Entry This function is executed in Kernel space. • Device drivers use this function to initialize themselves. • The Rootkit uses this function to make changes to the kernel structures and install “detours”.

11. ZwQuerySystemInformation • A function implemented in the kernel that constructs a list of processes currently running on the system and returns the list to the caller. • A user space application (such as the Windows Task Manager) cannot call this function directly because it exists only in the Kernel’s address space.

12. Background: Windows Task Manager • User Space application that displays all of the current processes running on the system. • Cannot call ZwQuerySystemInformation directly. • Must perform a System Call instead.

13. Windows Task Manager Control Flow • The Windows Task Manager issues a system call which causes control to jump to Kernel function ‘KiSystemService’. • The KiSystemService routine locates the ZwQuerySystemInformation in the System Service Dispatch Table using the system call number as an index. • KiSystemService then calls ZwQuerySystemInformation.

14. Kernel Space Rootkit Hook • The Rootkit module implements its own version of the ZwQuerySystemInformation routine named “NewZwQuerySystemInformation”.

15. Kernel Space Rootkit Hook • During execution of the Rootkit module’s DriverEntry routine, the Rootkit modifies the System Service Dispatch Table by replacing the kernel’s ZwQuerySystemInformation with the Rootkit’s NewZwQuerySystemInformation function. • From now on, when any process on the system (like the Task Manager) issues a System Call for the process list – the Rootkit’s NewZwQueryInformation routine will be called instead of the kernel’s ZwQuerySystemInformation function.

16. NewZwQuerySystemInformation • The Rootkit’s hook function NewZwQuerySystemInformation calls the kernel’s ZwQuerySystemInformation to masquerade as though the original function were called.

17. NewZwQuerySystemInformation • The original function returns the real list of processes back to the hook function. The hook function then iterates through the returned list and removes all process entries except for the “System Idle Process”. • The hook function then returns the modified list back to the original caller that invoked the system call.

18. Rootkit Filters The Process List

19. Task Manager After Kernel Hooked • The Task Manager receives back the process list that was filtered by the Rootkit’s hook function. • As a result, no processes show up in the list except for the System Idle Process.

20. User Space Rootkit Implementation • Implemented as a Dynamic Link Library using Visual Studio 2005. • The Rootkit logic was compiled into a Dynamic Link Library named “UserSpaceRootkit.dll”. • The Rootkit library is activated by a user-space loader program that “injects” the library into a target process that is already running on the system.

21. DLL Injection • In order to activate the Rootkit, the DLL that contains the Rootkit implementation needs to be loaded into the address space of a target process. • Forcing another process to load a DLL into its address space is called “DLL Injection” • This action is performed by the Rootkit loader program.

22. How Does a Process Load a DLL? • A process typically loads a DLL into its own address space by calling the Windows API function “LoadLibrary”.

23. How Does a Process Cause Another Process To Load a DLL? • A process can cause another target process to load a DLL by instructing any thread within the target process to execute the LoadLibrary function. • The Windows API defines a function named ‘CreateRemoteThread’ that allows a process to create a new thread within a another process.

24. How Does The Rootkit Loader Inject The Rootkit DLL Into the Target Process? • The Rootkit loader creates a remote thread in the target process. For the thread’s entry point function, the Rootkit passes the address of the LoadLibrary function. It passes the string “UserSpaceRootkit.dll” as a parameter to the LoadLibrary routine.

25. Rootkit Loader Continued • The remote thread begins executing the LoadLibrary function within the context of the target process and the UserSpaceRootkit.dll is loaded into the target process’s address space. • Once the UserSpaceRootkit.dll is loaded into the target process’s address space; the Operating System automatically calls the dllmain function that is exported by the DLL to allow it to initialize itself.

26. User Space Rootkit dllmain This function is executed in User space in the context of the target process. • DLL’s use this function to initialize themselves. • The Rootkit uses this function to make changes to a process’s structures and install “detours”.

27. FindNextFile • A Windows API function that is repeatedly called by user space programs that want to obtain a list of files contained on the file system. • When a process begins execution, the FindNextFile code is imported from the system library Kernel32.dll and mapped into the process’s address space. • The process can then call this function directly to iterate objects on the file system.

28. User Space Rootkit Hook • During execution of the dllmain function, the Rootkit locates the FindNextFile function in the target process’s address space. • After locating the function, the Rootkit changes the first 11 code bytes of the function to an unconditional jump to a “detour” function that is implemented in the UserSpaceRootkit.dll.

29. User Space Rootkit Detour Once the JUMP has been inserted, whenever any thread in the target process calls the FindNextFile function – control will jump to the Rootkit’s detour function which changes the value of the HANDLE function parameter to an invalid value.

30. Resume The Original Function • Once the detour has invalidated the function parameter, it needs to jump back to the original function to let it run “as normal” (but with corrupted parameters). • The detour function first executes the instructions that were overwritten by the inserted JMP instruction.

31. Resume The Original Function • After executing the overwritten instructions, the detour function then jumps back to the first code byte of FindNextFile that follows the inserted JMP instruction. • The FindNextFile function then proceeds as normally, but with a corrupted parameter that causes the function to return an invalid result.

32. The Windows Command Prompt • User space application that calls FindNextFile to list the contents of a directory when user types the command ‘dir’.

33. Command Prompt Injection • The UserSpaceRootkit.dll is injected into the command prompt using the DllInjector.exe loader.

34. Injected Command Prompt • Once the Rootkit is injected into the command prompt, whenever the user tries to list the files of a directory – an empty list is returned.

35. Thank you. Questions?