Week 7 System Calls, Kernel Threads, Kernel Debugging

1. Week 7 System Calls, Kernel Threads, Kernel Debugging Sarah Diesburg Florida State University

2. First… • Any questions on • Part 1 – 5 system calls • Part 2 – xtime proc module

3. Story of Kernel Development Some context…

4. In the old days… • There were no modules or virtual machines • The kernel is a program • Has code, can compile, re-compile, make executable • When changes needed to be made, developers make changes in source and re-compile

5. How is the kernel different from a regular program? • Mostly in how it is executed • Boot loader loads the kernel image/executable during boot time • Sets kernel mode • Jumps to the entry point in the image/executable • Remember the generic booting sequence?

6. Quick Question • How would you make changes to the kernel and run those changes? • Make changes to the source • Re-complie the kernel source • Re-install the kernel source • Make sure the bootloader sees the new kernel image (grub) • Reboot and profit!

7. Getting more modern.. • Modules were created as bits of code that can be loaded and unloaded by the kernel in kernel mode • Made development easier • Instead of re-compiling, re-installing, and rebooting into the new kernel, one could just re-compile and load a module

8. Quick Question • How would you make changes to a module and run those changes? • Make changes to module source code • Re-compile the module • Load the new module

9. Present Day • Reboots into new kernels and loading new modules often freezes machines • Enter virtual machine software • Process that emulates the hardware necessary to run an OS in user-space • Guest OS is executed inside the virtual machine process!

10. New System Calls Fun but tricky!

11. Implementing System Calls • Need to implement the functions above. But how? intstart_elevator(void); intissue_request(int #1, int #2, int #3); intstop_elevator(void);

12. Adding a System Call to Kernel • Files to add: • LINUX_DIR/PROJECT_NAME/Makefile • LINUX_DIR/PROJECT_NAME/PROJECT_NAME.c • LINUX_DIR/PROJECT_NAME/NEW_SYSCALLS.c • Files to modify: • LINUX_DIR/arch/x86/kernel/syscall_table_32.S • LINUX_DIR/include/asm-generic/unistd.h • LINUX_DIR/include/linux/syscalls.h • LINUX_DIR/Makefile

13. Sample System Call • Let’s add a sample module to the kernel that defines a sample system call • test_newsyscall(int test_int); • Takes an int test_int and issues a printk on it • Returns test_int • Problem – If our module isn’t loaded, what happens if we call our sample system call?

14. Project 2 System Call Model Elevator module Core kernel User program

15. Project 2 System Call Model Elevator module Core kernel User issues system call, core kernel looks up system call in system call table User program

16. Project 2 System Call Model Elevator module performs system call action Elevator module Core kernel User program

17. Project 2 System Call Model Elevator module returns result of system call Elevator module Core kernel User program

18. Project 2 System Call Model Elevator module Core kernel Core kernel forwards result of system call to user program User program

19. What happens if elevator module is not loaded? Core kernel User program

20. What happens if elevator module is not loaded? Core kernel User issues system call, core kernel looks up system call in system call table User program

21. What happens if elevator module is not loaded? Elevator module is not loaded to perform the action…. OOPS! Core kernel User program

22. Module System Calls • We must create a wrapper system call! • Wrapper will call module function if module loaded, else returns an error • Must be created in a separate, built-in kernel file in the project folder

23. Function Pointers • We will implement our system call wrapper with a function pointer • Pointer to a function • Function pointer can point to any function that you implement that • Takes the same input variable types • Returns the same return type

24. Function Pointers long (*STUB_test_newsyscall)(int test_int) = NULL; • Function pointer that • Returns a long • Name is STUB_test_newsyscall • Takes parameter int test_int • Function pointer set to NULL • Can set function pointer to a local function you implement

25. Elevator Project • Create a file in your elevator project that just contains the system call information • KERNEL_DIR/PROJECT_DIR/newsyscalls.c

26. KERNEL_DIR/PROJECT_DIR/newsyscalls.c #include <linux/linkage.h> #include <linux/kernel.h> #include <linux/module.h> /* System call stub. We initialize the stub function to be NULL. */ long (*STUB_test_newsyscall)(inttest_int) = NULL; EXPORT_SYMBOL(STUB_test_newsyscall); /* System call wrapper. If the stub is not NULL, it will be run, otherwise returns -ENOSYS */ asmlinkage long sys_test_newsyscall(inttest_int) { if (STUB_test_newsyscall) return STUB_test_newsyscall(test_int) else return -ENOSYS; }

27. KERNEL_DIR/PROJECT_DIR/newsyscalls.c #include <linux/linkage.h> #include <linux/kernel.h> #include <linux/module.h> /* System call stub. We initialize the stub function to be NULL. */ long (*STUB_test_newsyscall)(inttest_int) = NULL; EXPORT_SYMBOL(STUB_test_newsyscall); /* System call wrapper. If the stub is not NULL, it will be run, otherwise returns -ENOSYS */ asmlinkage long sys_test_newsyscall(inttest_int) { if (STUB_test_newsyscall) return STUB_test_newsyscall(test_int) else return -ENOSYS; } Function pointer

28. KERNEL_DIR/PROJECT_DIR/newsyscalls.c #include <linux/linkage.h> #include <linux/kernel.h> #include <linux/module.h> /* System call stub. We initialize the stub function to be NULL. */ long (*STUB_test_newsyscall)(inttest_int) = NULL; EXPORT_SYMBOL(STUB_test_newsyscall); /* System call wrapper. If the stub is not NULL, it will be run, otherwise returns -ENOSYS */ asmlinkage long sys_test_newsyscall(inttest_int) { if (STUB_test_newsyscall) return STUB_test_newsyscall(test_int) else return -ENOSYS; } Export the pointer so we can access it later

29. KERNEL_DIR/PROJECT_DIR/newsyscalls.c #include <linux/linkage.h> #include <linux/kernel.h> #include <linux/module.h> /* System call stub. We initialize the stub function to be NULL. */ long (*STUB_test_newsyscall)(inttest_int) = NULL; EXPORT_SYMBOL(STUB_test_newsyscall); /* System call wrapper. If the stub is not NULL, it will be run, otherwise returns -ENOSYS */ asmlinkage long sys_test_newsyscall(inttest_int) { if (STUB_test_newsyscall) return STUB_test_newsyscall(test_int) else return -ENOSYS; } System call wrapper

30. Elevator Project • Next create a separate file that • Holds your module code • Registers the system call pointer • Actually implements the system call behavior

31. Inside KERNEL_DIR/PROJECT_DIR/PROJECT_NAME.C /* Extern system call stub declarations */ extern long (*STUB_test_newsyscall)(inttest_int); long my_test_newsyscall(int test) { printk("%s: Your int is %i\n", __FUNCTION__, test); return test; } my_module_init() { STUB_test_newsyscall=&(my_test_newsyscall); return 0; } my_module_exit() { STUB_test_newsyscall=NULL; }

32. Inside KERNEL_DIR/PROJECT_DIR/PROJECT_NAME.C /* Extern system call stub declarations */ extern long (*STUB_test_newsyscall)(inttest_int); long my_test_newsyscall(int test) { printk("%s: Your int is %i\n", __FUNCTION__, test); return test; } my_module_init() { STUB_test_newsyscall=&(my_test_newsyscall); return 0; } my_module_exit() { STUB_test_newsyscall=NULL; } Gain access to stub function pointer.

33. Inside KERNEL_DIR/PROJECT_DIR/PROJECT_NAME.C /* Extern system call stub declarations */ extern long (*STUB_test_newsyscall)(inttest_int); long my_test_newsyscall(int test) { printk("%s: Your int is %i\n", __FUNCTION__, test); return test; } my_module_init() { STUB_test_newsyscall=&(my_test_newsyscall); return 0; } my_module_exit() { STUB_test_newsyscall=NULL; } Local function that implements syscall

34. Inside KERNEL_DIR/PROJECT_DIR/PROJECT_NAME.C /* Extern system call stub declarations */ extern long (*STUB_test_newsyscall)(inttest_int); long my_test_newsyscall(int test) { printk("%s: Your int is %i\n", __FUNCTION__, test); return test; } my_module_init() { STUB_test_newsyscall=&(my_test_newsyscall); return 0; } my_module_exit() { STUB_test_newsyscall=NULL; } Set stub function pointer to local function in init

35. Inside KERNEL_DIR/PROJECT_DIR/PROJECT_NAME.C /* Extern system call stub declarations */ extern long (*STUB_test_newsyscall)(inttest_int); long my_test_newsyscall(int test) { printk("%s: Your int is %i\n", __FUNCTION__, test); return test; } my_module_init() { STUB_test_newsyscall=&(my_test_newsyscall); return 0; } my_module_exit() { STUB_test_newsyscall=NULL; } Reset stub function pointer to NULL on module unload

36. KERNEL_DIR/PROJECT_DIR/Makefile obj-m := my_module.o obj-y := newsyscalls.o KDIR := /lib/modules/2.6.32/build PWD := $(shell pwd) default: $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

37. KERNEL_DIR/PROJECT_DIR/Makefile Compile as a module obj-m := my_module.o obj-y := newsyscalls.o KDIR := /lib/modules/2.6.32/build PWD := $(shell pwd) default: $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

38. KERNEL_DIR/PROJECT_DIR/Makefile obj-m := my_module.o obj-y := newsyscalls.o KDIR := /lib/modules/2.6.32/build PWD := $(shell pwd) default: $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules Compile as kernel built-in

39. Core Kernel Additions • Add the new system call to the core kernel system call table • Modify three files • Add the project directory to the main Makefile • Modify one file

40. Modifying syscall_table_32.S … .long sys_preadv .long sys_pwritev .long sys_rt_tgsigqueueinfo /* 335 */ .long sys_perf_event_open

41. Modifying syscall_table_32.S … .long sys_preadv .long sys_pwritev .long sys_rt_tgsigqueueinfo /* 335 */ .long sys_perf_event_open .long sys_test_newsyscall /* 337 */ • Add new system call to the end of the file. • Remember the number – you’ll need it in userspace!

42. Modifying unistd.h /* midfile */ #define __NR_perf_event_open241 __SYSCALL(__NR_perf_event_open, sys_perf_event_open) #undef __NR_syscalls #define __NR_syscalls 242 /* midfile */ • Can be found around line 623…

43. Modifying unistd.h /* midfile */ #define __NR_perf_event_open241 __SYSCALL(__NR_perf_event_open, sys_perf_event_open) #define __NR_test_newsyscall 242 __SYSCALL(__NR_test_newsyscall, sys_test_new_syscall) #undef __NR_syscalls #define __NR_syscalls 242 /* midfile */

44. Modifying unistd.h /* midfile */ #define __NR_perf_event_open241 __SYSCALL(__NR_perf_event_open, sys_perf_event_open) #define __NR_test_newsyscall 242 __SYSCALL(__NR_test_newsyscall, sys_test_new_syscall) #undef __NR_syscalls #define __NR_syscalls243 /* midfile */

45. Modifying syscalls.h asmlinkage long sys_perf_event_open( structperf_event_attr __user *attr_uptr, pid_tpid, intcpu, intgroup_fd, unsigned long flags); #endif /* EOF */

46. Modifying syscalls.h asmlinkage long sys_perf_event_open( structperf_event_attr __user *attr_uptr, pid_tpid, intcpu, intgroup_fd, unsigned long flags); asmlinkage long sys_test_newsyscall(inttest_int); #endif /* EOF */

47. Modifying KERNEL_DIR/Makefile # Objects we will link into vmlinux / subdirs we need to visit init-y := init/ drivers-y := drivers/ sound/ firmware/ net-y := net/ libs-y := lib/ core-y := usr/ endif # KBUILD_EXTMOD

48. Modifying KERNEL_DIR/Makefile # Objects we will link into vmlinux / subdirs we need to visit init-y := init/ drivers-y := drivers/ sound/ firmware/ net-y := net/ libs-y := lib/ core-y := usr/ my_module/ endif # KBUILD_EXTMOD • Found around line 475… • Can replace “my_module” with the name of your PROJECT_DIR

49. Getting it all to work • Re-compile the kernel • Install modules, install kernel • Make new initramfs image • Reboot • Test with a user-space program…

50. Sample User-space Program #include <stdio.h> #include <stdlib.h> #include <sys/syscall.h> #include <linux/unistd.h> #define __SYS_TEST_ELEVATOR 337 int main() { int test=5; long ret; ret=syscall(__SYS_TEST_ELEVATOR, test); if(ret<0) perror("system call error"); else printf("Function successful, returned %i\n", ret); return 0; }