"Enhancing Operating System Security: Threats and Countermeasures"

Operating Systems 14 - threats PIETER HARTEL

Security requirements • Confidentiality: to stop unauthorised users from reading sensitive information. • Availability: authorised users want the system to work as they expect it to, when they expect it to. • Integrity: Every data item/system component is as the last authorised modifier left it. 2

Threats

Access control model – AU3 • Authentication: determine who makes request • Authorisation: determine who can do which operation on an object • Auditing: make it possible to determine what happened and why Authentication Authorisation Request Subject (e.g.?) Reference Monitor Object (e.g.?) Audit log [Lam04] B. W. Lampson. Computer security in the real world. IEEE Computer, 37(6):37-46, Jun 2004. http://doi.ieeecomputersociety.org/10.1109/MC.2004.17 IIS 4 IntroSec

Attacks • Insider attacks • Trap doors (try the vi command :help 42) • Login spoofing • Exploiting code bugs • Malicious code (more…) • Buffer overrun (more…) • Privilege escalation (more…) • Exploiting the user • Phishing • Sony rootkit (more…)

char s[ ] = { … } ; /* * The string s is a * representation of the body * of this program from '0' * to the end. */ main( ) { int i; printf("char\ts[ ] = {\n"); for(i=0; s[i]; i++) printf("\t%d, \n", s[i]); printf("%s",s); } Malicious code • Output? • gcc Thompson.c • ./a.out > foo.c • gccfoo.c • ./a.out >bar.c • diff foo.cbar.c [Tho84] K. Thompson. Reflections on trusting trust. Commun. ACM, 27(8):761-763, Aug 1984 http://dx.doi.org/10.1145/358198.358210

void smash(constchar *fr) { char to[2]; strcpy(to,fr); } intmain(intargc, char * argv[]) { char fr[] = "abcdefghijklmnopqrstuvwxyz"; char to[2] ; strcpy(to,fr) ; printf("to=%p=%s\nfr=%p=%s\n", (void*)to, to, (void*)fr, fr); fflush(stdout); smash(to); return 0; } Buffer overrun • gcc -ggdbSmash.c • gdb ./a.out • break smash • run • bt • step • bt • Quit • gcc -fstack-protector-allSmash.c • ./a.out O. Mueller, Anatomy of a Stack Smashing Attack and How GCC Prevents It, Dr. Dobbs Journal, Jun. 2012, http://www.drdobbs.com/security/anatomy-of-a-stack-smashing-attack-and-h/240001832

Privilege escalation:course submission system intmain(intargc, char * argv[]) { char fn[N], buf[N]; uid_tid = getuid(); printf("rid=%d, eid=%d\n", id, geteuid()); snprintf(fn, N, "%s/%d", DIR, id); FILE *fp= fopen(fn, "w"); setreuid(id, id); printf("rid=%d, eid=%d\n", getuid(), geteuid()); fflush(stdout); while (gets(buf) != NULL) { fputs(buf,fp); fputc('\n',fp); } fclose(fp); return 0; } • lecturer: • mkdir/tmp/db • chmod700 /tmp/db • gcc'-DDIR="/tmp/db/"' Setuid.c • mv a.out /tmp/submit • chmod+s /tmp/submit • echo test | /tmp/submit • ls -lR /tmp/db /tmp/submit • id • student: • echo bbb | /tmp/submit find / -perm -4000 >junk 2>/dev/null&

Sony rootkit • 20M audio CDs with autorun.inf • Installed code to display license • Check for known copy programs which had to be stopped • Intercept all syscalls related to the CDROM • Permitting only the Sony music player from reading the CDROM • Cloaked! M. Russinovich, Sony, Rootkits and Digital Rights Management Gone Too Far, Blog 2005, http://blogs.technet.com/b/markrussinovich/archive/2005/10/31/sony-rootkits-and-digital-rights-management-gone-too-far.aspx

Linux rootkit • Modified system call table

Summary • Standard security requirements CIA • Code bugs and human behaviour facilitate attacks • The operating system is popular target of attacks • The operating system can do a lot to prevent, avoid or detect attacks • The reference monitor is the gold standard

"Enhancing Operating System Security: Threats and Countermeasures"