Program Security

1. Program Security Jagdish S. Gangolly School of Business State University of New York at Albany NOTE: These notes are based on the book Security in Computing, by Charles & Shari Pfleeger (3rd ed) and are prepared solely for the students in the course Acc 661 at SUNY Albany. They are not to be used by others without the permission of the instructor. Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

2. Program Security • What does it mean to say a program is “secure”? • Approaches to judge quality • IEEE Terminology for quality • Types of flaws: Inadvertent flaws • Intentional flaws • Nonmalicious • Buffer overflows • Incomplete mediation • Time-of-check to Time-of-use errors Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

3. Program Security • Malicious code Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

4. Program Security • What does it mean to say a program is “secure”? • Some measure of trust that it enforces confidentiality, integrity, and availability • It takes too long to break through its security controls • It runs for a period of time with no apparent failures • It meets the security requirements in its specification Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

5. Approaches to judge quality • Penetrate-and-patch: • Search for faults and develop patches • Problems: • Patches may introduce new faults • Pressure to repair fault leads to a narrow focus on fault rather than on its context • Compare requirements with behavior of programs Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

6. IEEE Terminology for quality • Error: when a human makes a mistake • Fault: an incorrect step, command, process, or data definition in a computer program • Failure: A departure from the system’s required behavior Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

7. Types of flaws: Inadvertent flaws • Inadvertent flaws • Validation error (incomplete or inconsistent) • Domain error • Serializing and aliasing • Inadequate identification and authentication • Boundary condition validation • Other exploitable logic errors Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

8. Intentional flaws: Nonmalicious • Buffer overflows: Buffer is a place where data is held. • Array bounds example (some languages do not check array bounds, and in others array bounds do not have to be pre-specified) Char sample[10]; Sample[10]=‘A’; Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

9. Buffer overflows (Continued) The out-of-array-bounds data can overflow into • User’s data space, overwriting other existing data • User’s program data, • Overlaying an already executed instruction, with no effect • Overlay an instruction not yet executed, with the result an instruction with operation code 0X42 (internal code for character ‘B’) will be executed. If there is no such instruction, the system will halt with illegal instruction exception. Otherwise the machine will use succeeding bytes as if they were rest of the instruction Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

10. Buffer overflows (Continued) • What can a malicious programmer can do with buffer overflows? • Replace code in the system space, insert overflow data corresponding to machine code for instructions, and gain control back from the operating system, with higher privileges. • Cause an overflow into the stack, change either • old stack pointer thereby changing the context for the calling procedure) or • Return address to cause control to transfer where (s)he wants • Pass parameters on to a web server that causes buffer overflow and crashes the program Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

11. Incomplete mediation • Pass parameters to a web server that causes it to fail due to data type error, or execute with a wrong result. • This problem can be alleviated by client side checking of input, or by limiting the client-side choices only to valid ones (by drop-down boxes, check boxes, etc.). However, this can be rendered useless if the client manually edits the URLs in the http requests. Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

12. Time-of-check to Time-of-use errors • Also known as serialization or synchronization flaw • During the time that an access to resources (files) is checked and the time the result of the check is used, the user can change the descriptor of the resource, thereby exploiting the lack of synchronization • Use of digital signatures can alleviate this problem Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

13. Malicious code or Rogue program • Malicious code can do anything that a program can • Malicious code runs under the user’s authority • Malicious code can do anything that a user can, but without his/her permission or knowledge • Definition: Unanticipated or undesired effects in programs or program parts, caused by an agent intent on damage. Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

14. Malicious code (Continued) • Kinds of malicious code: • Virus: program that can pass on malicious code to other nonmalicious programs by modifying them • Transient: its life depends on the life of the host • Resident: locates itself in memory, and can remain active or be activated as a stand-alone program even after its attached program ends • Trojan Horse: In addition to primary function, also has a non-obvious malicious effect • Logic bomb: detonates when a specified condition occurs Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

15. Malicious code (Continued) • Time bomb: logic bomb whose trigger is a time or date • Trapdoor or backdoor: Someone can access the program by other than obvious, direct call, perhaps with special privileges • Worm: program that spreads copies of itself (as a standalone program) through a network • Rabbit: virus or worm that self-replicates without bound, with the intent of exhausting some computing resource Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

16. Viruses & Targeted malicious code • How viruses attach & types of viruses • Qualities appealing to virus writers • Boot sector viruses and Memory-resident viruses • Virus signatures • Polymorphic viruses • Virus prevention • Targeted malicious code: Trapdoors, Salami attacks, covert channels Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

17. How viruses attach & types of viruses • For a virus to do its work, it must be executed. Once executed, it may install itself in permanent memory, or spread itself • E-mail attachments: A common means is via e-mail attachments • Virus appended to a program: when the program is executed, the virus is also executed • Virus that surrounds a program: Runs the original program, but has control before and after the program execution • Integrated virus: integrates itself into the original program • Document virus: virus implemented in a formatted document Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

18. Qualities appealing to virus writers • Hard to detect • Not easily destroyed or deactivated • Spreads infection widely • Can re-infect its home program or other programs • Easy to create • Machine independent and operating system independent Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

19. Boot sector viruses and Memory-resident viruses • Boot sector viruses: When a computer is turned on, firmware recognizes the hardware present, tests them, copies a fixed number of bytes from the disk to a location in memory (bootstrap), and jumps to that address in memory in transferring control to the operating system. Chaining bootstrap makes installation of viruses attractive (by breaking the chain). • Memory-resident viruses: attached to memory-resident code since such code is executed frequently while the machine is running Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

20. Polymorphic viruses • A virus that can change its appearance • Forms: • More than one alternative but with equivalent beginning words, on installation one of the words is installed • Move pieces of the virus around to make detection difficult • Embed random numbers • Encrypting viruses Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

21. Virus signatures • Pattern of bytes that provide telltale signs of a virus. They are used by virus scanners to detect the presence of the virus • In case of attached viruses, the start of the virus code becomes detectable signature • Suspicious patterns such as JUMP instructions at the beginning of programs used by scanners to detect viruses • Scanners are effective only if up to date signatures are used Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

22. Virus prevention • Not sharing executable code • Setting e-mail handlers not to automatically open attachments • Be careful with large datasets and graphics since they can have embedded in them executables • Be careful with Microsoft file design problem (operating system may switch to the application of the hidden file type) • Use only commercial software acquired from reliable, well-established vendors (?) Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

23. Virus prevention (Continued) • Test all new software on an isolated computer • Open attachments only when known to be safe • Make a recoverable system image and store it safely • Backups of executable system files • Use detectors regularly and update frequently Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

24. Targeted malicious code • Trapdoors: undocumented entry point to a module • Sources of trapdoors: • During unit and integration testing, programmers use stubs (to mimic output) and drivers (to mimic input). They also embed control sequences in the design to support testing. If such code is not removed at the end of the testing, they can be used as trapdoors • Poor error checking: Unacceptable input may not be caught but passed on to the user for use • Undefined opcodes: hardware equivalent of poor error checking Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

25. Trapdoors (Continued) • Trapdoors can be useful in finding security flaws, but • Left on purpose with full understanding of their potential consequences • Must be documented • Access to them must be controlled • Causes of trapdoors: Trapdoors can persist in production because developers • Forget to remove them • Intentionally leave them for testing • Intentionally leave them for program maintenance • Intentionally leave them as covert means of access Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

26. Salami attack • Programs that disregard small amounts of money in computations (interest, tax calculations, etc.) may be vulnerable to salami attack where such shavings from transactions can be accumulated elsewhere • Auditing of corrections in computations involving large volume of transactions involving small amounts of money can prevent salami attacks Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

27. Covert channels: Programs that leak information • Covert channels communicate information to those who should not be receiving it. • Ways of creating covert channels: • Trojan horses • By encoding the format of data values in another innocuous report by varying the format of the output (changing the length of lines, last digit in insignificant field, etc,) • Storage channels, using file locks. Service programs and the spy need a common timing source broken into intervals • Timing channels pass information by using the speed at which things happen. Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly

28. Covert channels: How to identify them • Covert channels need shared resources. • Shared resource matrix: Acc 661 Auditing of Adv Acctg Sys (Spring 2003) Gangolly