Download
software reverse engineering education n.
Skip this Video
Loading SlideShow in 5 Seconds..
Software Reverse Engineering Education PowerPoint Presentation
Download Presentation
Software Reverse Engineering Education

Software Reverse Engineering Education

308 Views Download Presentation
Download Presentation

Software Reverse Engineering Education

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Software Reverse Engineering Education http://www.reversingproject.info Teodoro Cipresso, tcipress@hotmail.com San José State University, Spring 2009 Advisor: Dr. Mark Stamp Committee: Dr. Robert Chun, Dr. David Taylor

  2. Background InformationIntroduction to Software Reverse Engineering • Software Reverse Engineering (SRE) can be described as the practice of analyzing a software system to create abstractions that identify the individual components and their dependencies, and, if possible, the overall system architecture [1]. • Once the components and design of an existing system have been recovered, it becomes possible to repair and even enhance them. • Reverse engineering skills are also used to detect and neutralize viruses, worms and other malware, as well as to protect intellectual property [1].

  3. Background Information (cont’d)Importance of SRE Education • “More emphasis is needed in SE [and CS] undergraduate and graduate programs on the issue of software evolution and change. Students need to be educated on the theory and practice of software comprehension, maintenance and reengineering. They need to learn how to live with the monsters from the past and tame them” [2]. • “Most of the time, students are trained in developing very small programs starting from scratch. This approach is really misleading since most students learn to believe that software engineering is just about developing brand new software. In fact many students will be involved in evolution-related activities after completion of their studies” [3].

  4. Background Information (cont’d)Student Feedback on SRE Education • Incorporation of software reverse engineering techniques and methodologies into regular course work was tried at the University of Missouri-Rolla [1]. • The results of this experiment were quite positive: • 77% of students thought that the incorporation of SRE techniques and methodologies reinforced concepts taught during lectures. • 82% of students wanted SRE to be included in future courses, especially those that deal with software design.

  5. Background Information (cont’d)Development-Related Reversing Scenarios Figure 1. Development-related software reverse engineering scenarios.

  6. Background Information (cont’d)Security-Related Reversing Scenarios Figure 2. Security-related software reverse engineering scenarios.

  7. Background Information (cont’d)Legacy Software Development Process Figure 3. Software development process in a typical enterprise software system.

  8. Educate programmers on software reengineering and reuse Computer programmers with an improved ability to understand, evolve, and secure software. Educate programmers on software security and malware detection Educate programmers on software reversing, antireversing, and patching Project OverviewBaseline Education in Software Reverse Engineering Figure 4. Activities related to providing a baseline SRE education.

  9. Materials and Methods • More than ten peer-reviewed articles on the topics of software reverse engineering, re-engineering, maintenance, reuse, and security were selected and used to address the research questions. • Of the articles selected, three were chosen for their specific coverage of experiences with teaching courses in software reversing, reengineering, and maintenance. • Drew upon my experience, which is just shy of a decade, with designing and developing legacy software modernization tools at IBM.

  10. ResultsOverview of Developed SRE Course Modules • Reversing and Patching Wintel Machine Code • Reversing and Patching Java Bytecode • Applying Anti-Reversing Techniques to Machine Code • Applying Anti-Reversing Techniques to Java Bytecode • Reengineering and Reuse of Legacy Software • Identifying, Monitoring, and Reporting Malware

  11. Results (cont’d)Overview of Developed SRE Course Modules • Reversing and Patching Wintel Machine Code • Reversing and Patching Java Bytecode • Applying Anti-Reversing Techniques to Machine Code • Applying Anti-Reversing Techniques to Java Bytecode • Reengineering and Reuse of Legacy Software • Identifying, Monitoring, and Reporting Malware

  12. Results (cont’d)Reversing and Patching Wintel Machine Code • An introduction to the compilation of high-level languages to machine code is provided. Assembly is contrasted as having a one-to-one mapping to machine code • The negative results of experimentation with two decompilers (Boomerang and REC) for machine code are documented. Given the current state of decompiler technology, it was concluded that working with disassembly is the most feasible approach. • A Wintel machine code reversing and patching exercise was developed against Password Vault, a non-trivial application that is provided with the exercise to avoid any legal concerns with reversing software written by others.

  13. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) • The machine code reversing and patching exercise asks the learner to create a new executable version of the application that no longer has a trial limitation of five password records per user. • A reliable, and repeatable reversing strategy is used: place a breakpoint on a memory artifact and trace back stack frames to locate the section in the disassembly. • For instructional purposes, an animated solution that demonstrates the application of this reversing strategy using OllyDbg, an interactive debugger-disassembler, was developed using Qarbon Viewlet Builder.

  14. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 5. Animated solution to the Wintel reversing and patching exercise.

  15. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 6. Animated solution to the Wintel reversing and patching exercise.

  16. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 7. Animated solution to the Wintel reversing and patching exercise.

  17. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 8. Animated solution to the Wintel reversing and patching exercise.

  18. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 9. Animated solution to the Wintel reversing and patching exercise.

  19. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 10. Animated solution to the Wintel reversing and patching exercise.

  20. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 11. Animated solution to the Wintel reversing and patching exercise.

  21. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 12. Animated solution to the Wintel reversing and patching exercise.

  22. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 13. Animated solution to the Wintel reversing and patching exercise.

  23. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Figure 14. Animated solution to the Wintel reversing and patching exercise.

  24. Results (cont’d)Reversing and Patching Wintel Machine Code (cont’d) Idea for an advanced Wintel machine code (**) exercise: • It should be feasible to patch in additional function to the Password Vault machine code: • The GCC compiler can generate assembly language instead of machine code, so the programmer can work in a high-level language. • Patching in the generated assembly code would require some significant amount of time spent in the program understanding phase. • Final integration of the new code would require modification of the Windows PE header to increase the size of the .code section, also the .rdata and .data sections if new variables and constants are added.

  25. Results (cont’d) Overview of Developed SRE Course Modules • Reversing and Patching Wintel Machine Code • Reversing and Patching Java Bytecode • Applying Anti-Reversing Techniques to Machine Code • Applying Anti-Reversing Techniques to Java Bytecode • Reengineering and Reuse of Legacy Software • Identifying, Monitoring, and Reporting Malware

  26. Results (cont’d)Reversing and Patching Java Bytecode • An introduction to interpreted/intermediate executable formats such as Java bytecode is provided. These formats are contrasted with machine code and assembly language. • Java bytecode “disassembly” using javap is covered for help with analysis of bytecode generated by javac. • The positive results of experimentation with the Jad Java bytecode decompiler are documented; it is concluded that direct reading/writing of bytecode is not necessary. • A Java bytecode reversing and patching exercise was developed against a Java version of Password Vault.

  27. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) • The Java bytecode reversing and patching exercise asks the learner to create a new executable version of the application that no longer has a trial limitation of five password records per user. • Since the Password Vault application consists of a small number of classes in a single package, a simple reversing strategy of unpacking the Jar archive, batch decompiling the classes, modifying the generated Java source, and recompiling is used. • For instructional purposes, an animated solution that demonstrates the application of this reversing strategy using FrontEnd Plus, a graphical interface to Jad, was developed using Qarbon Viewlet Builder.

  28. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 15. Animated solution to the Java bytecode reversing and patching exercise.

  29. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 16. Animated solution to the Java bytecode reversing and patching exercise.

  30. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 17. Animated solution to the Java bytecode reversing and patching exercise.

  31. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 18. Animated solution to the Java bytecode reversing and patching exercise.

  32. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 19. Animated solution to the Java bytecode reversing and patching exercise.

  33. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 20. Animated solution to the Java bytecode reversing and patching exercise.

  34. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 21. Animated solution to the Java bytecode reversing and patching exercise.

  35. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Figure 22. Animated solution to the Java bytecode reversing and patching exercise.

  36. Results (cont’d)Reversing and Patching Java Bytecode (cont’d) Idea for an advanced Java bytecode (**) exercise: • Use available Java class libraries, such as jclasslib, to directly read and write Java bytecode. • Write a Java program that scans through the bytecode for the Java Password Vault application and locates the instructions for the trial limitation. • Once the instructions are located, overwrite them with a sequence that disables the trial limitation. • This can be good practice for getting a feel for writing code that patches an executable.

  37. Results (cont’d) Overview of Developed SRE Course Modules • Reversing and Patching Wintel Machine Code • Reversing and Patching Java Bytecode • Applying Anti-Reversing Techniques to Machine Code • Applying Anti-Reversing Techniques to Java Bytecode • Reengineering and Reuse of Legacy Software • Identifying, Monitoring, and Reporting Malware

  38. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code • An brief introduction to basic anti-reversing techniques is provided: Eliminating Symbolic Information, Obfuscating the Program, and Embedding Anti-Debugger Code. • Machine code typically has very little symbolic information that can be altogether eliminated, therefore a discussion illustrates how debuggers insert quite a bit of information that makes machine code easier to reverse. • The technique Obfuscating the Program, is demonstrated in a Wintel machine code anti-reversing exercise where data, computation, and control flow obfuscations are applied to the C++ source code for Password Vault.

  39. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • Commercial tools such as EXECryptor www.strongbit.com, fully obfuscate and pack Windows executables, using advanced algorithms that are based on the elementary techniques described in this module. • It is difficult to provide a “before and after” illustration of machine code that is obfuscated using EXECryptor, so the examples and exercise in this module are implemented first at the source code level and then confirmed in the machine code using live and static analysis. • In the case of control-flow obfuscation, only static analysis is used, where subsequent run traces are compared using an edit-distance measurement.

  40. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • The Wintel machine code anti-reversing exercise asks the learner to create a new executable version of the Password Vault application where the following transformations are applied: • Encryption of string literals (data obfuscation). • Obfuscation of the numeric representation of the password record limit (computation obfuscation). • Obfuscation of the method that performs the record limit check (control flow obfuscation).

  41. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • Encryption of String Literals (data obfuscation): Figure 23. Strings are decrypted each time they are used using a bundled cipher.

  42. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • Obfuscation of the numeric representation of the password record limit (computation obfuscation): Figure 24. Complex evaluations obscure the actual condition.

  43. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • Obfuscation of the numeric representation of the password record limit (computation obfuscation) (cont’d): Figure 25. Testing for a function of a number can slow a reverser down.

  44. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • Obfuscation of the method that performs the record limit check (control flow obfuscation): • We introduce some non-essential, recursive, and randomized logic to the password limit check to make it more difficult for a reverser to perform static and/or live analysis. • Since no standards exist for control flow obfuscation, a custom algorithm was designed to hinder live and static analysis through use of recursive and randomized procedure calls. • Recursion grows the stack considerably, making stepping through the code difficult, while randomization makes execution unpredictable (breakpoints may not trigger & run traces differ).

  45. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) Depth of the recursion is randomized on each check of the limit. Random procedure call targets generate and return a number that is added to an instance variable, preventing the procedures from being identified as NOOPs by a code optimizer. Figure 26. A control flow obfuscation algorithm for the record limit check.

  46. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • To measure the effectiveness of the control flow algorithm in hindering analysis, three execution traces of the section of the code containing the record limit check were compared. • The Levenshtein Distance (LD) was computed between the three traces where each instruction in the trace was compared. LD was modified to consider each line as opposed to each character. • The execution traces were collected using OllyDbg and had to be cleaned of disassembly artifacts such as line numbers, base addresses, and comments in order to ensure that the analysis was fair.

  47. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) Figure 27. Comparison of executions of record limit check on identical program input.

  48. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • The Wintel anti-reversing module also demonstrates source code obfuscation which is a useful anti-reversing technique for source code. • There may exist a requirement to ship the source code of an application so that the machine code can be generated on the end user’s computer. • If the source code contains intellectual property that is worth protecting, one can perform transformations to the source code which make it difficult to read, but have no impact on the machine code that would ultimately be generated when the program is compiled.

  49. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) • Demonstration of the COBF source code obfuscator: VerifyPassword.cpp: 01: int main(int argc, char *argv[]) 02: { 03: const char *password = "jup!ter"; 04: string specified; 05: cout << "Enter password: "; 06: getline(cin, specified); 07: if (specified.compare(password) == 0) 08: { 09: cout << "[OK] Access granted." << endl; 10: } else 11: { 12: cout << "[Error] Access denied." << endl; 13: } 14: } COBF invocation: 01: C:\cobf_1.06\src\win32\release\cobf.exe 02: @C:\cobf_1.06\src\setup_cpp_tokens.inv -o cobfoutput -b -p C: 03: \cobf_1.06\etc\pp_eng_msvc.bat VerifyPassword.cpp

  50. Results (cont’d)Applying Anti-Reversing Techniques to Machine Code (cont’d) COBF obfuscated source for VerifyPassword.cpp: 01: #include"cobf.h" 02: ls lp lk;lf lo(lf ln,ld*lj[]){ll ld*lc="\x6a\x75\x70\x21\x74 03: \x65\x72";lh la;lb<<"\x45\x6e\x74\x65\x72\x20\x70\x61\x73\x73 04: \x77\x6f\x72\x64""\x3a\x20";li(lq,la);lm(la.lg(lc)==0){lb<<"\x5b 05: \x4f\x4b\x5d\x20\x41" "\x63\x63\x65\x73\x73\x20\x67\x72\x61\x6e 06: \x74\x65\x64\x2e"<<le;}lr{lb<<"\x5b\x45\x72\x72\x6f\x72\x5d 07: \x20\x41\x63\x63\x65\x73\x73\x20\x64" "\x65\x6e\x69\x65 08: \x64\x2e"<<le;}} COBF generated header (cobf.h):