1 / 16

An Empirical Study of the Reliability in UNIX Utilities

An Empirical Study of the Reliability in UNIX Utilities. Barton Miller Lars Fredriksen Brysn So Presented by Liping Cai. Overview. Introduction Tools Tests Results and Analysis Conclusions. Introduction. Why the study is important

Download Presentation

An Empirical Study of the Reliability in UNIX Utilities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. An Empirical Study of the Reliability in UNIX Utilities Barton Miller Lars Fredriksen Brysn So Presented by Liping Cai

  2. Overview • Introduction • Tools • Tests • Results and Analysis • Conclusions

  3. Introduction • Why the study is important • Contributes to the testing community a large list of real bugs; • Finds the bugs that might indicate future security holes; • Finds that crashes might be caused by carelessly typing; • Would like some meaningful and predictable response when inadvertently feeding programs noisy input; • Major utilities should not crash because of noisy phone lines; • Would like to see the interactions between random testing and more traditional industrial software testing;

  4. Introduction • The project proceeded in four steps: • Construct programs to generate random characters and to help test interactive utilities; • Use these programs to test a large number of utilities; • Identify the strings that crash the programs; • Identify the cause of the program crashes and categorize the common mistakes;

  5. The Tools • Fuzz: generating random Input Strings fuzz 100000 –o outfile | deqn • -- capable of producing both printable and control characters, only printable characters, or either of these groups along with the NULL character. • -- can specify a delay between each character. • -- allows you to specify the seed for the random number generators, to provide for repeatable test. • -- can record its output stream in a file, in addition to printing to its standard output. • -- there are options to randomly insert newline characters in the output stream, and to limit the length of the output stream.

  6. The Tools • Ptyjig: Testing Interactive Utilities -- allocates a pseudo-terminal files -- starts the specified utility program. -- ptyjig passes characters that are sent to its input through the pseudo-terminal to be read by the utility -- example: fuzz 100000 –o outfile | ptyjig vi

  7. The Tools • The scripts: automating the tests • Executes all the utilities for a given set of input characteristics • Checks for the existence of a “core” file after each utility terminates; indicating the crash of that utility • The core file and offending input data file are saved for later analysis

  8. The tests • 7 versions of UNIX • 88 utility programs such as the mail program, screen editors, compilers and document formatting packages • Each utility program was subjected to several different types of input streams. • The major variations in test data were including control characters, NULL, and maximum length of the input stream. • The input streams for interactive utilities have slightly different characteristics. • To avoid overflowing the input buffers on the terminal device, the input was split into random length lines with a mean length of 128 characters.

  9. The results and analysis • Were able to crash or hang a significant number of utility programs on each system(from 24-33%); • Detected 2 types of error results, crashing and hanging. • Considered crashed if the program terminated producing a core file • Considered hung if the program continued executing producing no output while having available input or continued to produce output after its input had stopped. • A utility might crash on one system but not on another; • Grouped the causes of the crashes into different categories;

  10. The results and analysis(Crash causes) • Pointer/Arrays • Not checking for exceeding the range of the array. • References through a null pointer. • Not Checking Return code • Sign of careless programming • Input Functions • gets(), scanf() – no means of bounds checking are possible. • Sub-Process • Access to sub-process should be carefully controlled or you should insure that the program input to the sub-process is first checked. Alternatively, the utility should be programmed to tolerate with the failure of a sub-process

  11. The results and analysis(Crash causes) • Interaction Effects • !o%8f causes the VAX version of csh to crash and !o%888888888f hung most versions of csh • Bad Error Handler • The error recovery routine only increments a count of the number of errors detected, did not exit the code. • Signed Characters • Can be considered sub-case of pointer/array error • Race Conditions • Undetermined Errors

  12. Conclusions(Comments on the Results) • All Arrays reference should be checked for valid bounds • All input fields should be bounded • Check all system call return values • Pointer values should often be checked before being used. • Judiciously extend your trust to others • If you redefine something to look too much like something else, you may eventually forget about the redefinition • Error handlers should handle errors • Goto statements are generally a bad idea

  13. Conclusions(Comments on Lurking Bugs) • Bugs often be ignored • It is often difficult to isolate exactly what activity caused the program to crash. • It quicker to try a slightly different method to get the current job done than it is to find and report a bug. • The number of bugs in UNIX might also be explained by its evolution. • UNIX should not be singled out as a buggy operating system

  14. FUZZ-Revisited • First, the continued prevalence of bugs in the basic UNIX utilities seems a bit disturbing. • Second, the reliability of network services and X-Window servers is good news. • Third, X-Window applications are no less prone to failure (and seem to be more so) than the basic utilities. • Fourth, the reliability of the freely-distributed GNU and Linux software was surprisingly good, and noticeably better than the commercially produced software.

  15. References • ftp://grilled.cs.wisc.edu/fuzz • ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-revisited.pdf • ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-nt.pdf

More Related