An empirical study of the reliability in unix utilities
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

An Empirical Study of the Reliability in UNIX Utilities PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

An Empirical Study of the Reliability in UNIX Utilities. Barton Miller Lars Fredriksen Brysn So Presented by Liping Cai. Overview. Introduction Tools Tests Results and Analysis Conclusions. Introduction. Why the study is important

Download Presentation

An Empirical Study of the Reliability in UNIX Utilities

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


An empirical study of the reliability in unix utilities

An Empirical Study of the Reliability in UNIX Utilities

Barton Miller

Lars Fredriksen

Brysn So

Presented by Liping Cai


Overview

Overview

  • Introduction

  • Tools

  • Tests

  • Results and Analysis

  • Conclusions


Introduction

Introduction

  • Why the study is important

    • Contributes to the testing community a large list of real bugs;

    • Finds the bugs that might indicate future security holes;

    • Finds that crashes might be caused by carelessly typing;

    • Would like some meaningful and predictable response when inadvertently feeding programs noisy input;

    • Major utilities should not crash because of noisy phone lines;

    • Would like to see the interactions between random testing and more traditional industrial software testing;


Introduction1

Introduction

  • The project proceeded in four steps:

    • Construct programs to generate random characters and to help test interactive utilities;

    • Use these programs to test a large number of utilities;

    • Identify the strings that crash the programs;

    • Identify the cause of the program crashes and categorize the common mistakes;


The tools

The Tools

  • Fuzz: generating random Input Strings

    fuzz 100000 –o outfile | deqn

  • -- capable of producing both printable and control characters, only printable characters, or either of these groups along with the NULL character.

  • -- can specify a delay between each character.

  • -- allows you to specify the seed for the random number generators, to provide for repeatable test.

  • -- can record its output stream in a file, in addition to printing to its standard output.

  • -- there are options to randomly insert newline characters in the output stream, and to limit the length of the output stream.


The tools1

The Tools

  • Ptyjig: Testing Interactive Utilities

-- allocates a pseudo-terminal files

-- starts the specified utility program.

-- ptyjig passes characters that are sent to its input through the pseudo-terminal to be read by the utility

-- example:

fuzz 100000 –o outfile | ptyjig vi


The tools2

The Tools

  • The scripts: automating the tests

    • Executes all the utilities for a given set of input characteristics

    • Checks for the existence of a “core” file after each utility terminates; indicating the crash of that utility

    • The core file and offending input data file are saved for later analysis


The tests

The tests

  • 7 versions of UNIX

  • 88 utility programs such as the mail program, screen editors, compilers and document formatting packages

  • Each utility program was subjected to several different types of input streams.

  • The major variations in test data were including control characters, NULL, and maximum length of the input stream.

  • The input streams for interactive utilities have slightly different characteristics.

    • To avoid overflowing the input buffers on the terminal device, the input was split into random length lines with a mean length of 128 characters.


The results and analysis

The results and analysis

  • Were able to crash or hang a significant number of utility programs on each system(from 24-33%);

  • Detected 2 types of error results, crashing and hanging.

    • Considered crashed if the program terminated producing a core file

    • Considered hung if the program continued executing producing no output while having available input or continued to produce output after its input had stopped.

  • A utility might crash on one system but not on another;

  • Grouped the causes of the crashes into different categories;


The results and analysis crash causes

The results and analysis(Crash causes)

  • Pointer/Arrays

    • Not checking for exceeding the range of the array.

    • References through a null pointer.

  • Not Checking Return code

    • Sign of careless programming

  • Input Functions

    • gets(), scanf() – no means of bounds checking are possible.

  • Sub-Process

    • Access to sub-process should be carefully controlled or you should insure that the program input to the sub-process is first checked. Alternatively, the utility should be programmed to tolerate with the failure of a sub-process


The results and analysis crash causes1

The results and analysis(Crash causes)

  • Interaction Effects

    • !o%8f causes the VAX version of csh to crash and !o%888888888f hung most versions of csh

  • Bad Error Handler

    • The error recovery routine only increments a count of the number of errors detected, did not exit the code.

  • Signed Characters

    • Can be considered sub-case of pointer/array error

  • Race Conditions

  • Undetermined Errors


Conclusions comments on the results

Conclusions(Comments on the Results)

  • All Arrays reference should be checked for valid bounds

  • All input fields should be bounded

  • Check all system call return values

  • Pointer values should often be checked before being used.

  • Judiciously extend your trust to others

  • If you redefine something to look too much like something else, you may eventually forget about the redefinition

  • Error handlers should handle errors

  • Goto statements are generally a bad idea


Conclusions comments on lurking bugs

Conclusions(Comments on Lurking Bugs)

  • Bugs often be ignored

    • It is often difficult to isolate exactly what activity caused the program to crash.

    • It quicker to try a slightly different method to get the current job done than it is to find and report a bug.

  • The number of bugs in UNIX might also be explained by its evolution.

  • UNIX should not be singled out as a buggy operating system


Fuzz revisited

FUZZ-Revisited

  • First, the continued prevalence of bugs in the basic UNIX utilities seems a bit disturbing.

  • Second, the reliability of network services and X-Window servers is good news.

  • Third, X-Window applications are no less prone to failure (and seem to be more so) than the basic utilities.

  • Fourth, the reliability of the freely-distributed GNU and Linux software was surprisingly good, and noticeably better than the commercially produced software.


References

References

  • ftp://grilled.cs.wisc.edu/fuzz

  • ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-revisited.pdf

  • ftp://ftp.cs.wisc.edu/paradyn/technical_papers/fuzz-nt.pdf


  • Login