Software fault injection
Advertisement
This presentation is the property of its rightful owner.
1 / 18

Software Fault Injection PowerPoint PPT Presentation

Software Fault Injection. Kalynnda Berens Science Applications International Corporation NASA Glenn Research Center. What is Software Fault Injection?. A testing technique that aids in understanding how software behaves when stressed in unusual ways. A product -based assurance technique.

Related searches for Software Fault Injection

Download Presentation

Software Fault Injection

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Software fault injection

Software Fault Injection

Kalynnda Berens

Science Applications International Corporation NASA Glenn Research Center


What is software fault injection

What is Software Fault Injection?

  • A testing technique that aids in understanding how software behaves when stressed in unusual ways.

  • A product-based assurance technique.

  • Variations in the technique allow it to be applied to many types of software and for different purposes.

Software Fault Injection NASA Glenn Research Center


How does sfi work

How does SFI work?

  • Legal permutations or faults are input at interfaces (external and/or internal).

  • Outputs show whether the injected fault propagates through the software.

  • Requires instrumentation (software code) to observe the propagation process.

Software Fault Injection NASA Glenn Research Center


Uses for software fault injection

Uses for Software Fault Injection

  • Finding defects in software

  • Robustness Testing

  • COTS Validation/Determining failure modes

  • Safety Verification

  • Security Assessment

  • Software Testability Analysis

Software Fault Injection NASA Glenn Research Center


Sfi examples

SFI Examples

  • Operating System Validation

    • Ballista (CM) – Linux and VxWorks robustness

    • WindowsNT

  • Network Security

    • NCSA httpd server

  • Safety

    • Advanced Automatic Train Control system

    • Magneto Stereoaxis System

SFI can be used with or without source code

Software Fault Injection NASA Glenn Research Center


Sfi without source code

SFI without Source Code

  • Create software wrapper for COTS functions and other interfaces

  • “Trick” OS to call wrapper functions first

  • Software under test usually run in debug mode

  • Wrapper can be used

    • Pass through for baselining response

    • Call alternative function

    • Call original function but change result

Software Fault Injection NASA Glenn Research Center


Sfi wrapper operations

SFI wrapper operations

Pass through wrapper:

OS or

Hardware

Application

Wrapper

Call alternative function:

New

Function

Wrapper

Application

Call original function but change result:

Wrapper

Application

OS or

Hardware

Software Fault Injection NASA Glenn Research Center


Center initiative on sfi

Center Initiative on SFI

  • Can SFI be used by an IndependentV&V engineer?

  • Is SFI a useful and cost-effective technique for NASA?

    • Are the errors and problems found of sufficient severity or abundance?

    • Are the costs of applying the technique reasonable for the number/severity of errors found?

  • Is SFI a good tool for safer software?

Software Fault Injection NASA Glenn Research Center


Methodology

Methodology

  • Determine scope

  • Select projects

  • Metrics

  • Perform SFI on projects

  • Create Test Plan (prototype due 1st quarter, FY02)

  • Lessons Learned

Software Fault Injection NASA Glenn Research Center


Determine scope

Determine Scope

  • Why narrow the scope?

    • SFI is a collection of related techniques

    • Comparison across projects requires using one technique for all

  • Why no source/interfaces technique chosen

    • IV&V perspective (cost effective)

    • “Outside” events or system limitations trigger many errors

  • Interfaces selected

    • COTS software

    • Hardware

    • User input

    • Communications medium

Software Fault Injection NASA Glenn Research Center


Project selection

Project Selection

  • Potential Projects

    • CM-2

    • Tempest Web Server (VxWorks and Java)

    • MDCA, FPP, SAMS, others

  • Selection Criteria

  • Selection difficulties

    • Project support not free

    • Contracted software not accessible

  • Final Choice

Software Fault Injection NASA Glenn Research Center


Metrics

Metrics

  • Time spent per task

    • Familiarization, researching errors, instrumenting software, testing

  • Subjective “effort” scale per task

  • Software project metrics

    • SLOC, #classes/modules, complexity, interface information

  • Fault Injection metrics

    • #faults, #failures, #faults no effect/correctly handled

Software Fault Injection NASA Glenn Research Center


Sfi process

SFI Process

  • Obtain Tempest software (completed)

  • Obtain access to VxWorks (completed)

  • ***Overcome compatibility problems

  • Determine all interfaces to test

  • Select errors to inject

  • Create necessary wrappers for SFI

  • Record test procedure and results

Software Fault Injection NASA Glenn Research Center


Tempest interfaces

Tempest Interfaces

  • VxWorks OS

    • Task creation and control functions

    • C/C++ language functions

    • File system functions

    • Networking functions

  • Outside world

    • Requests from external sources

    • Standard HTML, built-in functions

    • Tempest (VxWorks version) can execute OS functions

Software Fault Injection NASA Glenn Research Center


Example injection errors

Example Injection Errors

  • OS errors

    • Memory allocation failures

    • File errors (corrupted, not found)

    • Single task abort, hang

  • External World errors

    • Invalid request

    • Too many requests

    • Requests too frequent

Software Fault Injection NASA Glenn Research Center


Test plan

Test Plan

  • How to perform software fault injection on “generic” software

  • Steps prior to actual testing

  • Method of determining errors to inject

  • Procedure for performing the test

  • Appendices of lessons learned, example faults, other guidance

Software Fault Injection NASA Glenn Research Center


Difficulties encountered

Difficulties Encountered

  • Tempest documentation limited

  • VxWorks simulator does not support networking

  • Cost of hardware and full VxWorks not within the budget

  • Attempt to “fake” networking unsuccessful

Software Fault Injection NASA Glenn Research Center


Status and future work

Status and Future Work

  • VxWorks incompatibilities not easily overcome

  • Shift to Java version of Tempest for now

  • Test VxWorks version of Tempest on actual hardware (if possible) or alternate operating system (Linux, uClinux, eCos)

  • If funding continues, test on actual flight experiment (CM-2).

Software Fault Injection NASA Glenn Research Center


  • Login