Software reliability engineering a roadmap
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

Software Reliability Engineering: A Roadmap PowerPoint PPT Presentation


  • 158 Views
  • Uploaded on
  • Presentation posted in: General

Future of Software Engineering ICSE ’ 2007 Minneapolis, Minnesota May 24, 2007. Software Reliability Engineering: A Roadmap. Michael R. Lyu Dept. of Computer Science & Engineering The Chinese University of Hong Kong. Introduction.

Download Presentation

Software Reliability Engineering: A Roadmap

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Software reliability engineering a roadmap

Future of Software Engineering

ICSE’2007

Minneapolis, Minnesota

May 24, 2007

Software Reliability Engineering: A Roadmap

Michael R. Lyu

Dept. of Computer Science & Engineering

The Chinese University of Hong Kong


Introduction

Introduction

  • Software reliability is the probability of failure-free operation with respect to execution time and environment.

  • Software reliability engineering (SRE) is the quantitative study of the operational behavior of software-based systems with respect to user requirements concerning reliability.

  • SRE has been adopted by more than 50 companies as standards or best current practices.

  • Creditable software reliability techniques are still in urgent need.


Historical sre techniques fault lifecycle

Historical SRE Techniques: Fault Lifecycle

  • Fault prevention: to avoid, by construction, fault occurrences.

  • Fault removal: to detect, by verification and validation, the existence of faults and eliminate them.

  • Fault tolerance: to provide, by redundancy and diversity, service complying with the specification in spite of manifested faults.

  • Fault/failure forecasting: to estimate, by statistical modeling, the presence of faults and occurrence of failures.


Fault lifecycle technique

Fault Lifecycle Technique

Fault Manifestation and Modeling Process

Reliability

Fault Prevention

Fault Removal

Fault Tolerance

Fault/Failure Forecasting


Fault lifecycle technique1

Fault Lifecycle Technique

Fault Manifestation and Modeling Process

Reliability

Availability

Safety

Security

Fault Prevention

Fault Removal

Fault Tolerance

Fault/Failure Forecasting


Software reliability modeling

Software Reliability Modeling

R = e -t

Testing Time


Current sre process overview

Current SRE Process Overview


Current trends and problems

Current Trends and Problems

  • The theoretical foundation of software reliability comes from hardware reliability techniques.

  • Software failures do not happen independently.

  • Software failures seldom repeat in exactly the same or predictable pattern.

  • Failure mode and effect analysis (FMEA) for software is still controversial and incomplete.

  • There is currently a need for a creditable end-to-end software reliability paradigm that can be directly linked to reliability prediction from the very beginning.


Future direction 1 reliability centric software architectures

Future Direction 1: Reliability-Centric Software Architectures

  • The product view – achieve failure-resilient software architecture

    • Fault prevention

    • Fault tolerance

  • The process view – explore the component-based software engineering

    • Component identification, construction, protection, integration and interaction

    • Reliability modeling based on software structure


Future direc t ion 2 design for reliability achievement

Future Direction 2: Design for Reliability Achievement

  • Fault confinement

  • Fault detection

  • Diagnosis

  • Reconfiguration

  • Recovery

  • Restart

  • Repair

  • Reintegration


Software reliability engineering a roadmap

Fault Confinement

Offline

Online

Fault Detection

Fault Detection

Failover

Diagnosis

Repair

Recovery

Reconfiguration

Restart

Reintegration


Future direc t ion 3 testing for reliability assessment

Future Direction 3: Testing for Reliability Assessment

  • Establish the link between software testing and reliability

  • Study the effect of code coverage to fault coverage

  • Evaluate impact of reliability by various testing metrics

  • Assess competing testing schemes quantitatively


Positive vs negative evidences for coverage based software testing

Positive vs. negative evidences for coverage-based software testing


Rsdimu test cases description

I

II

III

IV

V

VI

RSDIMU test cases description


The correlation various test regions

The correlation: various test regions

  • Linear modeling fitness in various test case regions

  • Linear regression relationship between block coverage and fault coverage in the whole test set

Fault Coverage


The correlation normal operational testing vs exceptional testing

The correlation: normal operational testing vs. exceptional testing

  • Normal operational testing

    • very weak correlation

  • Exceptional testing

    • strong correlation


The correlation normal operational testing vs exceptional testing1

The correlation: normal operational testing vs. exceptional testing

  • Normal testing: small coverage range (48%-52%)

  • Exceptional testing: two main clusters

Fault Coverage

Fault Coverage


The spectrum in software testing and reliability

Coverage

Based

Testing

Time

Based

Models

The Spectrum in Software Testing and Reliability

- user oriented- tester oriented

- more physical meaning - less physical meaning

- abundant models- lack of models

- easy data collection- hard data collection

- less relevance to testing- more relevance to testing

New Model

Software Reliability

Growth Models

Coverage-Based

Analysis

  • A new model is needed to combine execution time and testing coverage


A new coverage based reliability model

Dependency factors

A New Coverage-Based Reliability Model

  • λ(t,c): joint failure intensity function

  • λ1(t): failure intensity function with respect to time

  • λ2(c): failure intensity functionwith respect to coverage

  • α1,γ1, α2, γ2: parameters with the constraint of

    α 1 + α2 = 1

joint failure intensity function

failure intensity function with time

failure intensity function with coverage


Software reliability engineering a roadmap

Estimation Accuracy


Future direc t ion 4 metrics for reliability prediction

Future Direction 4: Metrics for Reliability Prediction

  • New models (e.g., BBN) to explore rich software metrics

  • Data mining approaches

  • Machine learning techniques

  • Bridging the gap of the one-way function: feedback to building reliable software

  • Continuous industrial data collection efforts – demonstration of cost-effectiveness


Future direc t ion 5 reliability for emerging software applications

Future Direction 5: Reliability for Emerging Software Applications

  • “The Internet changes everything”

  • On-demand customizable software

  • Service oriented architecture, composition, integration

  • Customization by middleware – from metadata to metacode

  • A common infrastructure delivers reliability to all customers


A paradigm for reliable web service

Replication Manager

6.Invoke Web service

Web Service

Web service

selection

algorithm

  • Create Web services

  • Select primary Web

  • service (PWS)

Web Service

Web Service

IIS

Application

IIS

IIS

Database

WatchDog

Application

Application

Database

Database

  • Keep check the availability of the PWS

  • If PWS failed, reselect the PWS.

Client

3.Register

9.Update the WSDL

Port

Application

UDDI

Database

Registry

4. Look up

WSDL

5. Get WSDL

A Paradigm for Reliable Web Service


Conclusions

Conclusions

  • Software reliability is receiving higher attention as it becomes an important economic consideration for businesses.

  • New SRE paradigms need to consider software architectures, testing techniques, data analyses, and creditable reliability modeling procedures.

  • Domain specific approaches on emerging software applications are worthy of investigation.

  • Still a long way to go, but the directions are clear.


  • Login