Software architecture based performance and reliability evaluation
Download
1 / 21

Software Architecture based Performance and Reliability Evaluation - PowerPoint PPT Presentation


Software Architecture based Performance and Reliability Evaluation Vibhu S. Sharma Ph.D. Scholar CSE, IITk Client 1 Client 2 Web-server Application Server DB Client nc A web-based software system Software Architecture

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Software Architecture based Performance and Reliability Evaluation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Software Architecture based Performance and Reliability Evaluation

Vibhu S. Sharma

Ph.D. Scholar

CSE, IITk


Client 1

Client 2

Web-server

Application Server

DB

Client nc

A web-based software system

Software Architecture

  • Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution. - IEEE 1471-2000.

  • Software architecture styles are like templates which are followed by commonly used systems:

    • Layered style : Web-based systems

    • Pipe and Filter style : Compilers

    • . . .

IIT Kanpur


1-p12

1

p12

2

p23

3

1

C

1-p23

ESA Antenna array configuration tool architecture

S0

p12

p23

p(nl-2)(nl-1)

p(nl-1)nl

1

S1

S2

. . . . . . . . . .

Snl-1

Snl

1-p12

1-p23

1

1- p(nl-1)nl

Snl+1

Snl+2

. . . . . . . . . .

S2nl-1

S2nl

1

1

1

1

1

S2nl+1

1

Modeling the workflow of the Software Architecture

  • We use finite absorbing Discrete Time Markov Chains (DTMC) for representing the software architecture workflow.

  • Examples:

  • These DTMCs can be used to find the average visit counts to various components for a typical request as well as the limiting probabilities.

  • Very useful for reliability as well as performance prediction.

A typical layered architecture

IIT Kanpur


1

pf1

(1-pf1)p12

3

pf3

2

pf2

F

4

pf4

5

pf5

S

Architecture based Reliability prediction

  • Issues/Questions:

    • What is the reliability of a software system composed of unreliable components?

    • How does increasing the reliability of a particular component affects overall reliability ?

    • How does changing the architecture affect reliability ?

  • Reliability modeling using DTMCs

    • Model the component failure as a state in the DTMC and associate probabilities of transition (unreliabilities).

    • Overall reliability is the probability of ending in the correct state.

    • Another option is using a reward based method

IIT Kanpur


S0

p12

p23

p(nl-2)(nl-1)

p(nl-1)nl

1

S1

S2

. . . . . . . . . .

Snl-1

Snl

1-p12

1-p23

1

1- p(nl-1)nl

Snl+1

Snl+2

. . . . . . . . . .

S2nl-1

S2nl

1

1

1

1

1

S2nl+1

1

Architecture based performance prediction

  • Issues/Questions:

    • What is the maximum number of clients the system can able to handle before it saturates ?

    • What effect does varying the number of clients have on the throughput and the average response time ?

    • How should software components be allocated to the hardware nodes ?

    • What changes should be made in the system to improve performance ?

  • We take an example of layered software architecture.

    • A Discrete Time Markov Chain (DTMC) is used to characterize the control flow in the layered software architecture.

    • The DTMC is used to calculate the visit counts to various layers and then a closed queueing network (QN) model is constructed and solved.

IIT Kanpur


Some Analysis Results

  • Average response time and throughput and bottleneck analysis

  • Effect of change in individual reliabilities & workload on overall reliability

IIT Kanpur


Reliability and Performance Tradeoff

  • Given a set of components and a set of machines to deploy these onto

    • What are the performance & reliability characteristics of the setup under a given load ?

    • Which architecture should be chosen for optimal performance and reliability ?

    • What is the tradeoff ?

  • Software failures of different types cause different effects on the performance of the system.

  • Hardware availability also needs to be taken into account.

  • Some configurations will result in high throughput with decreased reliability and vice-versa.

IIT Kanpur


Combining Static Analysis and Testing Frameworks to find Software Defects

Vipindeep V


Write Code

Testing

Compile/Build

Customer

Evaluate

Defect Prevention during Software Development

Static analysis

Test Suite

Program

Specification

IIT Kanpur


Testing + Static Analysis – How?

  • Static Analysis Vs Testing

    • Different areas altogether, but motive – identify defects

    • Combining the approaches… improved program checking

  • Motivations for the Proposed Approach

    • Static analysis concerns

    • Testing is done.. Bugs are revealed

    • Ideal test suite => identify all bugs

    • Non-ideal scenario of Testing

      • Identify defects, partially

      • Some paths/blocks are “not (properly) tested” .. Why??

    • Static analysis effort on unexplored parts of the code

  • Our proposed approach

    • Focused Static Analysis with Path Pruning using Coverage Data

IIT Kanpur


Overview of the approach

IIT Kanpur


Pros and Cons

  • Advantages:

    • New real errors can be identified which escaped in _all_paths analysis

    • Rigorous testing effort counted for better static analysis

    • Reduce execution time

    • Reduced warnings => Reduced Noise

  • Disadvantages:

    • Some errors escape in the pruned path analysis

    • Depends on testing framework

IIT Kanpur


Preliminary Results

  • Applied the approach with FindBugs, Jlint for JAVA

  • Results from testing were encouraging

  • Tested the applicability using Random Path Selection

IIT Kanpur


Conclusions

  • Works well in practice

    • Finds enough real defects to be useful

    • Noise is low enough that people can use it

    • Scales well, so works on large very large code bases

  • More issues to explore..

    • Better algorithm to use testing data

    • Path profiles, Crash dumps – focused static analysis

    • Use static analysis results for focusing the testing effort

    • Combining static and dynamic analysis

    • Usability related issues

IIT Kanpur


State Based Testing Automation

Atul Gupta

Ph.D. Scholar


State Based Testing

  • Program behavior can be modeled as a state graph

  • A formal testing approach at all levels-unit, component or system level.

  • Effective testing strategy to O-O software

  • Test Cases and Test Oracles can be automatically generated and evaluated

Atul Gupta PhD Scholar

IIT Kanpur


vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

Q1

Q2

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

An Example : A CoinBox Class

  • Class CoinBox {

  • unsigned totalQtrs;

  • unsigned curQtrs;

  • unsigned allowVend;

  • public:

  • CoinBox( ) {

    • totalQtrs = 0;

    • curQtrs = 0;

    • allowVend = 0;

  • }

  • void retQtrs( ) {

  • curQtrs = 0;

  • }

  • void addQtr( ) {

  • curQtrs = curQtrs +1;

  • if (curQtrs > 1)

  • allowVend = 1;

  • }

  • void vend( ) {

  • if (allowVend) {

  • totalQtrs = totalQtrs + curQtrs;

  • curQtrs = 0;

  • allowVend = 0;

  • }

  • }

  • };

IIT Kanpur


All-Transition Coverage (AT)

All-Transition-pair Coverage (ATP)

Full Predicate coverage (FP)

Transition-Tree Coverage (TT)

All-Round-Trip path coverage (ART)

Complete Sequence (CS)

vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

Q1

Q2

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

State Based Testing Criteria

IIT Kanpur


vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

WAIT

WAIT

WAIT

WAIT

Q1

Q1

Q2

Q2

Q2

addQtr( )

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

addQtr( )

retQtrs( )

addQtr( )

retQtrs( )

vend( )

Generating Test Sequences

IIT Kanpur


Research Issues

  • How to generate test cases automatically from a state model

  • How to execute them in an automated manner

  • How to evaluate the results automatically

IIT Kanpur


Our Approach

  • Identify State Model of a Class

  • Generate test sequences based on some Coverage Criteria

  • Convert these sequences into JUnit format for Automatic Execution and Result evaluation

    AIM : Complete end-to-end Automation

IIT Kanpur


ad
  • Login