Software architecture based performance and reliability evaluation
Download
1 / 21

Software Architecture based Performance and Reliability Evaluation - PowerPoint PPT Presentation


  • 301 Views
  • Updated On :

Software Architecture based Performance and Reliability Evaluation Vibhu S. Sharma Ph.D. Scholar CSE, IITk Client 1 Client 2 Web-server Application Server DB Client nc A web-based software system Software Architecture

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Software Architecture based Performance and Reliability Evaluation' - Antony


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Software architecture based performance and reliability evaluation l.jpg

Software Architecture based Performance and Reliability Evaluation

Vibhu S. Sharma

Ph.D. Scholar

CSE, IITk


Software architecture l.jpg

Client 1 Evaluation

Client 2

Web-server

Application Server

DB

Client nc

A web-based software system

Software Architecture

  • Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution. - IEEE 1471-2000.

  • Software architecture styles are like templates which are followed by commonly used systems:

    • Layered style : Web-based systems

    • Pipe and Filter style : Compilers

    • . . .

IIT Kanpur


Modeling the workflow of the software architecture l.jpg

1-p Evaluation12

1

p12

2

p23

3

1

C

1-p23

ESA Antenna array configuration tool architecture

S0

p12

p23

p(nl-2)(nl-1)

p(nl-1)nl

1

S1

S2

. . . . . . . . . .

Snl-1

Snl

1-p12

1-p23

1

1- p(nl-1)nl

Snl+1

Snl+2

. . . . . . . . . .

S2nl-1

S2nl

1

1

1

1

1

S2nl+1

1

Modeling the workflow of the Software Architecture

  • We use finite absorbing Discrete Time Markov Chains (DTMC) for representing the software architecture workflow.

  • Examples:

  • These DTMCs can be used to find the average visit counts to various components for a typical request as well as the limiting probabilities.

  • Very useful for reliability as well as performance prediction.

A typical layered architecture

IIT Kanpur


Architecture based reliability prediction l.jpg

1 Evaluation

pf1

(1-pf1)p12

3

pf3

2

pf2

F

4

pf4

5

pf5

S

Architecture based Reliability prediction

  • Issues/Questions:

    • What is the reliability of a software system composed of unreliable components?

    • How does increasing the reliability of a particular component affects overall reliability ?

    • How does changing the architecture affect reliability ?

  • Reliability modeling using DTMCs

    • Model the component failure as a state in the DTMC and associate probabilities of transition (unreliabilities).

    • Overall reliability is the probability of ending in the correct state.

    • Another option is using a reward based method

IIT Kanpur


Architecture based performance prediction l.jpg

S Evaluation0

p12

p23

p(nl-2)(nl-1)

p(nl-1)nl

1

S1

S2

. . . . . . . . . .

Snl-1

Snl

1-p12

1-p23

1

1- p(nl-1)nl

Snl+1

Snl+2

. . . . . . . . . .

S2nl-1

S2nl

1

1

1

1

1

S2nl+1

1

Architecture based performance prediction

  • Issues/Questions:

    • What is the maximum number of clients the system can able to handle before it saturates ?

    • What effect does varying the number of clients have on the throughput and the average response time ?

    • How should software components be allocated to the hardware nodes ?

    • What changes should be made in the system to improve performance ?

  • We take an example of layered software architecture.

    • A Discrete Time Markov Chain (DTMC) is used to characterize the control flow in the layered software architecture.

    • The DTMC is used to calculate the visit counts to various layers and then a closed queueing network (QN) model is constructed and solved.

IIT Kanpur


Some analysis results l.jpg
Some Analysis Results Evaluation

  • Average response time and throughput and bottleneck analysis

  • Effect of change in individual reliabilities & workload on overall reliability

IIT Kanpur


Reliability and performance tradeoff l.jpg
Reliability and Performance Tradeoff Evaluation

  • Given a set of components and a set of machines to deploy these onto

    • What are the performance & reliability characteristics of the setup under a given load ?

    • Which architecture should be chosen for optimal performance and reliability ?

    • What is the tradeoff ?

  • Software failures of different types cause different effects on the performance of the system.

  • Hardware availability also needs to be taken into account.

  • Some configurations will result in high throughput with decreased reliability and vice-versa.

IIT Kanpur


Combining static analysis and testing frameworks to find software defects l.jpg

Combining Static Analysis and Testing Frameworks Evaluationto find Software Defects

Vipindeep V


Defect prevention during software development l.jpg

Write Code Evaluation

Testing

Compile/Build

Customer

Evaluate

Defect Prevention during Software Development

Static analysis

Test Suite

Program

Specification

IIT Kanpur


Testing static analysis how l.jpg
Testing + Static Analysis – How? Evaluation

  • Static Analysis Vs Testing

    • Different areas altogether, but motive – identify defects

    • Combining the approaches… improved program checking

  • Motivations for the Proposed Approach

    • Static analysis concerns

    • Testing is done.. Bugs are revealed

    • Ideal test suite => identify all bugs

    • Non-ideal scenario of Testing

      • Identify defects, partially

      • Some paths/blocks are “not (properly) tested” .. Why??

    • Static analysis effort on unexplored parts of the code

  • Our proposed approach

    • Focused Static Analysis with Path Pruning using Coverage Data

IIT Kanpur


Overview of the approach l.jpg
Overview of the approach Evaluation

IIT Kanpur


Pros and cons l.jpg
Pros and Cons Evaluation

  • Advantages:

    • New real errors can be identified which escaped in _all_paths analysis

    • Rigorous testing effort counted for better static analysis

    • Reduce execution time

    • Reduced warnings => Reduced Noise

  • Disadvantages:

    • Some errors escape in the pruned path analysis

    • Depends on testing framework

IIT Kanpur


Slide13 l.jpg

Preliminary Results Evaluation

  • Applied the approach with FindBugs, Jlint for JAVA

  • Results from testing were encouraging

  • Tested the applicability using Random Path Selection

IIT Kanpur


Conclusions l.jpg
Conclusions Evaluation

  • Works well in practice

    • Finds enough real defects to be useful

    • Noise is low enough that people can use it

    • Scales well, so works on large very large code bases

  • More issues to explore..

    • Better algorithm to use testing data

    • Path profiles, Crash dumps – focused static analysis

    • Use static analysis results for focusing the testing effort

    • Combining static and dynamic analysis

    • Usability related issues

IIT Kanpur


State based testing automation l.jpg

State Based Testing Automation Evaluation

Atul Gupta

Ph.D. Scholar


State based testing l.jpg
State Based Testing Evaluation

  • Program behavior can be modeled as a state graph

  • A formal testing approach at all levels-unit, component or system level.

  • Effective testing strategy to O-O software

  • Test Cases and Test Oracles can be automatically generated and evaluated

Atul Gupta PhD Scholar

IIT Kanpur


An example a coinbox class l.jpg

vend( ) Evaluation

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

Q1

Q2

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

An Example : A CoinBox Class

  • Class CoinBox {

  • unsigned totalQtrs;

  • unsigned curQtrs;

  • unsigned allowVend;

  • public:

  • CoinBox( ) {

    • totalQtrs = 0;

    • curQtrs = 0;

    • allowVend = 0;

  • }

  • void retQtrs( ) {

  • curQtrs = 0;

  • }

  • void addQtr( ) {

  • curQtrs = curQtrs +1;

  • if (curQtrs > 1)

  • allowVend = 1;

  • }

  • void vend( ) {

  • if (allowVend) {

  • totalQtrs = totalQtrs + curQtrs;

  • curQtrs = 0;

  • allowVend = 0;

  • }

  • }

  • };

IIT Kanpur


State based testing criteria l.jpg

All-Transition Coverage (AT) Evaluation

All-Transition-pair Coverage (ATP)

Full Predicate coverage (FP)

Transition-Tree Coverage (TT)

All-Round-Trip path coverage (ART)

Complete Sequence (CS)

vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

Q1

Q2

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

State Based Testing Criteria

IIT Kanpur


Generating test sequences l.jpg

vend( ) Evaluation

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

WAIT

WAIT

WAIT

WAIT

Q1

Q1

Q2

Q2

Q2

addQtr( )

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

addQtr( )

retQtrs( )

addQtr( )

retQtrs( )

vend( )

Generating Test Sequences

IIT Kanpur


Research issues l.jpg
Research Issues Evaluation

  • How to generate test cases automatically from a state model

  • How to execute them in an automated manner

  • How to evaluate the results automatically

IIT Kanpur


Our approach l.jpg
Our Approach Evaluation

  • Identify State Model of a Class

  • Generate test sequences based on some Coverage Criteria

  • Convert these sequences into JUnit format for Automatic Execution and Result evaluation

    AIM : Complete end-to-end Automation

IIT Kanpur


ad