software architecture based performance and reliability evaluation
Download
Skip this Video
Download Presentation
Software Architecture based Performance and Reliability Evaluation

Loading in 2 Seconds...

play fullscreen
1 / 21

Software Architecture based Performance and Reliability Evaluation - PowerPoint PPT Presentation


  • 307 Views
  • Uploaded on

Software Architecture based Performance and Reliability Evaluation Vibhu S. Sharma Ph.D. Scholar CSE, IITk Client 1 Client 2 Web-server Application Server DB Client nc A web-based software system Software Architecture

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Software Architecture based Performance and Reliability Evaluation' - Antony


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
software architecture based performance and reliability evaluation

Software Architecture based Performance and Reliability Evaluation

Vibhu S. Sharma

Ph.D. Scholar

CSE, IITk

software architecture
Client 1

Client 2

Web-server

Application Server

DB

Client nc

A web-based software system

Software Architecture
  • Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution. - IEEE 1471-2000.
  • Software architecture styles are like templates which are followed by commonly used systems:
    • Layered style : Web-based systems
    • Pipe and Filter style : Compilers
    • . . .

IIT Kanpur

modeling the workflow of the software architecture
1-p12

1

p12

2

p23

3

1

C

1-p23

ESA Antenna array configuration tool architecture

S0

p12

p23

p(nl-2)(nl-1)

p(nl-1)nl

1

S1

S2

. . . . . . . . . .

Snl-1

Snl

1-p12

1-p23

1

1- p(nl-1)nl

Snl+1

Snl+2

. . . . . . . . . .

S2nl-1

S2nl

1

1

1

1

1

S2nl+1

1

Modeling the workflow of the Software Architecture
  • We use finite absorbing Discrete Time Markov Chains (DTMC) for representing the software architecture workflow.
  • Examples:
  • These DTMCs can be used to find the average visit counts to various components for a typical request as well as the limiting probabilities.
  • Very useful for reliability as well as performance prediction.

A typical layered architecture

IIT Kanpur

architecture based reliability prediction
1

pf1

(1-pf1)p12

3

pf3

2

pf2

F

4

pf4

5

pf5

S

Architecture based Reliability prediction
  • Issues/Questions:
    • What is the reliability of a software system composed of unreliable components?
    • How does increasing the reliability of a particular component affects overall reliability ?
    • How does changing the architecture affect reliability ?
  • Reliability modeling using DTMCs
    • Model the component failure as a state in the DTMC and associate probabilities of transition (unreliabilities).
    • Overall reliability is the probability of ending in the correct state.
    • Another option is using a reward based method

IIT Kanpur

architecture based performance prediction
S0

p12

p23

p(nl-2)(nl-1)

p(nl-1)nl

1

S1

S2

. . . . . . . . . .

Snl-1

Snl

1-p12

1-p23

1

1- p(nl-1)nl

Snl+1

Snl+2

. . . . . . . . . .

S2nl-1

S2nl

1

1

1

1

1

S2nl+1

1

Architecture based performance prediction
  • Issues/Questions:
    • What is the maximum number of clients the system can able to handle before it saturates ?
    • What effect does varying the number of clients have on the throughput and the average response time ?
    • How should software components be allocated to the hardware nodes ?
    • What changes should be made in the system to improve performance ?
  • We take an example of layered software architecture.
    • A Discrete Time Markov Chain (DTMC) is used to characterize the control flow in the layered software architecture.
    • The DTMC is used to calculate the visit counts to various layers and then a closed queueing network (QN) model is constructed and solved.

IIT Kanpur

some analysis results
Some Analysis Results
  • Average response time and throughput and bottleneck analysis
  • Effect of change in individual reliabilities & workload on overall reliability

IIT Kanpur

reliability and performance tradeoff
Reliability and Performance Tradeoff
  • Given a set of components and a set of machines to deploy these onto
    • What are the performance & reliability characteristics of the setup under a given load ?
    • Which architecture should be chosen for optimal performance and reliability ?
    • What is the tradeoff ?
  • Software failures of different types cause different effects on the performance of the system.
  • Hardware availability also needs to be taken into account.
  • Some configurations will result in high throughput with decreased reliability and vice-versa.

IIT Kanpur

defect prevention during software development
Write Code

Testing

Compile/Build

Customer

Evaluate

Defect Prevention during Software Development

Static analysis

Test Suite

Program

Specification

IIT Kanpur

testing static analysis how
Testing + Static Analysis – How?
  • Static Analysis Vs Testing
    • Different areas altogether, but motive – identify defects
    • Combining the approaches… improved program checking
  • Motivations for the Proposed Approach
    • Static analysis concerns
    • Testing is done.. Bugs are revealed
    • Ideal test suite => identify all bugs
    • Non-ideal scenario of Testing
      • Identify defects, partially
      • Some paths/blocks are “not (properly) tested” .. Why??
    • Static analysis effort on unexplored parts of the code
  • Our proposed approach
    • Focused Static Analysis with Path Pruning using Coverage Data

IIT Kanpur

pros and cons
Pros and Cons
  • Advantages:
    • New real errors can be identified which escaped in _all_paths analysis
    • Rigorous testing effort counted for better static analysis
    • Reduce execution time
    • Reduced warnings => Reduced Noise
  • Disadvantages:
    • Some errors escape in the pruned path analysis
    • Depends on testing framework

IIT Kanpur

slide13
Preliminary Results
  • Applied the approach with FindBugs, Jlint for JAVA
  • Results from testing were encouraging
  • Tested the applicability using Random Path Selection

IIT Kanpur

conclusions
Conclusions
  • Works well in practice
    • Finds enough real defects to be useful
    • Noise is low enough that people can use it
    • Scales well, so works on large very large code bases
  • More issues to explore..
    • Better algorithm to use testing data
    • Path profiles, Crash dumps – focused static analysis
    • Use static analysis results for focusing the testing effort
    • Combining static and dynamic analysis
    • Usability related issues

IIT Kanpur

state based testing automation

State Based Testing Automation

Atul Gupta

Ph.D. Scholar

state based testing
State Based Testing
  • Program behavior can be modeled as a state graph
  • A formal testing approach at all levels-unit, component or system level.
  • Effective testing strategy to O-O software
  • Test Cases and Test Oracles can be automatically generated and evaluated

Atul Gupta PhD Scholar

IIT Kanpur

an example a coinbox class
vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

Q1

Q2

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

An Example : A CoinBox Class
  • Class CoinBox {
  • unsigned totalQtrs;
  • unsigned curQtrs;
  • unsigned allowVend;
  • public:
  • CoinBox( ) {
    • totalQtrs = 0;
    • curQtrs = 0;
    • allowVend = 0;
  • }
  • void retQtrs( ) {
  • curQtrs = 0;
  • }
  • void addQtr( ) {
  • curQtrs = curQtrs +1;
  • if (curQtrs > 1)
  • allowVend = 1;
  • }
  • void vend( ) {
  • if (allowVend) {
  • totalQtrs = totalQtrs + curQtrs;
  • curQtrs = 0;
  • allowVend = 0;
  • }
  • }
  • };

IIT Kanpur

state based testing criteria
All-Transition Coverage (AT)

All-Transition-pair Coverage (ATP)

Full Predicate coverage (FP)

Transition-Tree Coverage (TT)

All-Round-Trip path coverage (ART)

Complete Sequence (CS)

vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

Q1

Q2

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

State Based Testing Criteria

IIT Kanpur

generating test sequences
vend( )

allowVend = 0

curQtrs = 1

allowVend = 0

curQtrs = 0

addQtr( )

addQtr( )

addQtr( )

WAIT

WAIT

WAIT

WAIT

WAIT

Q1

Q1

Q2

Q2

Q2

addQtr( )

retQtrs( )

allowVend = 1

curQtrs >= 2

retQtrs( )

addQtr( )

retQtrs( )

addQtr( )

retQtrs( )

vend( )

Generating Test Sequences

IIT Kanpur

research issues
Research Issues
  • How to generate test cases automatically from a state model
  • How to execute them in an automated manner
  • How to evaluate the results automatically

IIT Kanpur

our approach
Our Approach
  • Identify State Model of a Class
  • Generate test sequences based on some Coverage Criteria
  • Convert these sequences into JUnit format for Automatic Execution and Result evaluation

AIM : Complete end-to-end Automation

IIT Kanpur

ad